Deploy a Swarm Cluster with Alexa

Serverless and Containers changed the way we leverage public clouds and how we write, deploy and maintain applications. A great way to combine the two paradigms is to build a voice assistant with Alexa based on Lambda functions – written in Go – to deploy a Docker Swarm cluster on AWS.

The figure below shows all components needed to deploy a production-ready Swarm cluster on AWS with Alexa.

Note: Full code is available on my GitHub.

A user will ask Amazon Echo to deploy a Swarm Cluster:

Echo will intercept the user’s voice command with built-in natural language understanding and speech recognition. Convey them to the Alexa service. A custom Alexa skill will convert the voice commands to intents:

The Alexa skill will trigger a Lambda function for intent fulfilment:

The Lambda Function will use the AWS EC2 API to deploy a fleet of EC2 instances from an AMI with Docker CE preinstalled (I used Packer to bake the AMI to reduce the cold-start of the instances). Then, push the cluster IP addresses to a SQS:

Next, the function will insert a new item to a DynamoDB table with the current state of the cluster:

Once the SQS received the message, a CloudWatch alarm (it monitors the ApproximateNumberOfMessagesVisible parameter) will be triggered and as a result it will publish a message to an SNS topic:

The SNS topic triggers a subscribed Lambda function:

The Lambda function will pull the queue for a new cluster and use the AWS System Manager API to provision a Swarm cluster on the fleet of EC2 instances created earlier:

For debugging, the function will output the Swarm Token to CloudWatch:

Finally, it will update the DynamoDB item state from Pending to Done and delete the message from SQS.

You can test your skill on your Amazon Echo, Echo Dot, or any Alexa device by saying, “Alexa, open Docker

At the end of the workflow described above, a Swarm cluster will be created:

At this point you can see your Swarm status by firing the following command as shown below:

Improvements & Limitations:

  • Lambda execution timeout if the cluster size is huge. You can use a Master Lambda function to spawn child Lambda.
  • CloudWatch & SNS parts can be deleted if SQS is supported as Lambda event source (AWS PLEAAASE !). DynamoDB streams or Kinesis streams cannot be used to notify Lambda as I wanted to create some kind of delay for the instances to be fully created before setting up the Swarm cluster. (maybe Simple Workflow Service ?)
  • Inject SNS before SQS. SNS can add the message to SQS and trigger the Lambda function. We won’t need CloudWatch Alarm.
  • You can improve the Skill by adding new custom intents to deploy Docker containers on the cluster or ask Alexa to deploy the cluster on a VPC

In-depth details about the skill can be found on my GitHub. Make sure to drop your comments, feedback, or suggestions below — or connect with me directly on Twitter @mlabouardy.

Highly Available WordPress Blog

In this post you will learn about the easiest way to deploy a fault tolerant and scalable WordPress on AWS.

To get started, setup a Swarm cluster on AWS by following this tutorial Setup Docker Swarm on AWS using Ansible & Terraform:

Now your cluster is ready to use. You are ready to go !

WordPress stores some files on disk (plugins, themes, images …) which causes a problem if you want to use a fleet of EC2 instances to run your blog in case of high traffic:

That’s where AWS EFS (Elastic File System) comes into the play. The idea is to mount shared volumes using the NFS protocol in each host to synchronize files between all nodes in the cluster.

So create an Elastic File System, make sure to deploy it in the same VPC on which your Swarm cluster is created:

Once created, note the DNS name:

Now, mount Amazon EFS file systems via the NFSv4.1 protocol on each node:

We can verify the mount with a plain df -h command:

WordPress requires a relational database. Create an Amazon Aurora database:

Wait couple of minutes, then the database should be ready, copy the endpoint of database:

To deploy the stack, I’m using the following Docker Compose file:

In addition to wordpress container, Im using Traefik as reverse proxy to be able to scale out my blog easily with docker service scale command.

In your Manager node run the following command to deploy the stack:

At this point, you should have a clean install of WordPress running.

Fire up your browser and point it to manager public IP address, you will be greeted with the familiar WordPress setup page:

If you’re expecting a high traffic, you can easily scale the WP service using the command:

Verify Traefik Dashboard:

That’s how to build a scalable WordPress blog with no single points of failure.

Highly Available Docker Registry on AWS with Nexus

Have you ever wondered how you can build a highly available & resilient Docker Repository to store your Docker Images ?

Résultat de recherche d'images pour "you came to the right place meme"

In this post, we will setup an EC2 instance inside a Security Group and create an A record pointing to the server Elastic IP address as follow:

To provision the infrastructure, we will use Terraform as IaC (Infrastructure as Code) tool. The advantage of using this kind of tools is the ability to spin up a new environment quickly in different AWS region (or different IaaS provider) in case of incident (Disaster recovery).

Start by cloning the following Github repository:

Inside docker-registry folder, update the variables.tfvars with your own AWS credentials (make sure you have the right IAM policies).

I specified a shell script to be used as user_data when launching the instance. It will simply install the latest version of Docker CE and turn the instance to Docker Swarm Mode (to benefit from replication & high availability of Nexus container)

Note: Surely, you can use a Configuration Management Tools like Ansible or Chef to provision the server once created.

Then, issue the following command to create the infrastructure:

Once created, you should see the Elastic IP of your instance:

Connect to your instance via SSH:

Verify that the Docker Engine is running in Swarm Mode:

Check if Nexus service is running:

If you go back to your AWS Management Console. Then, navigate to Route53 Dashboard, you should see a new A record has been created which points to the instance IP address.

Point your favorite browser to the Nexus Dashboard URL (registry.slowcoder.com:8081). Login and create a Docker hosted registry as below:

Edit the /etc/docker/daemon.json file, it should have the following content:

Note: For production it’s highly recommended to secure your registry using a TLS certificate issued by a known CA.

Restart Docker for the changes to take effect:

Login to your registry with Nexus Credentials (admin/admin123):

In order to push a new image to the registry:

Verify that the image has been pushed to the remote repository:

To pull the Docker image:

Note: Sometimes you end up with many unused & dangling images that can quickly take significant amount of disk space:

You can either use the Nexus CLI tool or create a Nexus Task to cleanup old Docker Images:

Populate the form as below:

The task above will run everyday at midnight to purge unused docker images from “mlabouardy” registry.

Continuous Monitoring with TICK stack

Monitoring your system is required. It helps you detect any issues before they cause any major downtime that effect your customers and damage your business reputation. It helps you also to plan growth based on the real usage of your system. But collecting metrics from different data sources isn’t enough, you need to personalize your monitoring to meet your own business needs and define the right alerts so that any abnormal changes in the system will reported.

In this post, I will show you how to setup a resilient continuous monitoring platform with only open source projects & how to define an event alert to report changes in the system.

Clone the following Github repository:

1 – Terraform & AWS

In the tick-stack/terraform directory, update the variables.tfvars file with your own AWS credentials (make sure you have the right IAM policies) :

Issue the following command to download the AWS provider plugin:

Issue the following command to provision the infrastructure:

2 – Ansible & Docker

Update the inventory file with your instance DNS name:

Then, install the Ansible custom role:

Execute the Ansible Playbook:

Point your browser to http://DNS_NAME:8083, you should see InfluxDB Admin Dashboard:

Now, create an InfluxDB Data Source in Chronograf (http://DNS_NAME:8888):

Create a new Dashboard as follow:

You can create multiple graphs to visualize different types of metrics:

Note: For in depth details on how to create interactive & dynamic dashboards in Chronograf check my previous tutorial.

You need to elaborate on the data collected to do something like alerting. So make sure to enable Kapacitor:

Define a new alert to send a Slack notification if the CPU utilization is higher than 70%.

To test it out, we need to generate some workload. For this case, I used stress:

Stressing the CPU:

After few seconds, you should receive a Slack notification.

Exploring Swarm & Container Overview Dashboard in Grafana

In my previous post, your learnt how to monitor your Swarm Cluster with TICK Stack. In this part, I will show you how to use the same Stack but instead of using Chronograf as our visualization and exploration tool we will use Grafana.

Connect to your manager node via SSH, and clone the following Github repository:

Use the docker-compose.yml below to setup the monitoring stack:

Then, issue the following command to deploy the stack:

Once deployed, you should see the list of services running on the cluster:

Point your browser to http://IP:3000, you should be able to reach the Grafana Dashboard:

The default username & password are admin. Go ahead and log in.

Go to “Data Sources” and create 2 InfluxDB data sources:

  • Vms: pointing to your Cluster Nodes metrics database.
  • Docker: pointing to your Docker Services metrics database.

Finally, import the dashboard by hitting the “import” button:

From here, you can upload the dashboard.json, then pick the data sources you created earlier:

You will end up with an interactive and dynamic dashboard: