Setting up an etcd cluster on AWS using CoreOS & Terraform

This post is part of “IaC” series explaining how to use Infrastracture as Code concepts with Terraform. In this part, I will show you how to setup an etcd cluster on AWS using CoreOS & Terraform as shown in the diagram below :

All the templates used in this demo can be found on my Github 😁.

So let’s start with “variables.tf” file which contains the global variables such as AWS region, cluster instances type …

Note: As of writing this article, the latest stable CoreOS version is 1465.6.0.

So make sure to find an AMI that is as close to the latest version as possible.

Next, we need to define a security group for our cluster. For simplicity, Im going to make this security group open to the world. Even though security is important, this tutorial serves an educational purposes and you should never have all ports open in production.

And finally, we will define our cluster which consists of 3 Nodes:

In order to bring up an etcd cluster, I used a cloud config file that I passed as a parameter to user_data attribut:

Note: Make sure to grab the discovery token, and place it into the discovery parameter:

Once you defined all templates required, just type the following command to bring up the etcd cluster:

Note: Don’t forget  to set the AWS credentials as an envrionment variables before:

Setting up an etcd cluster in action is shown below 😎 :

Once done, go to your AWS Management Console then navigate to your EC2 Dashboard:

Congratulations ! 🎉🎉 You have your CoreOS cluster.

To verify the cluster health, you can either point your browser to the discovery url you generated earlier:

or SSH to one of your cluster nodes using the command:

Then, use the etcd command line to fetch the cluster status:

Now we have an etcd cluster ready to use. Let’s see what we can do with it:

  • Through etcdctl:

  • Through HTTP API:

 

Manage AWS VPC as Infrastructure as Code with Terraform

In this tutorial, I will show you how to setup a VPC as described in the network diagram below in less than 1 min ⏱ using Terraform:

The VPC topology above is the best demonstration of what will be implemented:

  • The private subnet is inaccessible to the internet (both in and out)
  • The public subnet is accessible and all traffic (0.0.0.0/0) is routed directly to the internet Gateway

Before we dive in, all the code used in this demo is available at my Github.

Note: I already did a tutorial on how to get started with Terraform so make sure to read it for more details.

1 – Global variables

This file contains environment specific configuration like region name, CIDR blocks, and AWS credentials …

2 – Configure the AWS provider

3 – Create a VPC

4 – Create Subnets

To make the public subnet addressable by the Internet, we need an Internet Gateway:

 5 – Internet Gateway

To allow traffics from the public subnet to the internet throught the NAT Gateway, we need to create a new Route Table.

6 – Route Table

Next, we will create a security group for each subnet.

7 – Security Groups

7 .1 – WebServer SG

This Security Group allows HTTP/HTTPS and SSH connections from anywhere.

7.2 – Database SG

This Security Group enable MySQL 3306 port, ping and SSH only from the public subnet.

Now we will deploy the EC2 instances, but before that we need to create a key pair in order to connect later to the instances via SSH.

8 – Key Pair

9 – EC2 Instances

9.1 – WebServer Instance

This instance will play the role of a webserver. Therefore, we pass to the instance userdata a shell script install.sh which contains commands to install an Apache Server:

9.2 – Database Instance

Once you’ve defined all the required templates, make sure to set the AWS credentials variables as an envrionment variables:

Note: You can always use your root user which has access permission to everything, but for security perspective, its recommended to use only a limited permissions user account. So create a new one using AWS IAM.

To see how terraform plans to create the resources type “terraform plan“. To create the infrastructure type “terraform apply“:

That will bring up the VPC, and all the necessary resources. Now in your AWS Management Console you should see the resources created:

If you click on the “Subnets” menu, you should see the public & private subnets:

The same goes for the Route Tables:

And the Internet Gateway:

Security Groups also:

WebServer Security group:

Database Security Group:

And finally the EC2 Instances:

WebServer Instance:

Database Instance:

Don’t forget to destroy the resources if they are not needed by typing “terraform destroy“:

 

Manage AWS Infrastracture as Code with Terraform

This post is part of “IaC” series 😇 to teach you how to use Terraform to spin up infrastructure on AWS.

1 – Why Terraform ?

Terraform is a IaC tool written in Go, it gives you the ability to describe a complex infrastructre using a high level language (HashiCorp Configuration Language  – HCL). This feature allows you to treat your infrastrcture as you would any other code. In addition to that, Terraform provision infrastructure across multiple IaaS providers like: AWS, Azure, GCP, VMware 

2 – Install Terraform

To install Terraform, find the appropriate package for your system and download it:

After installing Terraform, verify the installation worked by executing ‘terraform version‘:

3 – Create AWS IAM User

Terraform uses AWS REST API to manage AWS resources, so we need to setup a new user account in order to use it

Note: We can use our root user which has access permission to everything, but for security perspective, its recommended to use only a limited permissions user account.

So after signin in your AWS Management Console, go to “Identity and Access Management – IAM” section:

Then, click on “Add user” :

Assign a username for the new account and select “Programmatic access“:

Add “AmazonEC2FullAccess” to user permissions list:

Next, click on “Review” then “Create User“:

Copy to clipboard the Access Key ID and Secret Access Key, then set them as envrionment variables by running the commands below

4 – Deploy an EC2 instance

Terraform templates are written in a declarative language called HCL with the extension “.tf“. So create a file called “main.tf” with the following content:

The above template tells Terraform that we will use AWS as a provider and the region on which our resources will be deployed on. It also tells that we will create an EC2 of type t2.micro that uses Amazon Linux AMI as an OS.  You can configure other parameters (The aws_instance resource documentation lists all the parameters it supports)

Note: The ami of the main images can be retrieved from the AWS console after being logged in. Simulate installing an instance by clicking “Lauch Instance” and browse through the main images. The image number starting with “ami” is displayed there:

Before we create the infrastrcture, we need to install the AWS plugin for Terraform by typing ‘terraform init‘ :

To see what Terraform will do before actually doing it. Its recommended to run ‘terraform plan‘ always to check your changes before unleashing them onto the world:

To apply the changes, issue the following command:

If we jump to the AWS Console, we indeed can see that “test-vm” instance has been created:

It works 😍 ! but still there’s a small problem 😐😣 , we cannot connect via SSH to the instance because we didn’t assign a key pair to it.

To add one, we will modify the template file “main.tf” : :

Run the plan command again:

As you can see above, the key_name will be applied to the new instance, allowing us to SSH into the server

Now if we goes back to AWS dashboard, we notice that a new instance has been launched and the correct key is assigned:

The key pair we created can be listed in the “Key Pairs” menu:

Now, we can connect to the server via SSH using the instance public DNS name 😎

5 – Cleanup the instance

To destroy the instance, we type ‘terraform destroy‘ then we confirm by typing “yes”:

6 – Taking this further

So that was a quick demo, on how to create a simple server on AWS, on my upcoming tutorials, I will show you guys how to setup a complex infrastructure on AWS with a VPC, bunch of EC2 instances, ELB, Virtual Gateway and Security Groups & Network ACL configurations. So stay tuned 😉