Highly Available Docker Registry on AWS with Nexus

Have you ever wondered how you can build a highly available & resilient Docker Repository to store your Docker Images ?



In this post, we will setup an EC2 instance inside a Security Group and create an A record pointing to the server Elastic IP address as follow:



To provision the infrastructure, we will use Terraform as IaC (Infrastructure as Code) tool. The advantage of using this kind of tools is the ability to spin up a new environment quickly in different AWS region (or different IaaS provider) in case of incident (Disaster recovery).

Start by cloning the following Github repository:

1
git clone https://github.com/mlabouardy/terraform-aws-labs.git

Inside docker-registry folder, update the variables.tfvars with your own AWS credentials (make sure you have the right IAM policies).

1
2
3
4
5
6
7
8
9
10
11
12
resource "aws_instance" "default" {
ami = "${lookup(var.amis, var.region)}"
instance_type = "${var.instance_type}"
key_name = "${aws_key_pair.default.id}"
security_groups = ["${aws_security_group.default.name}"]

user_data = "${file("setup.sh")}"

tags {
Name = "registry"
}
}

I specified a shell script to be used as user_data when launching the instance. It will simply install the latest version of Docker CE and turn the instance to Docker Swarm Mode (to benefit from replication & high availability of Nexus container)

1
2
3
4
5
6
7
#!/bin/sh
yum update -y
yum install -y docker
service docker start
usermod -aG docker ec2-user
docker swarm init
docker service create --replicas 1 --name registry --publish 5000:5000 --publish 8081:8081 sonatype/nexus3:3.6.2

Note: Surely, you can use a Configuration Management Tools like Ansible or Chef to provision the server once created.

Then, issue the following command to create the infrastructure:

1
terraform apply -var-file=variables.tfvars

Once created, you should see the Elastic IP of your instance:



Connect to your instance via SSH:

1
ssh ec2-user@35.177.167.36

Verify that the Docker Engine is running in Swarm Mode:



Check if Nexus service is running:



If you go back to your AWS Management Console. Then, navigate to Route53 Dashboard, you should see a new A record has been created which points to the instance IP address.



Point your favorite browser to the Nexus Dashboard URL (registry.slowcoder.com:8081). Login and create a Docker hosted registry as below:



Edit the /etc/docker/daemon.json file, it should have the following content:

1
2
3
{
"insecure-registries" : ["registry.slowcoder.com:5000"]
}

Note: For production it’s highly recommended to secure your registry using a TLS certificate issued by a known CA.

Restart Docker for the changes to take effect:

1
service docker restart

Login to your registry with Nexus Credentials (admin/admin123):



In order to push a new image to the registry:

1
docker push registry.slowcoder.com:5000/mlabouardy/movies-api:1.0.0-beta


Verify that the image has been pushed to the remote repository:



To pull the Docker image:

1
docker pull registry.slowcoder.com:5000/mlabouardy/movies-api:1.0.0-beta


Note: Sometimes you end up with many unused & dangling images that can quickly take significant amount of disk space:



You can either use the Nexus CLI tool or create a Nexus Task to cleanup old Docker Images:



Populate the form as below:



The task above will run everyday at midnight to purge unused docker images from “mlabouardy” registry.

Drop your comments, feedback, or suggestions below — or connect with me directly on Twitter @mlabouardy.

Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now

×