5 Keys to Running Workloads Resiliently with Docker and Rancher

Containers and orchestration frameworks like Rancher will soon allow every organisation to have access to efficient cluster management. This brave new world free Ops from managing application configuration to focus on automation and allows developers to focus on writing code; containers abstract complex dependency requirements, which allows ops to deploy immutable containerised applications and allows developers a consistent run-time for their code.

If the benefits are so clear, then why do companies with existing infrastructure practices not switch? Well, one of they key issues is risk. The risk of new unknowns brought by an untested technology, the risk of inexperience operating a new stack, and the risk of downtime impacting the brand.

Planning for risks and demonstrating that the Ops team can maintain a resilient workload whilst moving into a containerized world is the key social aspect of a container migration project. Especially since, when done correctly, Docker and Rancher provide a solid framework for quickly iterating on infrastructure improvements. Such as rancher catalogs for quickly spinning up popular distributed applications like ElasticSearch.

In regards to risk management, we will look into identifying the five keys to running a resilient workload on Rancher and Docker. The topics that will be covered are as follows:

At the end of this chain, we hope the reader will have access to completed automation templates and working knowledge to feel comfortable launching their projects on Rancher.

Setting Up A Local Rancher Cluster with Raspberry Pi(s) and an Old Laptop

Experiment Setup

Laptop Setup

For the Rancher Server I setup the following on my Laptop. First lets create a Rancher machine, I used Docker Machine provided by the Docker Toolbox to create my environment. It is the officially recommended way to bootstrap Rancher locally.

docker-machine create -d virtualbox \  
--virtualbox-memory "1024" \
--virtualbox-cpu-count "1" \
--virtualbox-disk-size "8000" \
--engine-storage-driver overlay \
rancher-server  

To make our Rancher Server's IP a bit more deterministic we run some Docker Machine commands so that our Rancher Server will have a static IP address. Run the following script on GitBash.exe for windows, or on your terminal.

$ ./makestatic.sh rancher-server 99 100  # sets our VM IP to 192.168.99.100

makestatic.sh contents:

#!/bin/bash

MACHINE=$1  
# IP address IP1.IP2.IP3.IP4, we leave the front two static 192.168.${IP3}.${IP4}
IP3=$2  
IP4=$3

# the first command kills the the DHCP server on the Rancher VM
docker-machine ssh ${MACHINE} "sudo cat /var/run/udhcpc.eth1.pid | xargs sudo kill"

# the second command configures a static IP for our VM
docker-machine ssh ${MACHINE} "sudo ifconfig eth1 192.168.${IP3}.${IP4} netmask 255.255.255.0 broadcast 192.168.${IP3}.255 up"

# resolve IP address mismatch error
docker-machine regenerate-certs ${MACHINE} -f 

echo "Updating boot2docker/profile"

echo "sudo cat /var/run/udhcpc.eth1.pid | xargs sudo kill" | docker-machine ssh ${MACHINE} "sudo tee -a /var/lib/boot2docker/profile  > /dev/null"  
echo "sudo ifconfig eth1 192.168.${IP3}.${IP4} netmask 255.255.255.0 broadcast 192.168.${IP3}.255 up" | docker-machine ssh ${MACHINE} "sudo tee -a /var/lib/boot2docker/profile > /dev/null"

echo "New boot2docker/profile:"  
echo "====="  
docker-machine ssh ${MACHINE} "cat /var/lib/boot2docker/profile"  
echo "====="  
echo "done"  

Then lets make this machine our default host for the docker command.

eval $(docker-machine env rancher-server)  

Finally, we can spawn a Rancher Server on our docker machine, and we will be able to access it on 192.168.99.100:8080.

docker run -d --restart=always --name=rancher-server -p 8080:8080 rancher/server  

Though this is still a problem for our network setup, since our Raspberry PIs can't access 192.168.99.100. We want our raspberry Pis on the local network to connect to our Rancher Server.

We resolve this by using VirtualBox's port forwarding feature. The following commands will forward our docker machine's 8080 port to our host machine's 8080 port. This allows us to access our Rancher Server from localhost:8080 and let our Raspberry PIs to connect to it through the laptop's local network IP.

docker-machine stop rancher-server  
VBoxManage modifyvm rancher-server --natpf1 'http,tcp,,8080,,8080'  
docker-machine start rancher-server  

Windows: you may need to add VBoxManage to your Path

Raspberry PI Setup

For my Raspberry PIs, I researched some notes on a couple of blogs swarming raspberry pi, withinboredom's post and github issues rancher issue#4703, rancher issue#5217.

With the preliminary research, there is an initiative to get Rancher available on Arm. issue#4703. The contents mention some non-trivial ways of building Rancher for arm, and that the Rancher Server will peg a Raspberry PI at 100% CPU usage. This is not something I want to test Rancher HA and other features, so I re-purposed an old laptop as my Rancher Server host.

After research, the first part was to get an OS on my Raspberry PIs capable of running docker. So I choose Ubuntu Snappy Core 16.04 for my Raspberry PI's OS. There are ready made images that make provisioning my SD Cards a breeze, Ubuntu Snappy Core for RPI

Do note that the images for RPI2 and RPI3 are different, so make sure that you download the correct one for your Raspberry PI device.

Once each Raspberry Pi is provisioned and boots; I connect them to my router, ssh in and run:

sudo apt-get install docker.io  

Next follow the steps on adding a Rancher host with some modifications based on Withinboredom's Raspberry PI setup https://www.withinboredom.info/post/rancher-on-pi/.

The following is an exert from Rob Landers from his blog https://www.withinboredom.com. Please visit his blog for an in-depth explanation and also a description of how to build theses images from scratch on your ARM device.


SOLUTION

If you don’t want to build all the required containers from scratch, you can just follow these simpler instructions on your arm device:

docker pull withinboredom/agent-instance:v0.8.3  
docker pull withinboredom/agent:v1.0.2  
docker tag withinboredom/agent:v1.0.2 rancher/agent:v1.0.2  

Now that you’ve gotten everything downloaded, go ahead and copy the command rancher gives you to add a custom host.

Within a few seconds you should see it appear in your infrastructure tab and you can deploy containers to it. When you see the network agent come online, jump back to your machine and run this, after it fails.

docker tag withinboredom/agent-instance:v0.8.3 rancher/agent-instance:v0.8.3  

Voila, you now have your own personal data center at home, courtesy of Docker and Rancher. This is the ultimate playground for mini-websites, testing distributed software, and figuring out more about the Rancher platform.