Using the docker-gen container with nginx as a reverse proxy: a practical example

Following on from the post using docker-gen stand alone in a docker container, we will now use the container in a more real world example.

docker-gen will be used to generate configuration files for nginx to act as a reverse proxy for other docker containers serving web content.

Purpose

You have a set of Docker containers ready for serving web content; these could be either Apache, lighthttpd, hugo, something you have written yourself, or anything which understands HTTP and returns content. There is also the possibily of needing to run multiple copies of the same container.

By default the IP addresses and port numbers of Docker containers are private to the Docker host, and we can not predict which IP address or port is assigned to a particular container. Docker does allow ports to be mapped from the container to the host but that means we can only have one container assigned to one port, not something we want if we are running a container multiple times or we have other containers who need the port.

This is where a reverse proxy and docker-gen will be useful. We can keep everything private and not have to worry about IP addresses or port assignments. There also comes an additional benefit in that containers can be upgraded or scaled up/down without causing any downtime!


Getting started

We require a minimum of three containers groups: docker API monitoring and configuration file writing (docker-gen); reverse proxy (nginx); and serving of web content (Apache, lighthttpd etc.).

The docker-gen image

First create a templates directory and place a docker-gen template file (reverse-proxy.tmpl) for nginx there. Here is a simple configuration for a reverse proxy:

{{ range $server, $containers := groupBy $ "Env.VHOST" }}
upstream {{ $server }} {
  {{ range $index, $container := $containers }}{{ with $address := index $container.Addresses 0 }}server {{ $address.IP }}:{{ $address.Port }};{{ end }}{{ end }}
}

server {
  server_name {{ $server }};

  location / {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_pass http://{{ $server }};
  }
}
{{ end }}

The template file will go through each container which has the environment variable VHOST set and print out a relevant upstream{} entry for it. If there are multiple containers running with the same VHOST, then they will be written to the same entry. This allows us to upgrade containers with zero downtime.

Now create a directory called share and create an empty file called reverse-proxy. You’ll probably also want to put nginx’s default sites-enabled configuration file here as well.

The Dockerfile

Make sure a docker-gen binary exists (see here on how to get one) and create the following Dockerfile.

FROM ubuntu:14.04

MAINTAINER JustAdam <adambell7@gmail.com>

RUN apt-get update && \
    apt-get upgrade -y

ADD docker-gen /usr/bin/docker-gen
ENV DOCKER_HOST unix:///docker.sock

VOLUME ["/etc/docker-gen/", "/share/"]
WORKDIR /etc/docker-gen/

CMD ["/usr/bin/docker-gen", "-watch", "-only-exposed", "-notify-sighup=nginx-reverse-proxy", "reverse-proxy.tmpl", "/share/reverse-proxy"]

The -notify-sighup flag specifies which container to send a HUP signal too; in this case it will be nginx-reverse-proxy which is what our nginx reverse proxy container will be called. -watch tells docker-gen to watch Docker for events and -only-exposed will watch only for containers with exposed ports. You should remove this if your web server containers do not use the Dockerfile EXPOSE instruction or relevant command line flag.

Build the image and start the container

$ docker build -t justadam/docker-gen:demo .
$ docker run -d --name docker-gen -v /var/run/docker.sock:/docker.sock -v $(pwd)/templates/:/etc/docker-gen/ -v $(pwd)/share/:/share:rw justadam/docker-gen:demo

docker-gen is now running and watching Docker for relevant events. Now we will create a nginx reverse proxy container so it can start to be useful.

The nginx image

You can use one of the already tried and tested nginx images out there; but for simplicities sake here is another one:

FROM ubuntu:14.04

MAINTAINER JustAdam <adambell7@gmail.com>

RUN apt-get update && \
    apt-get upgrade -y && \
    DEBIAN_FRONTEND=noninteractive apt-get install -y nginx
RUN echo "\ndaemon off;" >> /etc/nginx/nginx.conf

VOLUME ["/etc/nginx/sites-enabled/"]
EXPOSE 80

CMD ["nginx"]

Build the image and start the container.

$ docker build -t justadam/nginx:demo .

Update the path to the correct location of the share directory used with the docker-gen container.

$ docker run -d -p 80:80 --name nginx-reverse-proxy -v /path/to/share/:/etc/nginx/sites-enabled/ justadam/nginx-demo

nginx is now running and reading its configuration from the docker-gen share directory. At the moment that is empty and not particularly useful; so lets create some containers to serve web content.

The web server image(s)

If you have web server container already available, then use that - just remember to start it with the environment variable VHOST set to your relevant virtual host (domain name).

This example will be using hugo: a fast static site generator that also comes with its own web server.

If you require a sample hugo configuration and site, then clone this into a directory of your choosing and change /path/to/hugo/files/ a little later to match this.

Hugo

Pull a copy of hugo from the Docker repository.

$ docker pull justadam/hugo:0.12

This image specifies hugo’s working directory as a volume, so everything hugo expects to work should go in there. Hugo is then configured to write the output files into the container’s file system.

$ docker run -d --name hugo -e VHOST=www.yourdomain.com -v /path/to/hugo/files:/content justadam/hugo:0.12

Hugo is now running as a web server and monitoring its working directory so that any file changes will cause a regeneration of the site.

Because the VHOST environment variable is set, docker-gen detected this container starting and wrote an entry for it in nginx’s configuration file, and then requested the nginx container to reread its configuration.

Likewise when the container is stopped, docker-gen will detect this and remove the entry. This allows us to have zero downtime when carrying out container upgrades/maintenance.

Now browse to www.yourdomain.com in your web browser and you should see your hugo content served via nginx.


  • port-eighty is a project making use of these ideas and can be found on github.

Adam Bell-Hanssen

maybe, someday .. just another code and ops guy

Oslo, Norway