An interesting question came up at work today: how do you tag your Docker images? In previous projects, I’ve always used a short git sha, or sometimes a semver, but with no great consistency. As luck would have it, I had pushed for a change in tagging format at a client not so long ago as the method we were using didn’t make a lot of sense and, worst of all, it was a manual process....
Following on from my last post on Isolated Multistage Docker Builds, I thought it would be useful to cover another advantage to splitting your dockerfiles: building different output containers from a common base. The Problem When I have an application which when built, needs to have all assets in one container, and a subset of assets in a second container. For example, writing a node webapp, where you want the compiled/bundled static assets available in the container as a fallback, and also stored in an nginx container for serving....
Often when building applications, I will use a multistage docker build for output container size and efficiency, but will run the build in two halves, to make use of the extra assets in the builder container, something like this: docker build \ --target builder \ -t builder:$GIT_COMMIT \ . docker run --rm \ -v "$PWD/artefacts/tests:/artefacts/tests" \ builder:$GIT_COMMIT \ yarn ci:test docker run --rm \ -v "$PWD/artefacts/lint:/artefacts/lint" \ builder:$GIT_COMMIT \ yarn ci:lint docker build \ --cache-from builder:$GIT_COMMIT \ --target output \ -t app:$GIT_COMMIT \ ....
Recently, I noticed that when we pull a new version of our application’s docker container, it fetches all layers, not just the ones that change. The problem is that we use ephemeral build agents, which means that each version of the application is built using a different agent, so Docker doesn’t know how to share the layers used. While we can pull the published container before we run the build, this only helps with the final stage of the build....
One of the many features of Nomad that I like is the ability to run things other than Docker containers. It has built-in support for Java, QEMU, and Rkt, although the latter is deprecated. Besides these inbuilt “Task Drivers” there are community maintained ones too, covering Podman, LXC, Firecraker and BSD Jails, amongst others. The one I want to talk about today, however, is called exec. This Task Driver runs any given executable, so if you have an application which you don’t want (or can’t) put into a container, you can still schedule it with Nomad....
I had a major problem a few hours before giving my Nomad: Kubernetes Without the Complexity talk this morning: the demo stopped working. Now, the first thing to note is the entire setup of the demo is scripted, and the scripts hadn’t changed. The only thing I had done was restart the machine, and now things were breaking. The Symptoms A docker container started inside the guest VMs with a port mapped to the machine’s public IP wasn’t resolvable outside the host....
My normal development laptop runs Windows, but like a lot of developers, I make huge use of Docker, which I run under Hyper-V. I also heavily use the git bash terminal on windows to work. Usually, everything works as expected, but I was recently trying to run an ELK (Elasticsearch, Logstash, Kibana) container, and needed to pass in an extra configuration file for Logstash. This caused me a lot of trouble, as nothing was working as expected....
I gave a little talk at work recently on my use of Vagrant, what it is, and why it is still useful in a world full of Docker containers. So, What is Vagrant? Vagrant is a product by Hashicorp, and is for scripting the creation of (temporary) virtual machines. It’s pretty fast to create a virtual machine with too, as it creates them from a base image (known as a “box”....
When building libraries, not only is it a good idea to have a large suite of Unit Tests, but also a suite of Integration Tests. For one of my libraries (RabbitHarness) I have a set of tests which check it behaves as expected against a real instance of RabbitMQ. Ideally these tests will always be run, but sometimes RabbitMQ just isn’t available such as when running on AppVeyor builds, or if I haven’t started my local RabbitMQ Docker container....
My work’s wifi is much faster than my 4G connection, so periodically I want to update all my docker images on my personal laptop while at work. As I want to just set it going and then forget about it, I use the following one liner to do a docker pull against each image on my local machine: docker images | grep -v REPOSITORY | awk '{print $1}'| xargs -L1 docker pull If you only want to fetch the versions you have the tags for:...