Following on from my last post on Isolated Multistage Docker Builds, I thought it would be useful to cover another advantage to splitting your dockerfiles: building different output containers from a common base. The Problem When I have an application which when built, needs to have all assets in one container, and a subset of assets in a second container. For example, writing a node webapp, where you want the compiled/bundled static assets available in the container as a fallback, and also stored in an nginx container for serving....

Often when building applications, I will use a multistage docker build for output container size and efficiency, but will run the build in two halves, to make use of the extra assets in the builder container, something like this: docker build \ --target builder \ -t builder:$GIT_COMMIT \ . docker run --rm \ -v "$PWD/artefacts/tests:/artefacts/tests" \ builder:$GIT_COMMIT \ yarn ci:test docker run --rm \ -v "$PWD/artefacts/lint:/artefacts/lint" \ builder:$GIT_COMMIT \ yarn ci:lint docker build \ --cache-from builder:$GIT_COMMIT \ --target output \ -t app:$GIT_COMMIT \ ....

I write a lot of bash scripts for both my day job and my personal projects, and while they are functional, bash scripts always seem to lack that structure that I want, especially when compared to writing something in Go or C#. The main problem I have with bash scripts is that when I use functions, I lose the ability to log things. For example the get_config_path function will print the path to the configuration file, which will get consumed by the do_work function:...

Recently, I noticed that when we pull a new version of our application’s docker container, it fetches all layers, not just the ones that change. The problem is that we use ephemeral build agents, which means that each version of the application is built using a different agent, so Docker doesn’t know how to share the layers used. While we can pull the published container before we run the build, this only helps with the final stage of the build....

When it comes to implementing a new feature in an application’s ecosystem, I don’t like spending my innovation tokens unless I have to, so I try not to add new tools to my infrastructure unless I really need them. This same approach comes when I either want, need, or have been told, to implement a Service Mesh. This means I don’t instantly setup Istio. Not because it’s bad - far from it - but because it’s extra complexity I would rather avoid, unless I need it....

Before I start on this, I want to make it clear that if you can buy Honeycomb, you should. Outlined below is how I started to add observability to an existing codebase which already had the ELK stack available, and was unable to use Honeycomb. My hope, in this case, is that I can demonstrate how much value observability gives, and also show how much more value you would get with an excellent tool, such as Honeycomb....

One of the many features of Nomad that I like is the ability to run things other than Docker containers. It has built-in support for Java, QEMU, and Rkt, although the latter is deprecated. Besides these inbuilt “Task Drivers” there are community maintained ones too, covering Podman, LXC, Firecraker and BSD Jails, amongst others. The one I want to talk about today, however, is called exec. This Task Driver runs any given executable, so if you have an application which you don’t want (or can’t) put into a container, you can still schedule it with Nomad....

I noticed when running an Alpine based virtual machine with Consul DNS forwarding set up, that sometimes the machine couldn’t resolve *.consul domains, but not in a consistent manner. Inspecting the logs looked like the request was being made and responded to successfully, but the result was being ignored. After a lot of googling and frustration, I was able to track down that it’s down to a difference (or optimisation) in musl libc, which glibc doesn’t do....

I use Vagrant when testing new machines and experimenting locally with clusters, and since moving (mostly) to Linux, I have been using the LibVirt Plugin to create the virtual machines. Not only is it significantly faster than Hyper-V was on windows, but it also means I don’t need to use Oracle products, so it’s win-win really. The only configuration challenge I have had with it is setting up VM hostname resolution, and as I forget how to do it each time, I figured I should write about it....

I will update this post as I learn more (both positive and negative), and is here to be linked to when people ask me why I don’t like Kubernetes, and why I would pick Nomad in most situations if I chose to use an orchestrator at all. TLDR: I don’t like complexity, and Kubernetes has more complexity than benefits. Operational Complexity Operating Nomad is very straight forward. There are very few moving parts, so the number of things which can go wrong is significantly reduced....