Branching and Red Builds

So this is a bit of a rant…but hopefully with some solutions and workarounds too. So let’s kick things off with a nice statement: I hate broken builds. So everyone basically agrees on this point I think. The problem is that I mean all builds, including ones on shared feature branches. Currently, I work on a number of projects which uses small(ish) feature branches. The way this works is that the team agrees on a new feature to work on creates a branch, and then each developer works on tasks, committing on their own branches, and Pull-Requesting to the feature branch....

August 10, 2018 · 3 min

Managing AppSettings in Consul

Consul is a great utility to make running your microservice architecture very simple. Amongst other things, it provides Service Discovery, Health Checks, and Configuration. In this post, we are going to be looking at Configuration; not specifically how to read from Consul, but about how we put configuration data into Consul in the first place. The usual flow for an application using Consul for configuration is as follows: App Starts Fetches configuration from Consul Configures itself Registers in Consul for Service Discovery Ready Step 2 is very straightforward - you query the local instance of Consul’s HTTP API, and read the response into your configuration object (If you’re using Microsoft’s Configuration libraries on dotnet core, you can use the Consul....

August 7, 2018 · 4 min

Locking Vault Down with Policies

The final part of my Vault miniseries focuses on permissioning, which is provided by Vault’s Policies. As everything in Vault is represented as a path, the policies DSL (Domain Specific Language) just needs to apply permissions to paths to lock things down. For example, to allow all operations on the cubbyhole secret engine, we would define this policy: path "cubbyhole/*" { capabilities = ["create", "read", "update", "delete", "list"] } Vault comes with a default policy which allows token operations (such as looking up its own token info, releasing and renewing tokens), and cubbyhole access....

June 23, 2018 · 4 min

Secure Communication with Vault

I think Vault by Hashicorp is a great product - I particularly love how you can do dynamic secret generation (e.g for database connections). But how do you validate that the application requesting the secret is allowed to perform that action? How do you know it’s not someone or something impersonating your application? While musing this at an airport the other day, my colleague Patrik sent me a link to a StackOverflow post about this very question...

June 22, 2018 · 5 min

Fixing Docker volume paths on Git Bash on Windows

My normal development laptop runs Windows, but like a lot of developers, I make huge use of Docker, which I run under Hyper-V. I also heavily use the git bash terminal on windows to work. Usually, everything works as expected, but I was recently trying to run an ELK (Elasticsearch, Logstash, Kibana) container, and needed to pass in an extra configuration file for Logstash. This caused me a lot of trouble, as nothing was working as expected....

June 18, 2018 · 2 min

Managing Postgres Connection Strings with Vault

One of the points I made in my recent NDC talk on 12 Factor microservices, was that you shouldn’t be storing sensitive data, such as API keys, usernames, passwords etc. in the environment variables. Don’t Store Sensitive Data in the Environment My reasoning is that when you were accessing Environment Variables in Heroku’s platform, you were actually accessing some (probably) secure key-value store, rather than actual environment variables. While you can use something like Consul’s key-value store for this, it’s not much better as it still stores all the values in plaintext, and has no auditing or logging....

June 17, 2018 · 6 min

Writing Conference Talks

I saw an interesting question on twitter today: Hey, people who talk at things: How long does it take you to put a new talk together? I need like 50 hours over at least a couple of months to make something I don’t hate. I’m trying to get that down (maybe by not doing pictures?) but wondering what’s normal for everyone else. Source I don’t know how long it takes me to write a talk - as it is usually spread over many weeks/months, worked on as and when I have inspiration....

May 15, 2018 · 3 min

Test Expressiveness

We have a test suite at work which tests a retry decorator class works as expected. One of the tests checks that when the inner implementation throws an exception, it will log the number of times it has failed: [Test] public async Task ShouldLogRetries() { var mockClient = Substitute.For<IContractProvider>(); var logger = Subsitute.For<ILogger>(); var sut = new RetryDecorator(mockClient, logger, maxRetries: 3); mockClient .GetContractPdf(Arg.Any<string>()) .Throws(new ContractDownloadException()); try { await sut.GetContractPdf("foo"); } catch (Exception e){} logger....

February 26, 2018 · 2 min

Task Chaining and the Pipeline Operator

Since I have been trying to learn a functional language (Elixir), I have noticed how grating it is when in C# I need to call a few methods in a row, passing the results of one to the next. The bit that really grates is that it reads backwards, i.e. the rightmost function call is invoked first, and the left hand one last, like so: await WriteJsonFile(await QueueParts(await ConvertToModel(await ReadBsxFile(record)))); In Elixir (or F# etc....

February 20, 2018 · 3 min

Tweaking Processes to Remove Errors

When we are developing (internal) Nuget packages at work, the process used is the following: Get latest of master New branch feature-SomethingDescriptive Implement feature Push to GitHub TeamCity builds Publish package to the nuget feed Pull request Merge to master Obviously 3 to 6 can repeat many times if something doesn’t work out quite right. There are a number of problems with this process: Pull-request after publishing Pull requests are a great tool which we use extensively, but in this case, they are being done too late....

December 9, 2017 · 3 min