The unexpected positive upshot of automating deployments
The unexpected positive upshot of automating deployments
The paradise of city living
The paradise of city living
About 4 years ago now, I started a job as a software engineer in Melbourne, Australia. I was living in Geelong at the time; a country town approximately 70 minutes drive (without traffic) or 90 minutes train ride from the city. I would travel there each morning, and return home each evening. Should there be no train delays, I would leave at ~6:30am and arrive sometime after 8:30pm.
Very shortly, these longer days began to wear thin on me. Additionally, my wife was formerly from Melbourne, and the prospect of moving back became increasingly attractive. So, three months after I got the job we packed up and moved into the city, to an area called “Docklands” — 1.6 km from my new workplace.
OH MY GOD THE EXTRA TIME!
Moving to Docklands was expensive. For a short period it was a struggle, but oh my was it worth it.
I can’t overstate the impact of having the extra 3–4 hours per day. When it started, I wasn’t quite sure what to do with myself! I did the same things I did on the train; watch Netflix, read a book or whatever. But quickly I began to realise the opportunities around me; I would go out for dinners with friends, work a little later on an interesting problem or simply take the time to go for a little walk. Eventually I took up sports, and my health improved dramatically.
The interesting thing is, if you’d asked me from Geelong whether I would have experienced such an impact; found these opportunities or started training I would have said “eh, maybe. I’ll probably just sleep more.”
Solving the immediate problem
Initially, I moved to Melbourne as I was finding the lack of sleep was negatively affecting my health. However, looking back, that is not the reason I would recommend it.
The unexpected positive upshot
Not only could I take the time I would have otherwise spent travelling and spend it elsewhere, but travelling all together became cheaper. It was no longer a barrier to social events, and I became more involved in the community.
Solving the immediate problem of lost time travelling presented a whole range of opportunities that were both completely out of reach, but additionally completely unknown to me at the time.
Taking that lesson to work
As a software engineer, I work building and releasing software to a wide range of systems on behalf of the project owners. Many of the tasks I perform each day are inherently repetitive; the most common of which is to push changes I have made further down the delivery pipeline.
The development pipeline looks like diagram attached.
Development and review is straight forward; someone writes the code, and someone else checks the code.
However, the other three steps are all handoffs to other teams. At all three steps, a “deployment” is required; a process of making changes to the environment such that the new code, migrations and other changes are in a state where they’re suitable for that team.
The inherent difficulties of deployments
When taking changes from one system to the other, several steps must be performed:
The database reset with production-like data
The code updated
The application recompiled
Shared resources such as images, log paths etc wired into the application
Caches flushed
The application is made available to the public
At best, these steps can get tiresome. At worst, they cause slight bugs that do not show for many days, but are critical to the application lifecycle — like the breaking of a checkout. Additionally, these steps for an experienced developer can take half an hour, and for inexperienced developers can take 2 or more hours.
Solving the immediate problem
Given that I’m a programmer, I usually elect to script away much of the manual work that’s associated with deployments. Now, deployments take me about 3–4 seconds. A click a button, and a computer carries off and does the dull work of making sure things all work as expected, and lets me know if something is amiss.
As I’ve become more experienced it’s become more elegant, until the most recent deployments which are fundamentally superior to anything that I could ever do as a human.
However, it’s interesting to reflect what happens when deployments are this easy.
The unexpected positive upshot
There are a few unexpected side-effects of automating the deployments in this fashion.
Deployments carried out by humans are as perfect as humans — which is to say, not so much. Deployments carried out by machines will still fail, but when they fail they will fail for understandable reasons, and additional safeties can be put in place which prevent that particular class of failure in future. Humans fail because they were up late watching game of thrones and forgot to clear the cache.
Secondly, they’re cheap enough that deployments start happening for all sorts of trivial events. I deploy additional logging, instrumentation of a UX element, an additional topnav menu item. Rather than being the rate limiting step for any work completed, deployments are simply the success of that step of the pipeline — all the way through to production. This lets project owners innovate faster, make more accurate decisions and test more ideas without increasing technical risk.
Thirdly, they’re transparent. Automating deployments doesn’t obviate the requirement of making them visible; indeed, just the opposite — once they’re automatic, it becomes critically important to debug failures after they’ve happened. This visibility doesn’t simply apply to me, as the implementer though — other members of my team, including non-technical members such as the project manager or QA team now understand and appreciate this process, and can return to the customer with a greater understanding of how the workflow works.
Lastly, they’re democratic. By encoding the knowledge required to deploy in some form of automation I am not only delegating the ability to deploy to people who do not understand the deployment process entirely, but additionally making the deployment process vastly more accessible to all developers. There is no undefined magic that must be performed in order to make code live so if I disappear or go on holiday, others can use or repair the deployment process.
A more general trend to reducing the cost of common operations
More generally, as I become older as a person and more experienced as a developer, I find myself either automating complex tasks that I do not like doing, or simply jettisoning the requirement all together.
Reducing the burden of these repetitive operations has significant benefits, and they’re not always apparent prior to the release of the work. So, as a matter of course I tend to automate tasks that I do several times.
Thanks
To those that reviewed the post prior to publication including:
William de Groot
P.S.
I give a couple of talks on these things, including:
“Implementing DevOps”
“Automating Deployments”
“Using Docker for Fun + Profit”
If you’re keen to learn more, get in touch and we’ll work it out.