June 4, 2019
Here’s a crazy idea: software companies should require new engineers to make a production code change on their first day at work. Not allow engineers to deploy to prod on day one, but require it.
What exactly is it about this idea that makes it seem crazy, or at least ill-advised? Is it that it’s a bad idea, or is it that it doesn’t seem feasible? I do know that it’s possible: I first heard of it being done at Etsy back in the early 2010s, building on earlier ideas brought in from Flickr1.
If we start from the premise that an engineer’s job is to deliver value, and that value is only truly delivered once it’s in the hands of users, then surely we want a new engineer to be capable of pushing changes to prod pretty soon after they join a team. Based on that, having a new engineer capable of pushing changes as soon as their first day doesn’t really seem like a bad idea.
In this article I’ll argue that a policy of Day-One Deploys - expecting a new engineer to deploy to prod on their first day - is a good idea, for more reasons than you might expect. I’ll also explain how to make this crazy idea actually work.
Engineering organizations invest a lot of energy in streamlining their onboarding processes. When an engineer arrives for their first day they’ll have a fully configured laptop sitting waiting for them. Access to appropriate source code repositories, wikis, and other internal systems will have already been set up. The new engineer will often go through onboarding sessions in their first few days where they are shown the processes involved in making code changes, and perhaps in getting those changes into a staging or production environment.
Instituting Day-One Deploys takes this typical onboarding process and “turns up to 11”; pushing it to the extreme2.
Enabling rapid change
The obvious next question to ask - at least for me - is what systems you’d need to have in place if you wanted to turn it up to 11 and enable this crazy idea.
You’d need an extremely streamlined process for getting changes into production, simple enough that an engineer could use it successfully on their first day. You’d also need your deployment processes and your production systems to be extremely resilient3 - you wouldn’t want an engineer to accidentally break production on their first day! You’d also need a safe-to-fail culture - if an engineer does manage to break something their first day, the fault lies with the system that allowed a breaking change to make it all the way to production, not with the engineer that caused the breakage4. Finally, your teams would need to be comfortable using these capabilities to regularly push small changes to production within a very tight cycle time - the change would have to move through all quality gates (code review, QA, UAT, and so on) and deployment phases in less than a workday.
If we take a step back and look at these requirements, don’t they sound like a generally desirable state of affairs? Being able to push small, safe changes to production with only a day’s turn-around time seems like a pretty nice foundational capability for any software delivery team, doesn’t it?
The real benefit: a forcing factor for effective processes
There’s no coincidence here. The motivation for Etsy to start requiring Day-One Deploys was precisely because the policy acted as a forcing function, ensuring that their engineering tools and processes were set up to enable rapid, effective product delivery by reducing the cycle time, batch size, and therefore risk involved in production code changes5.
Sure, getting new engineers up to speed and productive in short order is valuable, but the real benefit that a team gets when adopting the policy of Day-One Deploys is the mature Continuous Delivery practices that they are compelled to follow in order to make the policy work.
OK, sure, getting changes into production with less than a day of lead time sounds terrific, as a general concept. But you might still be thinking that it’s a bit unreasonable to expect this of a new hire on their first day. If we were talking about a change to some critical business logic then I would tend to agree that maybe it’s a little ambitious. But in the case of Etsy the production change was simply adding a photo to the About page of the site. That’s a little less intimidating of a change to make on your first day. I also expect a new engineer would be paired up with an onboarding buddy: an experienced team member who can guide them through the processes involved in getting a change into prod6. A Day-One Deploy of a small, innocuous change with an experienced team member at your side to guide you doesn’t seem too bad.
When I’ve instituted this idea at companies we’ve used an even simpler change than the Etsy about page. We aimed to have every new team member add themselves to the site’s
humans.txt file on their first day. This is a very small change - a single line - in an area of the site with very low risk. Even if you really messed up the change it’s unlikely that you’d manage to cause any user-facing problems.
Requiring this type of small change on day one allows us to get the benefits of Day-One Deploys while minimizing the risk. This one line change to a production system, while small and safe, still requires that our new engineer has access to the appropriate internal systems (source control, project management systems, and what have you), and more importantly that they understand the processes and tools involved in getting a change from an engineer’s keyboard into the team’s production system.
This won’t work for my team!
For those of us that work in larger enterprise organizations or on systems with highly complex architectures, Day-One Deploys still may not seem feasible. That’s OK. The Day-One Deploy concept can still be used as a target state - a goal to aim towards. Even if that goal seems lofty today, your organization can still use it as a “North Star” and make small, continuous improvements in that direction. These will incrementally improve the ability of your teams to make small, safe changes.
Even if you don’t get all the way there, working towards Day-One Deploys will lead your organization to achieve many valuable Continuous Delivery practices along the way.
- In the late 2000s and early 2010s a lineage of startups were pioneering a radically new set of practices which roughly correspond to what’s now known as Continuous Delivery. Etsy and Flickr are prominent in that lineage. [return]
- Kent Beck has described Extreme Programming as taking a set of good engineering practices and “turning them up to eleven” to see what happened. [return]
- Flickr and Etsy referred to this resiliency as a “Production Immune System” [return]
- If a new hire manages to break things this should, in fact, be celebrated - an excellent learning opportunity to improve the system has occurred, and likely in a safe way with little business impact. [return]
- John Allspaw, who ran operations at both Flickr and Etsy, has a nice visual metaphor showing why lots of small changes are less scary than fewer, larger changes. [return]
- Assigning a well-prepared onboarding buddy for every new hire is an excellent practice for any engineering org, regardless of whether or not the idea of Day-One Deploys motivates you. [return]