Pete Hodgson

Software Delivery Consultant

Engineering Values and Architectural Principles

January 18, 2021

Every engineering leader at a growing startup encounters the same challenge at some point: how to ensure your teams are making decisions in a roughly coherent way.

When a startup is young, it’s easy to keep all the engineers on the same page. Everyone is in the same codebase, working on the same thing and talking through technical decisions on a daily basis. But as a startup begins to scale, this natural alignment starts to fall away. Cliques and tribes form as different teams evolve distinct approaches to designing and building software.

This post describes a powerful approach for addressing incoherent technical direction: defining a set of Engineering Values and Architectural Principles. Together these twin manifestos articulate clear “rules of the road” for your engineering teams, allowing them to maintain their autonomy while acting in a coordinated way - what I refer to as aligned autonomy.

What worked before won’t work now

the practices that have kept 10 engineers aligned won't be sufficient for 100 engineers

As an engineering leader at a growing startup, you must embrace a fundamental reality: the practices that have kept 10 engineers aligned won’t be sufficient for 100 engineers. When your engineering org consists of a handful of engineers, they operate as a tight clan. You can get away with relying on unstructured communication channels to keep to clan together: pairing and code review, design discussion in the #engineering channel, getting lunch as a team.

As the organization grows, there is a shift from a single clan to a small set of cooperating clans. Keeping this more disparate group aligned requires more than the informal mechanisms that kept a single team together. It requires a more intentional, structured approach.

Clans and cliques

This shift towards distinct clans is human nature. Teams that work together every day develop a shared set of values, but different teams won’t naturally maintain the same set of values. Over time engineers explore new ideas and approaches, and each team’s shared set of values evolve. What’s more, since each team has different personalities and is exploring different ideas, these values evolve in different directions.

Startups that are scaling rapidly experience this shift at an accelerated pace. The constant influx of new engineers - each bringing values and preferences from past experiences - provides more energy to the forces pulling your once-cohesive engineering culture apart.

weak inter-personal cohesive forces overcome by the power of different ideas

By the time a startup is approaching 100 engineers it’s not unusual for a few distinct tribes to have formed, with their own value systems and architectural philosophies. The Accounts team are embracing an event sourcing paradigm, while the Conversion team has gone all-in on hand-crafted SQL. The Growth squad is very responsive to incoming feature requests but a tad chaotic, while the Partner Integrations squad is extremely methodical, but slow to react. The front-end devs are mob-coding and practicing trunk-based dev, while the data engineers have embraced gitflow and rigorous code review.

Twin Manifestos: Engineering Values and Architectural Principles

While diverse opinions are valuable, engineers need to operate in a coherent way if we want to make headway on major initiatives. To keep larger groups of engineers from splintering, we must define some common beliefs - guard rails which help bring the value systems of each clan into rough alignment.

shared values allow autonomous teams to remain aligned

I like to structure these high-level “rules of the road” as a pair of manifestos: Engineering Values and Architectural Principles. These manifestos provide a clear articulation of the common values and beliefs which all engineers within the organization should share. Engineering Values describe the softer stuff - ways of working and collaborating - while Architectural Principles guide technical decisions.

Engineering Values

Engineering Values encapsulate the engineering organization’s culture - what we value and how we work together as humans.

Some examples of Engineering Values are “Move Quickly and Iterate”, or “Outcome over Effort”, or “Minimal Viable Process”.

You might see Engineering Values used for things like:

Architectural Principles

Architectural Principles lay out a shared technical philosophy that underpins the design decisions that engineers make.

Examples of Architectural Principles are “Prefer using existing software”, or “Secure by Design” or “Build Dependable Interfaces”.

You might see Architectural Principles used for things like:

A tool for coherent decision-making

You know you have a good set of Engineering Values and Architectural Principles when engineers use them as a guide when weighing up a tough decision - their primary purpose is to nudge engineers gently towards a coherent way of working and a consistent architectural approach.

each Value and Principle needs to be general enough that it can be applied in a variety of contexts, but specific enough to avoid become a hand-wave-y platitude

As such, each Value and Principle needs to be general enough that it can be applied in a variety of contexts, but specific enough to avoid become a hand-wave-y platitude. An Engineering Value of “Do Your Best Work” is too vague to provide useful guidance when making a decision, while an Architectural Principle of “Prefer Relational Data” is likely too specific to be applicable beyond a small set of decisions.

In my experience, a good target to aim for is a pair of manifestos containing 5-10 tenets each, with each one encapsulating a belief or value which is not wildly controversial but which also wouldn’t be universally held by all engineers in your organization by default.

I recommend formatting each Engineering Value or Architectural Principle as a catchy phrase, along with a sentence or two to describe the concept in more detail. Here’s a good example from Scout24’s Engineering Values:

Outcome over effort
We celebrate the impact of our work over the effort we’ve been putting in or the amount of code we produced. We understand the potential impact of our work to our business and search for an easy, fast, secure way to validate our ideas before writing any code.
source

But our engineers don’t like top-down direction!

Some organizations, particularly smaller ones, can be a little skittish about introducing this sort of high-level direction. It feels a little too much like a top-down edict. Can’t we just trust our engineers to be reasonable adults and make good decisions?

Firstly, any productive engineering team needs to work in a consistent way. There will always be a set of values and principles at work, it’s just a matter of whether they’re implicit and unspoken, or explicit and clearly defined. Secondly, in my experience most engineers want clear direction. Having these shared values explicitly laid out means engineers can avoid making design decisions that are declared “bad” after your team has spent weeks implementing them.

Most importantly, these manifestos should not be created in a vacuum - they should be a collaborative effort that involves the engineers who will be adopting them, and reflects (broadly) the existing culture and philosophy within the group. Done correctly these manifestos become a refined expression of what was already shared by many engineers, rather than a stack of freshly-carved tablets carried down from the mountain.

Provide direction to achieve aligned autonomy

Every startup goes through growing pains; periods where what used to work suddenly falls apart. When it comes to engineering alignment, the best way to counter tribalism and incoherent architectural decisions is to provide teams with a clear high-level direction. This allows teams on the ground to maintain the autonomy which has allowed them to move fast, while also delivering enough alignment to ensure that everyone is moving in roughly the same direction. A set of shared Engineering Values and Architectural Principles are a great way to articulate that shared high-level direction without being overly prescriptive.

Acknowledgements and inspiration

I initially encountered the concept of Architectural Principles in Sam Newman’s excellent Building Microservices book.

For inspiration, here’s a collection of manifestos which hew close to the ideas of Engineering Values and Architectural Principles:

Key Values is a site that catalogs the culture of different engineering orgs, using something similar to Engineering Values.

Thanks to David Golden for some excellent feedback on a draft of this post.