Process Has Value; Focus On Process Drives Mediocrity
“No plan of operations can with any certainty reach beyond the first encounter with the enemy” — Helmuth von Moltke the Elder
“Plans are worthless, but planning is everything” — Dwight D. Eisenhower
“Everyone has plans until they get hit for the first time” — Mike Tyson
I am a total pain to work with if you’re a project manager. I dislike plans and process to a near pathological degree — I prefer to make it up as I go along, and for today’s work to be whatever grabs my whim rather than what’s actually most important.
I have learned to appreciate the value of plans and process, much as I still instinctively dislike them. For instance, I try to have people who are more process-oriented in leadership positions in my teams, explicitly as a counterbalance (and yes, I do let them know this so we can call each other out if we’re leaning too far one way or other). The hill I will die on, though, is that whenever a process becomes an end in itself it will end badly — not in disaster, but in mediocrity and long-term stagnation that’s hard to break out of because it’s comfortable.
A Pertinent Example: Estimation
Planning poker. T-shirt sizing. Dot voting. Story points. Agile practice has so many different ways to estimate how long it will take to build a piece of software, and yet I think we’re all aware of how flawed they are at actually predicting delivery.
One response to this is to double down on getting your estimates to be accurate. You can spend more time on scoping tickets than actually doing them, avoid taking on anything you don’t have full certainty of, over-estimate everything so as to always have lots of buffer, or refuse to take on any work that’s not strictly pre-planned in a sprint. None of these behaviours are good for solving business problems, but I’ve seen all of them done in the name of “we always get all our tasks done on time”.
The more healthy response is understanding what you’re trying to achieve by estimating. Estimation helps with prioritisation, it helps shake out what things are ambiguous, it helps stop things going on endlessly and it helps get everyone on the same page — all of which are ultimately about having a healthy team delivering value, and not about your estimates being perfectly predictive.
Processes aren’t an end in themselves, and forgetting that will undermine what you wanted them to achieve in the first place.
Accountability to results
Continuing with the example of estimation — at the most successful Big Tech companies, it’s rare to find much team-level process going on at all (Gergely Orosz talks in depth about it here), and while there will typically be some finger-in-the-air estimation for bits of work there is vanishingly little done below the OKR (or other company planning) level.
How then do you know how effective a person or a team is? The answer is that teams and individual engineers are held directly accountable for business impact, with varying levels of strictness — and in your performance review you need to justify the impact that you’ve had, not how good your task estimates were.
That doesn’t mean you can’t have a process — some teams at Big Tech companies do variations on scrum or kanban, all will have some kind of issue tracking with basic sizing/prioritisation, and large cross-cutting projects will typically have a project manager coordinating between everyone and keeping things on track. Each team and project has the freedom to determine what processes work for them though, and it’s all in service to having impact rather than to the process itself.
Where process helps: effective communication, especially under duress
The Tenerife airport disaster in 1977 is the most deadly aviation accident in history with 538 fatalities and 61 people further injured. It happened primarily as a result of poor communication. The crew of KLM Flight 4805 mistakenly believed they had been given clearance to take off while another aircraft was taxiing across the runway, and they were unable to see it due to thick fog. Amongst the communication mistakes were the captain interrupting the co-pilot’s readback, loose use of language, and interference in the radio transmissions to KLM 4805 due to transmissions from the taxiing plane.
One of the changes instituted as a result of the investigation of this incident was the introduction of Crew Resource Management, of which communication is a key component. For a simple example of CRM in action, here’s a clip from the movie Sully (about US Airways flight 1549):
They establish roles (the pilot flying — “my aircraft” — and the pilot monitoring), each with clear responsibilities. The communication style becomes explicitly turn-taking, and instructions from the pilot monitoring (based on the appropriate checklist) are explicitly verbally confirmed by the pilot flying — not only increasing the chances they’re done properly, but also increasing the shared knowledge about the situation for both pilots and explicitly encouraging the pilots to challenge each other rather than relying on seniority or falling into groupthink.
Most software project work isn’t as life-and-death as this, but the idea of using a process to improve your communication to ensure greater shared knowledge and alignment is a sound one. This is what most planning frameworks (OKRs and such) are at their core — and why I think the choice of which particular framework is largely irrelevant as compared to the value of communicating your teams’ plans clearly to each other.
Just be sure to remember to measure whether the implementation of your planning framework is actually doing that. If you don’t measure that, it’ll become an ass-covering exercise — “I wrote it down as an OKR in our shared space as per the process so it’s your fault you weren’t aware” is a huge red flag.
Where process helps: establishing your minimum standards
It’s reasonable to have things that are non-optional for individuals or teams. They can range from operational beliefs such as “we should write tests”, to values-based statements like “we want to pay for performance so will do regular performance reviews”, to compliance needs such as “we will consider the privacy implications in the design of all new features”.
This is great when you have individuals or teams who are falling short of these baselines, or who would if left fully to their own devices. It’s helpful for new or inexperienced teams — having a baseline from which to grow rather than starting from scratch is helpful and comforting. It’s also useful for informing roadmaps for developer experience teams and the like.
The challenge comes when people don’t understand what you’re trying to achieve with them. At best they’ll find them a frustrating checkbox exercise that’s slowing them down; at worst, they’ll actively work around them. “This code is so obvious it doesn’t need a test”. “This feature is too small for a design review”. “This outage didn’t impact enough users to warrant a full debrief”.
Worse still is the idea that because the process for ensuring the bare minimum is followed, then everything is rosy. You might enforce all pull requests including tests, but that doesn’t make the tests good. You might require an incident debrief after any outage, but that isn’t especially helpful if people fix the immediate symptoms but not underlying systemic issues. Your performance review process might ensure that people get feedback on how they’re doing at least once every 6 months, but if feedback is given only every 6 months that is wildly insufficient.
Worst of all is that, left unchecked, such processes cause a reversion to the mean for everyone — including those who didn’t need the process because they were going above and beyond already. If you have a team doing thorough and thoughtful test-first development, and you treat them the same as a team who add a single low-quality unit test in every pull request because they’ve been told to, then that first team is going to figure out your values and play to them really quickly.
My personal best practice if introducing a new process: make sure everyone understands what we are trying to achieve with it and that the process itself is only a minimum baseline, explicitly incentivise that outcome, and publicly champion those who go above and beyond the baseline.
Oh, and make sure there’s a feedback cycle built-in:
Retrospectives make the world go round
If you’re going to have a process, and it’s going to be in service of outcomes other than the process itself, you need a mechanism to make sure it is and to course-correct if necessary. This is why retrospectives of some sort are built in to every agile framework — and not only there, but also in high-stakes settings like accident investigations and military debriefs. I would go so far as to say that if you’re going to do something scrum-like, the retrospective is the only truly indispensable piece.
It’s not just holding a meeting or collecting feedback asynchronously though — you also have to actively make changes and retrospect on how they went, even if you think everything’s going well. That’s how you avoid complacency.
As a concrete example, Skyscanner ran a program where we measured the dependability of our squads — how often they completed all their sprint goals. A squad I support had racked up 13 sprints in a row where they’d hit every single goal — so I challenged them to try something new. Take on a stretch goal, try a new way of sizing tickets, organise ceremonies a bit differently — and see what happens. As it turned out they failed a sprint goal that week and reverted the change they made, but they learned something from it, shared what they learned with other squads, and felt good about it — like they were able to make more changes in the future.
So — do retrospectives, but also try something new even if everything’s going well so you’ve something interesting to talk about in them. If nothing else, it will remind you that the process itself is not the goal.