#15: The Ultimate Approval Process
From Junior to CTO Weekly Thoughts
The result of the software engineering process is an application that fits business requirements. So, how to guarantee the shippable code makes sense for business and does not violate any rules? In this issue, we discuss how different aspects impact The Ultimate Approval Process which is usually initiated by The Ultimate Merge Request but goes beyond. So let’s understand how to build the best fit for you.
From time to time, we put different aspects at the forefront. For some businesses, functional requirements are more critical than non-functional ones (they just need to get results asap, ignoring security/performance/maintenance and other stuff). For others, balance is important. For some of them approval process is overkill for others it is a crucial point of control. So let’s first clarify why we need The Ultimate Approval Process.
Motivation
Any process consists of steps that should be done to complete the entire process. The quality of every step impacts the final result. Software development is not an exclusion. Here we have the same rules that external quality control (by another actor) helps to improve the quality of results and reduce risks significantly. 4 eyes principle says that at least 2 people should review results. But why is this not a cargo cult? Why should we keep it as a part of The Ultimate Software Engineering? The essence of the answer is failure.
We live in a world where people make mistakes. It is impossible to avoid them fully, but we can minimize the risk in the places where it is important. I have already mentioned in The Ultimate Code Review you SHOULD NOT review and comment on stylistic mistakes, and it is more important to focus on such hard things to change later. For example, a badly designed API will keep client dependencies and limit your move forward, or wrong business logic can break data that might have a cost-sensitive impact on your business and waste your time on careful bug fixes in already working functionality. The cost of failure is the key motivation to set up your approval process. Applying the right people at the right time in your approval process makes your business less error-prone and more efficient. Let’s clarify who are “the right people”
Who
First of all, let’s clarify the features of the right approver. He or she:
must be in the context of the change or at least must have access to the requirements
has expertise in the area of change or is going to get this expertise
The right choice makes any review meaningful for approvers and brings motivation. Approvers might be not only other engineers, tech-lead, or qa. The Ultimate Approval Process reflects your business needs so this might include lawyers, security engineers, and even clients in some cases.
What
An approval process is not only about pure code reviews. It validates all different aspects of functional and non-functional requirements. Try to answer the following questions to find the best defaults for your approval process:
does this change require a state update and zero-downtime at the same time? Does this make sense to request reviews from well-experienced teammates?
does this change have infrastructure changes and request review by infrastructure engineers?
does this change require compliance review and validation with lawyers? (GDPR, CCPA, etc.)
should we document changes in any form and validate them with an architect? (technical documentation, ADRs, etc.)
should we revalidate that we have a budget for this change run because we know the implementation details?
should performance engineers have a closer look at that change?
Some of these practices can require activities from time to time but not by default. It means that someone needs to decide when to request extra approval. If you want to answer YES everywhere, please remember that you have to balance between time/money and scope of work.
When
Now we know the right people to approve, but it still might not be clear when they could control the results. Some practices like Shift Left move the review stage early, but they might be overkill in the beginning. So I recommend just starting reviewing incrementally do the left shift as much as possible. The bigger your team's scale and your product's complexity, the more important to move all such activities to the earliest possible time. For example, you should review and approve architecture before implementation, but you can approve the final implementation only when dedicated merge requests appear.
Where
This is one of the challenging questions because, due to the diversity of roles needed for the approval process, just a regular gitlab/github might not work. It depends on your company culture. Lawyers usually do not have gitlab accounts. So I’d say that if you are a small startup with a few components or bigger but do not have any approval process, start with approval process management embedded into your VCS hosting. Iteratively, retrospective-based, while your team/company is growing, consider what other approvals might be required and try to find common ground. For example, Jira has approval process management as well.
How
All good. People know what they should review and approve, where to do it, and when exactly, but how to organize the right people in the right place? The simpler system, the better, so formal approvals will be required not immediately from the beginning. When it is time to set approval rules, consider the following options:
semi-automated approvers choice
define code owners
manually choose others who will positively impact change confidence
manual reassignment - assign a lead software engineer / qa engineer / … by default to guarantee either own review and informed choice
I’d not recommend assigning random people and explaining it in “Who” section above
you also might apply predefined approval rules, but they are less flexible than code owners
the full intersection of expertise must cover the entire change. For example, if you had a review from QA and the security team but no other developers looked at the change - this breaks the process
on the first iteration
send a message to a chat because notifications might not work as you expected for some people
grow a culture of proactive reviews
every next iteration
should happen until approvals obtained
use the “reviewers” section to resend for the next iteration (at least GitLab supports that)
every chosen reviewer
must approve
approval means that the reviewer thinks that even if you introduce another change, it doesn’t require another review (it depends on your business and the criticality of changes, so that might be replaced with the necessity of repeated review)
These rules might look like reviewers will be a bottleneck, so I recommend reading The Ultimate Code Review to understand how to focus code reviewers only on essential things.
Conclusion
The approval process setup might look boring, but it protects your business from human factors. This unnecessary moderation will slow down the process in your earliest days, so be careful and choose the implementation time. In the next issue, we will discuss The Ultimate Branching Model that will help developers collaborate efficiently and ship your product without issues. Subscribe to not miss
The Video Of The Week
To help you assess the complexity of zero-downtime updates, look the video (primarily the first part)