Main Line - Updated
Getting working code integrated working isn't always a matter of just doing it. The Main Line Pattern describes why getting code to Main Line rapidly is a good thing.
See also Task Branch
When you use an agile software development process like Scrum, you plan iteratively. You want your codeline process to support your agile delivery model. This pattern describes the high-level structure — a Main Line — that makes it easier to deploy code quickly to reveal business value while maintaining stability and traceability by providing a central integration point for code.
Agile software development is based on frequent iteration and feedback as measured by inspecting working software. But stability and speed can appear to be at odds. Some teams try to reduce the risk of error by slow, disciplined steps when integrating work into a delivery code line. These teams may have:
More manual testing
More peer reviews
Intricate integration testing
Strict approval gates to prevent ‘accidental’ integration
One or more staging branches
The team integrates into a single shared code line only once it is “certain” that the code works. For example, in a GitFlowmodel, work in progress is integrated into a “Develop” branch, which is considered a working branch. The changes are merged to the shared mainline only after they have been approved for release.
There are variations, all of which work on the belief that isolation and moving slowly is safer. Keeping work on isolated branches preserves the stability of the eventual target branch in the short term but defers the problem: the target branch receives changes slowly, leading to process and business risk:
Process risk: The longer work stays isolated, the greater the risk of merge conflicts and divergent design decisions, as well as more work-streams to maintain.
Business risk: Slower code delivery means that features take longer to be released, which can lead to opportunity costs and longer feedback cycles based on larger features.
Since agile software development is about adapting to uncertainty in the project space, it is valuable to evaluate the current state of the code sooner.
Moving slowly can lead to a self-fulfilling dynamic:
A slow integration process leads to a temptation to integrate larger, more complex units of work.
The longer you keep your changes isolated, the harder the integration will be, both for your work and work started after your work stream started.
The more overhead for a merge, such as more testing due to change set size, the greater the temptation to introduce more work before you merge.
Faster integration means you could miss an error, but slow integration doesn't guarantee perfect software. Regardless of the size of the unit of work or how quickly you integrate it, merging code that breaks the shared integration codeline will slow down the entire team.
Frequent integration is more productive: the more frequently you integrate, the simpler each integration will be because the change is smaller and the work started with more recent code.
Create a Stable Baseline
Therefore:
Work on a Main Line, where all work is integrated. Use mechanisms to allow work to be integrated frequently while maintaining stability so that the Main Line is potentially deployable.
A Main Line is a code line that:
contains the “record” of the latest work
tracks the current state of working code and is the starting point for any new work
is the source of all releases (with the rare exception of emergency patch releases).
The Main Line is never deleted. It lives throughout the entire project, and the entire team contributes to it.
Work in progress, before it is merged to the Main Line could be:
on another branch (as this pattern language describes) or
In a developer workspace with no independent tracking branch.
In either case, the prerequisites are the same. This pattern language describes how to use short-lived branches, which is consistent with what Accelerate describes:
Following our principle of working in small batches and building quality in, high- performing teams keep branches short-lived (less than one day’s work) and integrate them into trunk/master frequently.
The goal of Agile Software development is to manage uncertainty. As Mike Cohn wrote in Agile Estimation and Planning:
The best way of dealing with uncertainty is to iterate. To reduce uncertainty about what the product should be, work in short iterations, and show (or, ideally, give) working software to users every few weeks…
Being biased toward quicker integration into a shared Main Line, rather than reducing error, can help you manage uncertainty by showing you an accurate current state. Errors will still happen; being able to recover when they do is more valuable than slowing down in an attempt to avoid them all. As Gary Klein writes in Seeing What Others Don’t:
“When we put too much energy into eliminating mistakes, we’re less likely to gain insights. Having insights is a different matter from preventing mistakes.”
A goal of an agile project is to gain insights into the state of the software by frequently inspecting the latest code in a running application.
Example
A Main Line development flow will look like the following, though each team can decide what the correct length of time is:
Checkout the HEAD of the Main Line into a development workspace
Code, backed by a Task Branch
Within a day, merge the code after an appropriate feedback process.
Cautions
Maintaining an active, healthy, Main Line takes discipline. An occasional error is inevitable, so while you might feel comfortable eliminating intermediate branches and getting code to the Main Line quickly, you may be tempted to add extra gates between “merged to main” and “released.” While this might be a reasonable starting place, you want to work to get to a point where the merge to the Main Line is quick, automated and gives you high confidence.
Next Steps
While the Main Line model’s simplicity, with fewer codelines, has advantages, you need some mechanisms to allow frequent integration to happen safely and reliably. You can’t avoid all errors, but you can avoid major ones and reduce the impact of any that slip through.
To help ensure a healthy Main Line you need to:
Define the rules for integrating to the Main Line and when to use other codelines: Codeline Policy.
Provide a place to reliably do development with the correct dependencies and tools: Developer Workspace
Allow for delivery of critical fixes to released code: Release Line
Enable Parallel, Independent Work that can be integrated into the Main Line quickly and reliably: Task Branch
Get feedback on work before it’s integrated into the Main Line: Code Review
Build and Test automatically: Integration Build
Get Feedback on design and implementation: Pull Request
Create a Retrospective Culture that is robust in the face of the inevitability that things will break despite best efforts and has a continuous improvement mindset.