Trunk based development- A journey

Posted on: 31 Jul 2018

Category: Technical

Blog Views: 1177

"The trunk-based development methodology simply believes in one branch for everybody. And that one branch is … TRUNK [trumpets playing…], yes, the TRUNK as the name itself suggests! The Trunk? Many of us jumped out of our seats when we first heard it and I jumped the highest! How can it be? How can anyone develop on trunk? Wasn’t it supposed to be the “holy grail”, “stable” branch? It’s not going to work. This is a recipe for disaster. Are you out of your mind? "

Silhouette, Women, Tree, Yoga       

I belong to a generation of programmers who started coding in the pre-Google era and I still love to code. I guess that makes me a rare species? I will probably write another blog someday about coding during the pre-Google era. Having seen different phases of development practice and a number of “Version control & Branching Techniques” over the past couple of decades, I feel I should be writing these blogs to share my experiences as well as the learnings during my journey and why I consider trunk(master) based development is the best among all.

That Central Folder!

        During my initial days as a programmer, our version control system was a folder in a shared computer that had a read-only access to developers. Only the project lead and manager were able to write to that “central folder”. They would receive the files and documents from team members through e-mails, to be blessed with their review and ultimately your file and changes found a place in the central folder! That job of project lead / manager was considered privileged niche and highly sought after, but now I would pity them!

That Central Version Control System!

       Slowly, we changed to one of most famous centralized version control software. But the change was only for the leads and managers. For everyone else everything remained the same. The developer was still sending files to managers and they would use the version control software to check-in the files. At one point, we had a system of records, wherein  for every file, you had to submit a paper form with the nature of changes described! Yes, paper forms! Now, laugh as much as you want at this process, but in those days,  this was something agreed and designed considering the relative infancy of the version control processes and tools.

      Eventually, the leads and managers grew tired of managing these centralized version control systems and decided to give the control to each developer. Viola! Every developer had a user created in the version control system and granted access to check in the code, he/she no longer had to fill those forms and depend on someone else, the life looked better. Under privileged turned privileged!  But there were heart burns, since the code was all in one “branch” which is called trunk, it was super easy to break the system accidentally or intentionally. Having one branch meant that you could simultaneously break production, staging and your dev environment and people did break it!

Those Branches!

      This is when we started exploring the concept of branches in version control system. So, every one of us got excited about these little isolated branches and started creating branches at the drop of a hat. There was a release branch which was meant to go into production system, so that branch was the sanctum sanctorum, and no one could break it because we all had our own “private”, “feature”, “mom-n-pop” branches that we could break if we want to. The night before the release date, one person or couple of them at best sat and merged the code from all these mini-islands called branches onto “the release” branch and prepared the release. It was an event at best, with lots of midnight oil, some oily chapatis and pizzas until the branch worked. Very often the magic branch was ready few hours before the change window.

Now it's distributed vs central! 

    Next in the evolution cycle were the distributed version control systems which took the sheen away from their cousins called central version control systems (CVS vs DVCS). The idea was good, it was a huge paradigm shift. You could clone, fork and branch from more than one system. But the problem of “merge event” just transformed from one form to another. You still had to sit and merge hundreds of branches, thousands of lines of code and configuration, and yet deliver a stable release. Almost every release was anything but “stable” (in the eyes of the customer), it took couple of weeks, sometimes months to deliver stable release branch. In that time frame, the maverick development teams had already started committing code to de-stabilize the “stable” branch in the days to come.

Enter Trunk-based development

  This led way to “trunk-based development” concept (and I am finally coming the topic of this blog! You do have a lot of patience!). The trunk-based development methodology simply believes in one branch for everybody. And that one branch is … TRUNK [trumpets playing…], yes, the TRUNK as the name itself suggests! The Trunk? Many of us jumped out of our seats when we first heard it and I jumped the highest! How can it be? How can anyone develop on trunk? Wasn’t it supposed to be the “holy grail”, “stable” branch? It’s not going to work. This is a recipe for disaster. Are you out of your mind? These were my and most people’s reaction. But you cannot judge how deep or how bad the water current is, unless you jump into water. We had to give it a try. And we did, mostly unwillingly.

     The very concept of trunk-based development and everything around it is described here.  What it will do is, take your “stable” branch, add your code to it and continue it to be “stable” even after your changes went in. Well, that’s a challenge! How can I or anyone merge our mammoth changes and tens of files that I worked for two weeks into a branch and expect it to be stable? Well, the answer is, commit as little as possible and commit as often as possible. Yes, this new species of beast brought in an element of discipline in all of us. Everyone dissected their work and user stories into as small a task as possible and committed the code to the trunk. They committed daily, sometimes more than that. The potatoes were boiling quicker and better when we cut them into smaller pieces. We got the latest code from trunk as often as we had to commit, sometimes more than that. As a result, we now have a continuous delivery of functions and user stories into trunk (i.e master in the case of git). It was not a picture-perfect scenario; the people did break the build. And when they did it was a huge embarrassment. They stalled the work of hundreds of developers for few hours. But that embarrassment made them a better developer. It had a huge price to pay, it was largely offset by writing nice unit and integration testing and the advent of continuous integration and delivery (CI/CD) tools. The bad code commits, and merge was spotted much early in the release cycle. When you spot it early, you can fix it easy, you can fix it cheaper. Now compare that to finding a code conflict or broken piece of functionality the night before the release in the all the earlier cases. The trunk-based development enabled the branch to be release-ready any given time. Almost ready, the best one can offer at any time. The trunk or the master branch itself turned into a release branch. The release branch is no longer I-don’t-know-how-it-works-in-this-branch! It’s exactly what everyone has been testing for months continuously. After about 3 months the entire team realized how valuable this approach is, how much time it saved, and proved how wrong we all were at the beginning. It works and we started having pizzas during regular outings, but not during “the code-merge-day” (or night?)! More and more companies are following this approach.

   Of course, there are lot of unanswered questions in this blog, my intention was not to describe every aspect of trunk-based development, but just to share the experience of transforming from central folder approach to this new kid on the block called trunk-based development. It’s not really a new kid for many people, but if you are not yet there, then it’s worth considering. It’s more than 7 years and counting, since I (and my team back then) made this huge paradigm shift. 

   Make sure you go through feature flag, code reviews, CI-CD, short lived branches etc and adopt what’s best for you and what’s must for you. The url again is:

Good luck!  

Image courtesy:

I do believe a lot of you out there have something interesting, something to counter on this topic? Feel free to add a comment below.






Tags: Trunk based development, version control, code merge, development practice

List of blogs (Technical)

Puzzles with Java-1: Traveler and the coins
The design of everyday things- a gas stove
  Trunk based development- A journey