What are some best practices for deploying bug fixes to multiple code bases?

Question

Just a bit of a background, I am working to manage a big legacy system that has at least 32 branches for it's sub systems.

The challenge is how to manage and not miss required change points or defect fixes to be deployed to different branches. Currently the team has a 30% chance that the deployment is missed, and I am targeting to make this close to zero as much as possible.

I use JIRA for bug tracking and the process goes as follows:

1. Fixed defect/completed implementation activity is reported by a technical leader in charge to PM
2. PM asks for expert judgement from technical leader on which branches is it necessary to be deployed

3. Based on the expert Judgement PM reviews, creates the records and updates a deployment log similar to the following.

4. The created records are then assigned and processed to be implemented.

Current known problems with the said process are

1. Some branch deployment implementation are past the deadline when a defect is detected and gets rejected for release. This causes some fixes to have been logged in the deployment log to have inconsistencies with them, saying they are deployed but not really deployed. I am thinking of creating an additional file where these items can be tracked.
2. PM and technical leader gets very busy and this task of filling up for 32 branches take 4 hours (at least) to analyze and fill up. This yields rushing analysis work by technical leaders and further potential misses. Also it implies costs to the project's budget and development timeline.

My question is at what part of the process is it best to improve on? Or what points of this process needs improvement?

Answer 1

Your known problems already have a fix planned. It's called the "legacy code base" for a reason.

One obvious big red flag is the fact that your system tracks deployments that have been rejected as "deployed" even if they aren't. This is something you should be able to fix in your process immediately, because this has nothing to do with the number of branches at all. If it happens for one branch of many, it would be happening if you had no branches at all too. we cannot help yo with it, that is your organisational structure, workflow and ticketing system. It seems that this would be your primary concern.

Once this is fixed and your reports don't lie anymore, you will have to face one hard truth: the system is too big to be handled in the time management wants it to be handled.

There is very little you can actually "improve" as in gain some obvious benefit for little cost. 4 hours to make 32 technical decisions is already fast, to the point mistakes are made. If you want to get your error rate down, you have to put in more effort there. So it will take even longer.

Whether that's worth it, where the sweet spot between time spent planning and time spent fixing the planning errors later is is up to you to figure out. Gather facts, measure times or work and prepare a plan accordingly.

Since you did not mention the implementation time of the bugfix in all branches as a problem, I don't think there is any technical silver bullet to get you out of this. You want to get the errors down? Then you need more effort to not make them in the first place.