I'm trying to solve following problem we have in our company: We are trying to follow SCRUM We are using git-flow-like branching model. When we encounter bigger story that takes 2,3 sprints it's not merged back until all substories are done. As a result we have long running branch that is difficult to merge at the end. We start other stories based on a master branch which is completely not up to date dues to big feature-story branch. I proposed to merge early substories back to master whenever possible, but management said it's too risky as the changes might break deployments. How do you deal with a long running stories and branches?
The idea of Scrum is to finish work in sprints so that you can make transparent your progress. If you have stories that span 2-3 sprints then you will be hiding progress and making the Scrum framework less transparent.
It would be a good idea to focus on breaking the larger stories down. This can be a challenge at first, but it is a skill that Scrum teams need to acquire to be successful.
If you find you still need to maintain long-lived feature branches then consider the following approach:
- Merge feature branches back into master as frequently as possible.
- If it is not possible to merge back to master, consider merging from master into your branch. That way your branch will never get too far away from master.
- If you have more than one long-lived feature branch consider merging them together regularly. That way the risk of feature branch merge conflicts is reduced.
- Finally, consider using continuous integration for the merging process. It is possible to configure routine merges that give the team rapid notice of any merge conflicts. This means that the team will be solving merge conflicts soon after they write the code that causes them.
'When we encounter bigger story that takes 2,3 sprints'.
Don't work around that, fix it. You should be able to merge an INVEST story. Notice the S stands for small. Make your substories valuable and mergable. Having monolithic stories is a huge (but very common) problem. I suspect your sub-stories resemble stages in a technical design, rather than deliverable increments. That's not quite right. People write books about this sort of thing, but I'll just link you to a very simple and useful flowchart for how to approach subdividing your stories.
'We start other stories based on a...' You have too much work in progress (WIP) and this is one of the minor annoyances. Get better at collaborating and pile onto fewer stories.
'but management said it's too risky'.
Firstly this might be related to your WIP problem. When you finish your multi-sprint story (epic might be a better word), there's always some other half done work in flight as well. They don't want that all in master, but the solution isn't to take it out, it's to have less WIP.
Secondly, the stories you pick up each iteration need to be done (Dev, QA in test environments etc) using the skills inside your team. You need to release every iteration, that's the point. It's how you deliver value and get customer feedback quickly. You need to push back against this. But you also need to make sure that you have high quality standards (ie lots of test automation) so that people don't feel that taking your code is a big risk.
TLDR The solution isn't cunning branch management and git can't save you. Split your stories, and read up on what iterations are mean to achieve. Or, I suppose, budget for horrible merges.
We use feature toggle technique ( [Wikipedia feature toggle]: https://en.wikipedia.org/wiki/Feature_toggle ) and so far we have been very effective in being able to deliver in time, handling large feature gracefully and turning off features that are not done by the release date.
A feature toggle, (also feature switch, feature flag, feature flipper, conditional feature, etc.) is a technique in software development that attempts to provide an alternative to maintaining multiple source-code branches (known as feature branches).
Continuous release and continuous deployment provide developers with rapid feedback about their coding. This requires the integration of their code changes as early as possible. Feature branches introduce a bypass to this process. Feature toggles bring developers back to the track, but the execution paths of their features are still "dead" if a toggle is "off". But the effort is low to enable the new execution paths just by setting a toggle to "on".
The technique allows developers to release a version of a product that has unfinished features. These unfinished feature are hidden (toggled) so they do not appear in the user interface. This allows many small incremental versions of software to be delivered without the cost of constant branching and merging.
Another benefit of feature flags are canary launches. A canary launch (aka canary deployment or canary release) is rolling features out to a small number of users to assess the reaction of the overall system. A canary launch allows you to roll out a feature slowly, and measure the reaction from real user “canaries”, looking for early indicators of danger. If a feature is not good, it can be rolled back. Canary launches are a best practice for agile development organizations practicing continuous delivery to move faster.
We define shipable feature as an minimum viable product MVP, that usually will be translated as an Epic that can spread across many sprints or within a Sprint (our sprint is a week long), an Epic will be broken into several INVEST user stories, high level and customer-centric. Any finished user stories get merged into the the master branch on a daily basis.
Others have addressed the fact that your stories are too large, but if you break them down (and you should), you'll have done stories that may not quite be ready for "prime time". You don't want to deploy these half done features and neither does your marketing department.
But here's the secret...
You do want to deploy them. It's your marketing dept that doesn't want to. And you're both right.
Marketing doesn't want half done features going to prod, but you want to merge these changes in and deploy so that it's easier. So you do both.
Write in a feature toggle for these long in progress features. Merge your stories in when they're done, just like any other story. Now, for testing/demo purposes, you turn the feature on, but turn it off for production. This allows you to frequently merge, ship the feature to your QA & Product Owner for fast feedback, yet leave it "unshipped" for your end users.
How to do that could easily fill a series of blog posts, so I'll leave further research on feature toggles to you.
Funny reasoning, your management. Merging the branch early risks breaking deployments, so it's better to add lots more code to that branch before merging it. How exactly, pray tell, does adding many weeks of code decrease the risk of breaking the deployment?
Automate your deployment and deploy often to a testing environment. Aim for daily but more frequent if you can. That way your 'broken deployment' is always caused by the few lines of code that you just wrote. Easy to fix.
Your management has a sense of humour.
Merging an outdated branch to master is easier if you create a new directory structure for the new feature, then do an interactive merge rather than an automatic merge:
Writing inline assertions helps immensely for large stories. Have a code review of the assertions and generate a test report at the end of each sprint to show progress. This avoids the need for:
- Scope creep due to a smoke and mirrors demo
- Requirement gaps due to creating APIs under pressure
- Security issues due to delayed code review
- Last minute refactoring due to false assumptions