How to complete many development phases within 2-week Sprints

Question

I work in a tech company, a few weeks ago we moved from traditional project management to Scrum multidisciplinary Teams.

Our sprints' length is two weeks and the experiment is not working well for now, for some reason we don't achieve the goal at the end of the Sprints.

• We consider that a user story can be started when the problem is defined and concrete acceptation criteria is defined as well.
• We consider that a user story is finished when a version, potentially deployable to production with that feature, is created (no further phases needed, only physical deployment).

Our work phases are:

1. Demo, retrospective, planning (meeting day) -> 1 day
2. Team talks about the solutions we are going to implement -> Usually 1 day to review possibilities, see how others are doing...
3. When the Team agrees with the solution, the developers may spend 1 day or two investigating some new technology we are going to use -> 1 or 2 days
4. When we know the implementation details of the new technology, we start the implementation knowing the final flows and architectural details -> 3 - 5 days.
5. When implementation is done, we start writing the unit tests, user interface (UI) tests, integration tests... -> 2 - 3 days.
6. When tests are done and new code is covered, we start the code review phase...a reviewer from another Team must be available to check, then we solve any reviewed problems... -> ideally 1 day
7. OK now implementation is done, code review passed, tests written...new feature is sent to a Team QA member to perform final tests and verifications -> 1-2 days.
8. When test passes, the feature is delivered to the Product Owner who performs a User Validation Test -> 1/2 day
9. OK now the feature is ready to merge with the develop branch and a new version is created (and maybe released to production) -> 1 day

Problems:

• If you count, in the best scenario (completing the feature in 3 days and doing tests in 2 days) we spend 11 days to complete the user story...in the worst case we can spend 16 days or more depending on the complexity of the user story...in both cases we exceed the length of the sprint.
• We could remove phases...but we give importance to quality so writing tests, doing code review, doing a final test phase is relevant to us.
• We will never be able to complete user stories which need more than 4 or 5 days of 'active development'...

What are we doing wrong?

Answer 1

What are we doing wrong?

You are tackling backlog items that are too big for your sprint length and you are wasting resources by working in traditional phases.

You need to break down your backlog items even further than you are now. Your situation is the prime example why this is so important. Imagine you were to break down everything into items roughly a third the current size. Then you would know for sure that you should at least be able to finish 2 of them in one sprint. Instead of finishing 1 maybe, maybe not...

Here is a good article on strategies to break down backlog items. The gist of it:

• Vertical slices are still king
• But sometimes that vertical slice has a heavy bulge somewhere in the middle (multiple ways to do the same thing / less common scenarios). You can split that bulge if you need to divide it further.
• If you can, split less important aspects away from more important aspects
• All else being equal favor splitting into similar sizes

You also seem to have adopted something of a design-by-committee approach and it is eating time. You don't need consensus on how to solve a problem. Whoever picks up a task has the authority to decide how it is done and responsibility to do it right. If they feel they need or would benefit from someone elses insight then they can consult with that team member ad hoc.

If you have significant uncertainty behind a backlog item (like a new technology or unclear strategy), consider planning a research spike instead of putting a lot of time into researching possible paths during the actual backlog item. This may not always be visible beforehand but this is one more way to break the item down. You will always have to look up a few new/unfamilliar things but those should be highly specific details not 101 tutorials because you've never touched that library before.

According to your step 7 your QA team members only seem to get utilized late in the sprint. If so this wastes much of their capacity. If so you need to integrate them earlier. They should cooperate with the more development oriented teammembers from the beginning. A good way to do that is to have them start writing unit tests as the code is taking shape. Another way to reduce this problem is again, to reduce backlog item size. The smaller the items the earlier QA gets their first candidates.

Similarly your step 8 looks like another potential bottleneck, depending on one person to move everything forward. Again this would be helped by smaller items and by distributing the work over the whole team.

Merging would also be simpler if it could be distributed throughout the sprint instead of everyone scrambling to get their changes in on the last day. Smaller items will help with that. Using smaller items would also lessen the need to run dedicated branches instead of just using the local repo as a soft branch (I assume a DVCS is in use).

Similarly a build script/server could help reduce the time needed to create the final version. Though this benefit doesn't come for free unlike several of the others here.

That's several days you can outright scrap or interweave with other concurrent work. The rest you'll have to do with smaller items.

Answer 2

This is a great question and a common problem when teams first make the switch.

As a rule of thumb, you probably want to target any given backlog item taking no more than half the sprint length. This is by no means a hard and fast rule, it's just a helpful guideline.

There are two sides to addressing this. The first is to shrink the size of backlog items. Everything can be broken down smaller. I've worked with teams who finish any given user story in a few hours (from start to deployment). It takes time to get there, so if you don't get it at first, that's ok. This video is a bit long, but I still feel like it's one of the best at describing user stories. In my experience, it is far easier to break down an item from the point of view of solving user needs than it is to break down an implementation-focused backlog item.

The other is the team breaking up the traditional cycle. Is the whole team sitting together? I'd look at XP at some practices that might help break down those phases. Pair programming or even small-group development with developers, testers, and even stakeholders can take away late time and break down artificial barriers. ATDD and TDD puts testing first, which will inherently disrupt the normal process and force the team to organize a new way of working.

Answer 3

To directly address the branching question: Keep the number minimal. One master/trunk/main that is the production code base, and one development/edge that is used by all developers with Continuous Integration and Continuous Deployment (CI/CD) may be enough. Mo' branches, mo' problems; reduce the number of branches, reduce the merge maintenance.

As a framework, Scrum is not prescriptive about the details for creating the product. Due to contextual differences, each Scrum Team and Development Team have to find the techniques and practices that help them to be effective and efficient in serving the customer with a highly valuable product of great quality. As Daniel discussed, making the transition can be challenging. He also mentioned several common practices, many from eXtreme Programming (XP), that have been successful for many over the years.

Collaboration is key. The first value and several principles of the Manifesto for Agile Software Development highlight that aspect. Quality is also a high priority. Silos and walls between development, quality assurance, design, infrastructure, etc. are great impediments. A test-first mentality can be helpful in making that transition. By working together from the onset, quality will improve and cycle times will be reduced. As you have noticed, the ScrumWaterFail that you are currently executing is not working; it is a common pattern for groups new to the agile philosophy.

Product Backlog Refinement (~8 hours throughout the Sprint as needed) is not an official Scrum event. However it is a foundation for each Sprint's effort. The various skills that each member brings to the team are invaluable for examining the top Product Backlog Items from different perspectives in order to create a holistic understanding. The result is less noise and confusion which reduces the need for hashing out basics during the Sprint when bringing the forecast PBIs to 'Done' is the Focus. This is a great opportunity for applying vertical slicing techniques. How do you eat an elephant? One bite at a time.

Sprint Planning (~4 hours to begin the Sprint) utilizes the shared understanding of the top Product Backlog Items achieved during refinement when the Development Team forecasts what they believe they can accomplish. Collaboratively a sketch is created for how the effort will be approached. The first items to be addressed should be planned with greater detail with the quality strategy as a priority. This is a good time to outline test scenarios.

During the Sprint, two or more individuals with mixed skills should work together collaboratively on completing a PBI using pair and mob programming reducing the need for code reviews.¹ The number of PBIs being worked on concurrently should be kept to a minimum: A Development Team is 3-9 people, so only 1 or 2 (maybe 3) in progress items. New tests for the item should be developed locally, executed often, then added to the shared suite. Code should be committed frequently to avoid merge hell; merges don't create conflicts, they reveal them. Ideally tests should be executed by automation in the Continuous Integration and Deployment pipeline. Issues will be discovered sooner and should be addressed quickly. There are no hand offs between people or groups.

As collaboration increases, the Daily Scrum ( < 15 minutes daily) will become less about sharing progress of work, because people are having discussions throughout the day's activities, and will become more about validating the forecast from Planning. Because Quality goals do not decrease it should be more acceptable to reduce scope over cramming to complete all items. The only failed Sprint is one that doesn't produce a 'Done' Increment.

Demonstrating that Increment is only part of the Sprint Review (~2 hours near the end of the Sprint) which also provides feedback on quality and value. This is a great opportunity to connect with users and stakeholders. This is not a Pass/Fail phase gate. As a an important aspect of quality, the Product Owner should be available to answer questions and review 'Done' work throughout the Sprint.

Sprint Retrospective (~2 hours to close the Sprint) will provide an opportunity to inspect and adapt the collaborative effort of the Scrum Team. It is also the event where the Definition of 'Done', usually the expression of quality, is improved for future Sprints. It is important to Inspect how the last Sprint went with regards to people, relationships, process, and tools because each is a factor in providing quality, and working effectively and efficiently. Create a safe environment for candid discussion throughout the Sprint, and especially during the Retrospective.

^{All times based on the stated two-week Sprint time-box. Feel free to chat. HTH}