How can these problems with agile be solved?

Question

I'm a lead developer and for the last four years I've owned a code base which has steadily been improving in quality. I worked at a fairly small company and we previously worked in small waterfall cycles. Typically we'd do a release of our software every 3->5 months. We didn't use many formal project management methods but we had a good team and everybody was intensely collaborative. I spent the vast majority of my time writing features, fixing bugs and improving the code base as I saw fit. I refactored code when I thought we were starting to see too many bugs in that area and I focused on trying to make our code unit testable and of high quality. QA would pick up our work once features became dev complete but we didn't work in sprints. It's fair to say that things didn't always go smoothly and we did have some quality problems but productivity was very high.

Fast forward to now.. We've been bought by larger company who have already fully adopted agile methodologies. We've had to adopt their processes and some in our team were very keen to move to full agile. Initially, for a period of about 2/3 months our work ground to a halt.

• Decisions that were previously made quickly and efficiently, ad-hoc, by groups of interested individuals became part of the grooming process and we found that design by committee took forever compared to what we did before.
• A great deal of our time was taken up by grooming and planning meetings and at one point I realized that more than 50% of my time was now used for meetings.

Over time, during our retrospectives we've improved this process. One of the things we did was to get individual team members to pre-groom Product Backlog Items (PBIs) so that some of the thinking had been done in advance and we didn't have to spend quite so much time just thrashing out the basics of each item during the meetings.

However, some problems still remain.. Here are the big ones for me:

• Generalization and group think has reduced the extent to which anyone "owns" the code base. This has been really bad and it feels to me like the code base has been left to rot. Nobody can do anything of any significance to it without a massive discussion first and all those little tasks that used to just happen for free no longer happen. Instead specialists who used to do this stuff are busy doing other things which are the sprint priority even though they do them very slowly. There's basically no time left to put serious thought into our code. Over the longer term I think we'll pay a big price for this.

• Developer passion has gone.. Nobody wants to spend so much time in meetings and all the decision making power has evaporated leaving the devs demotivated. Decisions about what development tasks should be undertaken are now often taken by testers or others who don't actually understand the code and so now everybody just drudges on not really enjoying their work like they used to.

• Generalization is wasting our time and damaging our code. Right now I'm trying to learn a new programming language to fix one bug and bugs that I could be fixing in a language I know are being fixed by another scrum team by a guy who I don't think knows the code in that area well enough to be able to efficiently or effectively fix it. This kind of thing has already caused us some quality problems.

So, how can we fix this within Scrum? We can't get rid of the process, but can we change it somehow so that we can work effectively again?

Answer 1

Generalization and group think has reduced the extent to which anyone "owns" the code base.

That's bad. The goal is shared ownership, not nonownership.

Are you using slack time? Or aiming for 100% utilization? If the latter, change to the former.

During the slack time (one hour a day, one day a week, one week a month, whatever), developers are able to work on whatever they want - research, side projects, whatever.

Provided no one's already working on a feature, they could just pick one up and start working on it.

Just make sure you implement a code review before merging that work into the main codebase. You do not want some person going off on their own, doing something, and then shoving it into production code without anyone else knowing what it is.

Also, it was mentioned in comments already, but make sure the prioritization by the Product Owner is done at the story level, not the task level. The Product Owner decides what work gets done. The Development Team decides how that work gets done. If they don't want to have massive design-by-committee meetings to determine the how for each feature, then... that's their call. The Product Owner (nor the Scrum Master, unless s/he is also a developer, and even then the Scrum Master's words should not have any more weight than any other developer's) should not have say in that.

From the Scrum Guide:

They are self-organizing. No one (not even the Scrum Master) tells the Development Team how to turn Product Backlog into Increments of potentially releasable functionality;

So, if you don't like how you're turning items in the Product Backlog into 'Increments of potentially releasable functionality'... then just change it.

If you're not allowed to, then you're not really doing Scrum. Or Agile.

Answer 2

• Decisions became part of the grooming process and we found that design by committee took forever ...
• 50% of my time was now used for meetings. Nobody wants to spend so much time in meetings.

You should not be spending such an excessive amount of time in meetings. You are likely not timeboxing them. Ideally under Scrum these decisions should STILL be going on ad-hoc between the team members responsible for the corresponding tasks. Backlog grooming is not a design meeting. Its goal is to break items down into a size that can be sheduled a on what should be done (will we have user avatars? will we have multiplayer?) not how (Fidget-Architecture powered by Gadget-engine)

• Group think has reduced the extent to which anyone "owns" the code base. Nobody can do anything without a massive discussion first.
• Developer passion has gone. All the decision making power has evaporated leaving the devs demotivated. Right now I'm trying to fix one bug and bugs that I could be fixing are being fixed by another guy who don't knows the code in that area well enough.

SHTAP! You are seriously missing the point of crosstraining in a scrum team. The goal is that eventually every team member should be able to pick up any task with decent proficiency. But even then it would be insanity to "assign" a task to a team member who can barely do it when a specialist is available. Someone is trying to forcefully turn your team into interchangeable gears instead of allowing a natural cross-training to take place. That person needs to stop immediately!

Shared code ownership does not mean everyone must negotiate every decision with the whole team but that you consult with coworkers who know a piece of code better than you but you don't need their permission to touch it.

Answer 3

We always include a Feature "Technical Debt" in every release. The developers make user stories under the feature regarding code refactoring, code cleanup, framework updates, increasing code coverage etc. The Product Owner is made to understand that these catching up on our Technical Debt stories have to be done first and then we should build functionality on top of the existing code base.

Also, our releases also take 3-5 months. At the end of Release-1, the workload is more on Testing while at the beginning of Release-2, the workload is more on the Product OWner/BA side. These 2-3 weeks between Release-1 closure and Release-2 Dev/Test Iterations beginning, that's when this Technical Debt work kicks off.

Answer 4

You described all of issues of the planning process in larger than medium size IT companies which are common nowadays:

1. Extremely long planning meetings
2. Desire to move from code specialization to code ownership
3. High bureaucracy and fragility of the planning process

The reason of the first point is an attempt to "synchronize everything with everyone". Which then leads to the third point where schedules become irrelevant once feature priority of part of schedule changes, then this process needs to be repeated all over again (thus point #3). Second point is associated with a desire to move teams around and use for system components which are vital for business at particular point of time. I'd recommend to do the following changes:

1. Retain responsibility and specialization for teams, but make it "many to many". More knowledge allows teams to be faster in coding and development in particular system components. In case of scares resources for a particular component it's better to consider adding teams to the pool of specialized teams for this component rather than to pick teams for every task because this is much more stressful for teams and requires more time for learning and maintenance.
2. Each team has a Project Manager assigned who should be responsible for the projects this team is delivering. Project Manager should coordinate with other Project Managers in the company and manage team schedule
3. Each team has a Tech Lead assigned who should be responsible to provide high level decomposition, including level of effort and dependencies for projects his team is delivering. Tech Lead should be in tight collaboration with Project Manager and provide all necessary data for the projects they're requesting. Tech lead should be in collaboration with other tech leads to learn about system components they can reuse.
4. Developers, team leads need to be focused on coding and delivery of system components. They can't and don't need to know the whole planning picture, priorities and other information. They should execute their tasks based on information provided by project manager and tech leads.
5. Company Stakeholders should work with Project Managers to provide business priorities and create company roadmap, however this process should be in general highly automated. You need to use the tools which allow tech lead to enter decomposition, and for PM to quickly evaluate schedule based on that input and a particular team setting. Same with multi project schedules. Doing all of this manually and in meeting collaborations is highly ineffective and not agile friendly.

Answer 5

Re-establish Code Ownership

Problem: Generalization and groupthink have reduced code ownership, leading to neglect of code quality.

Solution:

• Assign code ownership: Designate specific team members as owners for different parts of the codebase. This can help maintain accountability and ensure that someone is consistently looking after each section of the code.
• Encourage code reviews: Implement regular code reviews where code owners review changes. This promotes shared ownership while still maintaining specialized knowledge.
• Create technical debt backlogs: Include technical debt as part of your backlog, so that code quality issues are systematically addressed alongside new features.

Reduce Meeting Overhead

Problem: A significant amount of time is spent in meetings, reducing actual development time and impacting morale.

Solution:

• Streamline meetings: Focus on making meetings more efficient. For example, set clear agendas, use time-boxing, and prioritize critical discussions.
• Leverage pre-grooming: Continue and enhance the practice of pre-grooming PBIs to minimize discussion time during meetings.
• Empower smaller decisions: Allow smaller, less impactful decisions to be made by individual team members or sub-teams to reduce bottlenecks and meeting overload.

Reignite Developer Passion

Problem: Developer motivation has declined due to loss of decision-making power and excessive meetings.

Solution:

• Delegate decision-making: Empower developers to make decisions about technical aspects and code improvements, within defined boundaries. This restores a sense of ownership and autonomy.
• Foster innovation: Allocate time for developers to work on technical improvements or innovations. This can be part of a sprint or a dedicated innovation time.
• Recognize and reward: Celebrate successes and recognize individual contributions to keep morale high and acknowledge the importance of their work.

Improve Specialist Utilization

Problem: Generalization is leading to inefficient use of specialists, causing quality issues.

Solution:

• Match skills to tasks: Ensure that tasks are assigned to team members with the appropriate expertise. Use skill matrices to align tasks with team members' strengths.
• Encourage knowledge sharing: Foster a culture of knowledge sharing where specialists document and share their expertise with the team, reducing the dependency on any one individual.
• Prioritize refactoring: Include refactoring and bug fixes as part of the sprint backlog to address quality issues proactively.

Enhance Scrum Practices

Problem: The Scrum process itself may not be well-tuned for your team's needs.

Solution:

• Adapt Scrum artifacts: Customize Scrum artifacts like the Definition of Done (DoD) to include code quality standards and ensure they are consistently met.
• Optimize sprint planning: Make sprint planning more focused by prioritizing tasks that have clear value and technical feasibility.
• Conduct effective retrospectives: Use retrospectives to continually improve processes. Actively address issues related to meetings, code quality, and team dynamics during these sessions.

By implementing these strategies, you can help your team navigate the transition to Scrum more effectively, maintain high code quality, and improve overall developer satisfaction. Adjusting Scrum to better fit your team's context and challenges can lead to a more productive and engaged development environment.