Team consistently over commits

Question

I'm a developer on a small-ish team. 1 DBA, 2 Senior Devs, 2 Juniors, and 1 QA (sometimes, most of the time we do our own QA). No PM at the moment. We do mostly maintenance/upgrades to a large legacy system right now, but will likely split the team soon so a group of us can focus on new product development.

We're a relatively new team that's been together for about 6 iterations now (3 months of 2 week iterations). Although we're using two week iterations, we're not down right time boxing. If a task is in progress, it stays in progress as it was already deemed the highest priority item. The iteration is more of a way of building in a time to retrospect, course correct, groom the backlog, and measure our capacity. It's Kanban with some ceremony stolen from Scrum.

So, this all works fine for this maintenance project, but with the new product development, we're going to have to really commit to getting the tasks done that we say we will. The problem is that the team is consistently overly optimistic during our planning session. We've never gotten all of our stories completed, regardless of the fact that we know exactly how much work we can do, on average.

We're able to consistently complete 13 stories in an iteration, but the team keeps insisting on scheduling 15-18. Any advice on how I can get them to understand the importance of scheduling to our capacity and committing to our stakeholders?

---

I'm sure someone will ask about story size/points. I don't quite trust scheduling to average points yet. The number of points completed in an iteration vary too wildly yet for scheduling purposes.

Answer 1

I don't know very much about agile and scrum and iterations and stories but coming from a more general point of view around estimation, statistics, and human factors, I am not sure you have enough data to assert that you even have an issue of optimistic planning. You have only performed six iterations and you are only experiencing a three to five story variance. This is very likely right in the sweet spot of the probabilistic distribution where your next six iterations you may experience a three to four favorable variance.

Look at a fair coin toss as an example. You have almost a 10% chance of throwing five consecutive heads in six throws despite a coin being 50/50. Compare this with throwing 50 heads in 60 throws with a chance of 0.000006%. With only six iterations, you might just be experiencing a string of bad luck.

Optimism biases certainly affect estimation and you need to be aware of this phenomena in planning. You don't want to chase, however, statistical noise so you need to collect data, a lot of it, to determine you have a real issue versus something that is random. And, finally, you have to battle the opposite phenomena which is that work tends to grow in available time--Parkinson's Law--where pessimistic planning becomes the norm and you become inefficient and costly. Aggressive, optimistic planning can force the team to remain highly efficient and nimble despite the harsh fact that they may miss their targets.

For example, if you target $500 to do a job and you overrun your target by $100, is this more favorable then targeting $800 to do the same job and underrun by $100? $600 versus $700. Overrunning an aggressive target has its place but it does take a sophisticated customer to understand that.

Answer 2

here are few suggestions:

1. Prepare some statistics for your team for the next retro, better visual. You had 6 iterations already, so I assume you have data to show. It should help to convince them.
2. Suggest to take smaller amount of stories for the sprint (e.g. your historical 13 stories), but if they have time - they can take stories from the backlog.
3. Simply count the estimates for stories vs team available time (effective development time which is obviously not 8 hours per day). all that goes beyond of available time - waits in the backlog.

Answer 3

The Scrum guide talks about a forecast instead of commitment. (They changed this from commitment in 2013.)

However, enough work is planned during Sprint Planning for the Development Team to forecast what it believes it can do in the upcoming Sprint.

It is not so bad to not complete your teams full forecast. What is important is that the highest priority items did get completed. Thats why you need too prioritize your iteration backlog and stick to that order. Focus together on completing the tasks and prevent all tasks from being nearly done. The thing with software development is that things are done when they are done. Just don't try to over engineer and understand when good is good enough.

Stack holders can be sure the top items will be finished each iteration. This should be good enough :)

Personally I like to plan a little bit extra, since this makes sure the team can continue on work even when they get stuck or if they overestimated the other tasks. Nothing wrong with working out requirements/acceptance criteria a little bit ahead.

Answer 4

Thank you for the interesting question. I wonder, what is the reason people might want to take too many items into the sprint/iteration. I can think of several reasons, may be it's one of these:

• team members are afraid, they will have to wait on others => they rarely co-work around items
• team members need to show activity to upper management
• team members don't know what to invest their time into, if they have slack time => not seeing potential for investing in technical debt/learning, etc.
• team members aren't supposed to do the above
• they expect to improve velocity, but continuously fail in doing so.
• ...

I believe you have to dig a little deeper for the root cause here. Then you'll know what to do.