How to measure Kanbans effectiveness over Scrum?

Question

I am running an experiment with my team to see if they are more efficient overall using Kanban or Scrum.

We don’t have a specific reason other than we are striving for better.

If we don’t see any benefit using Kanban we will go back to Scrum after the experiment.

Can you recommend any ways of measuring the experiments success?

Thus far I will look at the following:

• the teams Spotify health results which I do every 10 weeks
• Average Lead time
• Average Cycle time
• I will interview the product development people and ask them to ask stakeholders

Is there anything else I should measure to have a good nice experiment?

Answer 1

Your list is really good already. I particularly like the last one; If the stakeholders and product people are happy, chances are things are going well.

Other things you might consider would focus on the elements that differ between Scrum and Kanban.

For example:

• How frequently do we need to make changes to priorities at short notice (if this happens a lot if favours Kanban)
• How well does the team and the stakeholders get on without a cadence of regular meetings - for example do the stakeholders come chasing updates
• Does the team ever run out of work that is ready to be started (the cadence of Scrum can help when refining work)

Answer 2

I would recommend that you reframe your hypothesis, and ask (your team) the question - "What is the problem we are trying to solve?"

Based on your question, it would appear that your team is a software development team and is already doing Scrum. You are running this experiment because you are either trying to solve specific known problems or trying to see how you can improve. Either way, they are GREAT reasons to try out Kanban.

Kanban - as defined by the Kanban Method - is not a software development or project management methodology. Rather, it is a way to (continuously) improve whatever it is that you are currently doing. If you see it that way, then your experiment would not be "Scrum OR Kanban?"; rather, it would (and should) be "Scrum AND Kanban!".

As you have already identified, Kanban can help you reduce Lead/ Cycle Time. Kanban does this by helping you identify flow bottlenecks (by identifying what stages of your workflow might be causing delays, and other impediments - or blockers - in your workflow), and enabling you to figure out how to change your processes and thus improve Flow (measured by Flow Efficiency).

Kanban helps you improve your team's ability to forecast its throughput and lead time, by giving you insights into their process variability and the causes for the variations - and eliminate/ reduce those causes gradually. It helps you improve your team's throughput (or velocity) with the help of WIP (Work-in-progress) Limits that enable your team to reduce multitasking and focus on completing what is already in progress before taking up new work.

In this manner of process improvement, you might find yourself doing away with some elements of Scrum (such as having a fixed 2-3 week cadence for making software releases), and if so, you might - like many other teams who have already done so - call your new and improved process "Scrumban" - or you might continue to "do Scrum" which has now been informed and improved by your application of Kanban principles. Either way, you would have arrived at a process that works best for YOUR team. Instead of a predetermined approach of "Scrum or Kanban?", it would be far more beneficial to follow the "What are we trying to fix - or improve?" approach.

Other aspects of Kanban (such as Upstream Kanban) provide great benefits for stakeholders (customers/leadership), product management and dev teams stay aligned on what needs to be built at any point of time. Even just using Kanban in upstream product backlog management and portfolio management would provide great benefits to the organization.