Is Scrum actually suitable for all kinds of projects?

Question

The project that I am working on is purely non functional and deeply technical in nature, focused on improving the performance of the product as a whole. I have trouble seeing how Scrum is an appropriate methodology for this kind of project:

• The business person, aka the PO, has no idea what we are talking about during our plannings since all the work is deeply technical and has no user facing consequences.
• As a result of the previous, he cannot manage the backlog and we have to create all the user stories and manage them.
• The concept of a user story also does not seem to make any sense. In our project there is no user. All the interactions happen between different system components.
• Estimations are quite hard to do since almost all of the work implies doing stuff that we did not do before, so assigning points to stories seems almost a useless exercise.
• Delivery times of working software are usually in terms of months not weeks for most of the stuff we have to do since they involve a lot of investigation before even starting on doing something. So usually sprints mean not that much for us.
• There is no customer with whom to check on progress, to give feedback on our work and adjust accordingly since it is purely performance related and for that we do our performance tests to see the progress.

So with all these points said, either I am missing some part of Scrum or is it true that it is not suited for this kind of project? And in general if true, what kind of projects are not suited for Scrum?

Answer 1

To your overall question, while Scrum can be applied in most projects, it is not necessarily the best approach for some projects. That said, it is well suited to complex problems that require discovery of the solution and adaptation to new information. Your project sounds like exactly the kind of project Scrum was designed to tackle. However, you raise some important points and I'll try to address them:

1) A product owner needs to understand the domain they are working in at a level that they can intelligently identify and discuss the key problems that need to be solved in order to create value. For example, the PO for the large hadron collider better know their quantum physics. It is completely possible that you have the wrong PO for your work. It is worth noting, however, that they do not need to understand how to solve the problem to be effective - that's the team's job. In fact, sometimes it's better if they don't so they can get out of the team's way.

2) While product backlog items can come from anyone, the PO must understand them enough to speak to their value and be able to prioritize. Are your backlog items expressions of problems to be solved or tasks in the solution. If the former, then again you may want to look at a different PO. If the latter, you might have the wrong items in your backlog.

3) First, you don't need to use user stories. However, you do have some person or group of people who benefit from your work. Your backlog items should each provide value to them. If a product provides no value to anyone, you should cancel it. (I am, of course, being hyperbolic. I've never actually encountered a project that provided no value to anyone)

4) Relative estimation is designed to be able to handle uncertainty or unknowns and helps in most projects. However, some projects have so much uncertainty as to make estimations useless. Luckily, Scrum does not require them. A recommendation I would make is that if you don't use estimations, use time boxes. A time box sets the amount of time to work on something before coming back to the group to see if it is worth continuing to pour time into or if something else is more important.

5) This is a common challenge. The solution is simple, but takes practice to get good at. The solution is to either a) break down the problem into smaller problems or b) run an experiment to gain validated learning rather than long investigative learning processes. The first is used in cases where the investigation cycle is long due to simply trying to investigate a lot of things. The second is used when the investigation cycle is long due to many complicating factors needing to be accounted for in complex problems. Of course, it is way easier to type this paragraph than to do it, so don't feel bad if you run into some problems you don't know how to tackle in a sprint.

6) Who wants higher performance, how much do they want and how will it impact them? That is who you have in your review. Your problem space sounds very narrow, so I would not be surprised to find out that your reviews are very straight-forward.

Answer 2

Is Scrum actually suitable for all kinds of projects?

Like with many things in the software industry, Scrum is not a silver bullet. It works nicely for some types of projects, and less so for others. I've often seen the Cynefin Framework mentioned when trying to identify projects types where Scrum might be used, so maybe have a look at it and see under what category your project might fall under.

Another thing I've often seen, is Scrum being imposed on teams without considering the type of work being performed. From your question it seems that this might be the case. Who chose Scrum for you and your team? Why? Have you looked at other ways to organize your work? Would Kanban look more promising?

The problem with how you worded the question is that it mentions things against adopting Scrum, but doesn't say anything about the rest. Are these things major "no go"s or just something you identified? What I'm saying is that any project has its challenges. So treat this as a challenge, analyze it, and look for solutions, be those Scrum, Kanban, or something else.

Then, if Scrum can address the work, use it. But if you find something better, it's best not to "bend" your work to fit Scrum, just use the better thing you found. If on the other hand you don't have a saying in choosing the way you work and you need to use Scrum because some higher entity says so, then you have bigger problems than Scrum not being the best thing to use for your work.

Answer 3

Adding a bit to Daniel's excellent answer.

You say:

all the work is deeply technical and has no user facing consequences

But you also say:

focused on improving the performance of the product as a whole

I can think of two reasons why you might improve the performance of the product:

1. To improve the user experience (quicker response times, etc.)
2. To reduce the amount of hardware you need to get the same level of performance

If it is reason 1, then you do have user facing consequences and those will define your stories. For example, something like:

As a user I want the product to respond more quickly so that I can get more done

If it is reason 2, then the customer is likely internal and is somebody that wants to save money. For example, something like:

As a CIO I want to reduce my recurring hardware budget so that I can increase profitability

Note that these stories are perfectly understandable by somebody who is non-technical, such as a business person or Product Owner. They describe the thing that is wanted, rather than how that thing will be delivered.

Answer 4

TL;DR

The Scrum framework can usually be adapted to any product or service that can benefit from time-boxed effort and incremental delivery. That doesn't mean it's the best fit for every project, but the original question does not describe anything that can't fit into a Scrum implementatation.

Question Analysis

The question, as currently constituted, described a series of framework implementation smells that are collectively too broad to address within a single Q&A. As described, this project doesn't appear to clearly articulate a measurable business need that fits the empirical process control model of many agile frameworks. That is not a failure of Scrum or agile, but rather a flaw in one or more of the following:

• project initiation
• conceptualization or selection of project controls
• organizational or project communications

The question, as currently written, is also presented as more of a confirmation-bias support request than a concrete problem to be resolved. As such, that aspect of the question is out of scope and will not be addressed within this answer.

What Scrum Was Designed For

The Scrum framework has only three clearly-defined roles and four prescribed events. Other than the mandatory framework elements called out in the framework, organizations are free to adapt it for their specific needs.

Scrum is primarily intended as a product-delivery framework. The guide itself calls out a number of concrete uses:

1. Research and identify viable markets, technologies, and product capabilities;
2. Develop products and enhancements;
3. Release products and enhancements, as frequently as many times per day;
4. Develop and sustain Cloud (online, secure, on-demand) and other operational environments for product use; and,
5. Sustain and renew products.

So long as a project can be decomposed into iterative or incremental units that can be time-boxed, Scrum can be adapted for the use case. The original question strongly implies that the perceived failures of the Scrum model for the current use case have more to do with difficulties in conceptualizing the project as a set of time-boxed increments that each contain a central cohesion. This is most likely an implementation or experiential gap rather than a negative use case for the applicability of Scrum to a specific problem domain.

When Not to Use Scrum

There are certainly alternatives to using formal Scrum when the problem domain doesn't meet the theory or values of the framework. Such a list can never be exhaustive. However, there are certainly common negative use cases, including:

1. Open-ended support or service delivery processes.
2. On-demand processes (help desk or call center are two frequent examples).
3. When just-in-time planning of increments is not feasible or desirable.
4. Extremely short projects.
5. Individual-contributor projects.
6. Very small or very large teams.
7. Deliverables that lack central cohesion.
8. Deliverables that lack a measurable Definition of Done.
9. Projects without active stakeholder collaboration.

If your problem domain doesn't fundamentally fit the Scrum model, other frameworks or methodologies may be a better fit. Which framework is "best" is going to be subjective, but any established framework should have clearly-defined design goals that you can use to guide your selection process.

Answer 5

Scrum is a buzz word among managers, especially those never doing actual coding/analysis/testing. It suits some specific projects, but not all. It is not suitable for many, these dealing with highly technical topics is one of them. Scrum is actually known for accumulating technical debt (is is easy to just shuffle these 'not nice' tasks somewhere down in the backlog). I think the best way for every project is to use a basic kanban and then tailor it according to the developer and project needs to either go more into the agile way or more rigid (which is not a bad per se, especially for high-risk projects). Mind you, waterfall is not a dirty word like all these scrum coaches, masters and gurus would like us to believe.