4

The project that I am working on is purely non functional and deeply technical in nature, focused on improving the performance of the product as a whole. I have trouble seeing how Scrum is an appropriate methodology for this kind of project:

  • The business person, aka the PO, has no idea what we are talking about during our plannings since all the work is deeply technical and has no user facing consequences.
  • As a result of the previous, he cannot manage the backlog and we have to create all the user stories and manage them.
  • The concept of a user story also does not seem to make any sense. In our project there is no user. All the interactions happen between different system components.
  • Estimations are quite hard to do since almost all of the work implies doing stuff that we did not do before, so assigning points to stories seems almost a useless exercise.
  • Delivery times of working software are usually in terms of months not weeks for most of the stuff we have to do since they involve a lot of investigation before even starting on doing something. So usually sprints mean not that much for us.
  • There is no customer with whom to check on progress, to give feedback on our work and adjust accordingly since it is purely performance related and for that we do our performance tests to see the progress.

So with all these points said, either I am missing some part of Scrum or is it true that it is not suited for this kind of project? And in general if true, what kind of projects are not suited for Scrum?

  • 2
    Can you go deeper into what you mean by "purely non functional"? A lot of typical non functional requirements can actually be transformed into a functional b requirements. For example, "high performance" can be transformed into "average response time for a request with 1 million currently active users is x milliseconds". This goes to the point that I have seen some people argue that bin functional requirements do not exist – Manziel Nov 15 at 10:30
  • 2
    "the PO, has no idea what we are talking about during our plannings since all the work is deeply technical and has no user facing consequences" Then you're talking about the wrong thing in your planning meeting. You're not supposed to solve the task in the meeting. You're supposed to state what needs to be done and why it's good for the business to have it done. If you can't do that then you shouldn't be doing the task. (Reducing technical debt counts as a good thing for the business.) – Lightness Races with Monica Nov 15 at 14:01
  • 2
    As for achieving abstractions, ehm, I don't understand, that takes literally no time at all. It's a way of speaking. If anything it should make your conversation in planning much much shorter. Instead of "we need to flob the bar and make xarg a char*", you say "we need to refactor this because it's too duplicated and it's wasting development time maintaining it". Bonus: that's also the business case. – Lightness Races with Monica Nov 15 at 17:40
  • 4
    It's not a specific mindset, it's literally how scrum works, and not only that but project and people management more generally, but okay you go ahead and good luck :) (I reviewed your question and "[am I] missing some part of Scrum" seems to fit well here.) – Lightness Races with Monica Nov 15 at 18:51
  • 2
    >>>the whole project was started from the business side in order to improve general performance of the platform<<< Performance in itself is not a business value. It starts to become a business value as soon as you know what you are doing with the better performance. Examples are improved user experience by shorter response times, simplified architecture because better performance removes the need for load balances, cost savings because the 5 servers do the work previously done by 10 servers, etc. That is what the non-technical product owner should decide – Manziel Nov 16 at 11:43
12

To your overall question, while Scrum can be applied in most projects, it is not necessarily the best approach for some projects. That said, it is well suited to complex problems that require discovery of the solution and adaptation to new information. Your project sounds like exactly the kind of project Scrum was designed to tackle. However, you raise some important points and I'll try to address them:

1) A product owner needs to understand the domain they are working in at a level that they can intelligently identify and discuss the key problems that need to be solved in order to create value. For example, the PO for the large hadron collider better know their quantum physics. It is completely possible that you have the wrong PO for your work. It is worth noting, however, that they do not need to understand how to solve the problem to be effective - that's the team's job. In fact, sometimes it's better if they don't so they can get out of the team's way.

2) While product backlog items can come from anyone, the PO must understand them enough to speak to their value and be able to prioritize. Are your backlog items expressions of problems to be solved or tasks in the solution. If the former, then again you may want to look at a different PO. If the latter, you might have the wrong items in your backlog.

3) First, you don't need to use user stories. However, you do have some person or group of people who benefit from your work. Your backlog items should each provide value to them. If a product provides no value to anyone, you should cancel it. (I am, of course, being hyperbolic. I've never actually encountered a project that provided no value to anyone)

4) Relative estimation is designed to be able to handle uncertainty or unknowns and helps in most projects. However, some projects have so much uncertainty as to make estimations useless. Luckily, Scrum does not require them. A recommendation I would make is that if you don't use estimations, use time boxes. A time box sets the amount of time to work on something before coming back to the group to see if it is worth continuing to pour time into or if something else is more important.

5) This is a common challenge. The solution is simple, but takes practice to get good at. The solution is to either a) break down the problem into smaller problems or b) run an experiment to gain validated learning rather than long investigative learning processes. The first is used in cases where the investigation cycle is long due to simply trying to investigate a lot of things. The second is used when the investigation cycle is long due to many complicating factors needing to be accounted for in complex problems. Of course, it is way easier to type this paragraph than to do it, so don't feel bad if you run into some problems you don't know how to tackle in a sprint.

6) Who wants higher performance, how much do they want and how will it impact them? That is who you have in your review. Your problem space sounds very narrow, so I would not be surprised to find out that your reviews are very straight-forward.

  • for both #4 and #5. In cases where extensive investigation/research is required and the overall tasks are currently too large and unknown to reliably estimate, breaking them down into smaller chunks is definitely the answer. To go along with both the timebox suggestion and to keep sprints meaningful... They can break the research out into a timeboxed task and have the results of the investigation be the deliverable for that task in one sprint. They can then use what they learned to more accurately chunk/estimate the implementation in a future sprint. – Mr.Mindor Nov 15 at 16:24
  • 1
    @Mr.Mindor is it really worth it all the overhead of spending tons of time of trying to(maybe even artificially) break down tasks into smaller tasks that can fit inside a 2 week sprint? It involves a lot of effort and time that ultimately could have gone to actually doing the work. What is the main benefit that makes it worth it? – dragosb Nov 15 at 17:10
  • 2
    @JimmyBreck-McKye that's a "task" that could take multiple people multiple months. I'd be deeply concerned if it couldn't be broken into sub-tasks. Of course, later tasks will probably evolve as earlier ones are worked, but that's probably fine. Leave far future tasks vague, and refine them as their prerequisites approach completion. – mbrig Nov 15 at 18:13
  • 1
    @dragosb to explicitly answer the last question from your comment: The main benefit(s) that makes it worth it are the same as main benefit(s) of following scrum at all! (in no particular order) Visibility/transparency (expectations are clear to stake holders and can be verified); increased Quality(fewer bugs for unforeseen situations); Flexibility(make informed decisions to abandon/change course); decreased Risk( see quality and visibility). improved productivity(waste less time on dead ends, reworking mistakes). – Mr.Mindor Nov 15 at 19:25
  • 2
    Going through the comments, I recall that common phrase on software development... "weeks of coding can save hours of analysis". – Tiago Cardoso Nov 16 at 18:45
5

Is Scrum actually suitable for all kinds of projects?

Like with many things in the software industry, Scrum is not a silver bullet. It works nicely for some types of projects, and less so for others. I've often seen the Cynefin Framework mentioned when trying to identify projects types where Scrum might be used, so maybe have a look at it and see under what category your project might fall under.

Another thing I've often seen, is Scrum being imposed on teams without considering the type of work being performed. From your question it seems that this might be the case. Who chose Scrum for you and your team? Why? Have you looked at other ways to organize your work? Would Kanban look more promising?

The problem with how you worded the question is that it mentions things against adopting Scrum, but doesn't say anything about the rest. Are these things major "no go"s or just something you identified? What I'm saying is that any project has its challenges. So treat this as a challenge, analyze it, and look for solutions, be those Scrum, Kanban, or something else.

Then, if Scrum can address the work, use it. But if you find something better, it's best not to "bend" your work to fit Scrum, just use the better thing you found. If on the other hand you don't have a saying in choosing the way you work and you need to use Scrum because some higher entity says so, then you have bigger problems than Scrum not being the best thing to use for your work.

  • 1
    "Are these things major "no go"s or just something you identified?" The items I listed are what I identified as things that do not make sense to me in the whole process we have to follow and which made me think that our project may not be suited that well for scrum. As from Daniels response I realize that some of the points are not actually part of Scrum, rather some "Scrum enhancements" that sadly are mandatory. – dragosb Nov 14 at 22:27
  • 2
    If any process is "mandatory" you are not doing scrum at all and might need to change your question. One principle of Scrum is that the team has the power to change any process or any ceremony. Agile Principle 12: "At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly." Manifesto value 1: "We have come to value Individuals and interactions over processes and tools" agilemanifesto.org – dcorking Nov 15 at 18:02
  • 1
    @dcorking While Scrum is an excellent container, you can't do whatever you like and call it Scrum. Scrum does have immutable aspects. --Don't mistake inspect-and-adapt for "cowboy agility." All agile frameworks and methodologies require adherence to some basic rules of engagement. The difference between inspect-and-adapt and cowboy agile is subtle, but it's very real. – Todd A. Jacobs Nov 16 at 19:01
4

Adding a bit to Daniel's excellent answer.

You say:

all the work is deeply technical and has no user facing consequences

But you also say:

focused on improving the performance of the product as a whole

I can think of two reasons why you might improve the performance of the product:

  1. To improve the user experience (quicker response times, etc.)
  2. To reduce the amount of hardware you need to get the same level of performance

If it is reason 1, then you do have user facing consequences and those will define your stories. For example, something like:

As a user I want the product to respond more quickly so that I can get more done

If it is reason 2, then the customer is likely internal and is somebody that wants to save money. For example, something like:

As a CIO I want to reduce my recurring hardware budget so that I can increase profitability

Note that these stories are perfectly understandable by somebody who is non-technical, such as a business person or Product Owner. They describe the thing that is wanted, rather than how that thing will be delivered.

  • 2
    Thanks for the input. The problem I find with the kind of stories you mention is that they are very generic. "As a user I want the product to respond more quickly so X" ...that would literally be a valid statement for all the work we have done for the past years and I dont see how we can have a backlog of these kind of stories. That sounds to me more like the general goal of the project rather than a user story, – dragosb Nov 14 at 22:20
  • 3
    The examples I gave are generic because I have little idea about the details of your situation. They can be way more specific: "As a user I want page load times to be under 10 seconds even during peak load times so that I enjoy my browsing experience". In fact how specific these stories are will reflect how well thought out the non functional requirements are. It can be tempting to ignore the 'why' aspect of non functional requirements, but really it is just as important as with functional requirements. – Barnaby Golden Nov 15 at 8:21
  • 1
    @dragosb Why do you need improving performance at all? Write down say five specific situations, where poor performance is creating problems (and what problems exactly) - if you cannot, then maybe it is not performance, but something else to improve? (I commented in context of this answer because Barnaby Golden suggests principially same approach.) – Arvo Nov 15 at 13:32
  • @Arvo I do not need it, the business needs it and this whole project was started for that purpose which effectively is replacing an old execution/computation engine with a new one. – dragosb Nov 15 at 17:04
  • @BarnabyGolden even the "specific" example you give still sounds very generic to me. Our goal is to have a general better response time and we achieve that by replacing the underlying execution engine. I really don't see how a user comes into the picture of doing this job since changes are completely transparent for them and the business value itself is generic...better performance which leads to cheaper hardware and more happy customers, this is the goal of the project not a user story. – dragosb Nov 15 at 17:08
4

TL;DR

The Scrum framework can usually be adapted to any product or service that can benefit from time-boxed effort and incremental delivery. That doesn't mean it's the best fit for every project, but the original question does not describe anything that can't fit into a Scrum implementatation.

Question Analysis

The question, as currently constituted, described a series of framework implementation smells that are collectively too broad to address within a single Q&A. As described, this project doesn't appear to clearly articulate a measurable business need that fits the empirical process control model of many agile frameworks. That is not a failure of Scrum or agile, but rather a flaw in one or more of the following:

  • project initiation
  • conceptualization or selection of project controls
  • organizational or project communications

The question, as currently written, is also presented as more of a confirmation-bias support request than a concrete problem to be resolved. As such, that aspect of the question is out of scope and will not be addressed within this answer.

What Scrum Was Designed For

The Scrum framework has only three clearly-defined roles and four prescribed events. Other than the mandatory framework elements called out in the framework, organizations are free to adapt it for their specific needs.

Scrum is primarily intended as a product-delivery framework. The guide itself calls out a number of concrete uses:

  1. Research and identify viable markets, technologies, and product capabilities;
  2. Develop products and enhancements;
  3. Release products and enhancements, as frequently as many times per day;
  4. Develop and sustain Cloud (online, secure, on-demand) and other operational environments for product use; and,
  5. Sustain and renew products.

So long as a project can be decomposed into iterative or incremental units that can be time-boxed, Scrum can be adapted for the use case. The original question strongly implies that the perceived failures of the Scrum model for the current use case have more to do with difficulties in conceptualizing the project as a set of time-boxed increments that each contain a central cohesion. This is most likely an implementation or experiential gap rather than a negative use case for the applicability of Scrum to a specific problem domain.

When Not to Use Scrum

There are certainly alternatives to using formal Scrum when the problem domain doesn't meet the theory or values of the framework. Such a list can never be exhaustive. However, there are certainly common negative use cases, including:

  1. Open-ended support or service delivery processes.
  2. On-demand processes (help desk or call center are two frequent examples).
  3. When just-in-time planning of increments is not feasible or desirable.
  4. Extremely short projects.
  5. Individual-contributor projects.
  6. Very small or very large teams.
  7. Deliverables that lack central cohesion.
  8. Deliverables that lack a measurable Definition of Done.
  9. Projects without active stakeholder collaboration.

If your problem domain doesn't fundamentally fit the Scrum model, other frameworks or methodologies may be a better fit. Which framework is "best" is going to be subjective, but any established framework should have clearly-defined design goals that you can use to guide your selection process.

  • 1
    "The question (...) is (...) a confirmation-bias support request than than a concrete problem to be resolved." This. There'll be no answers that will change one's mind if the OP is not willing to change. – Tiago Cardoso Nov 16 at 18:51
-2

Scrum is a buzz word among managers, especially those never doing actual coding/analysis/testing. It suits some specific projects, but not all. It is not suitable for many, these dealing with highly technical topics is one of them. Scrum is actually known for accumulating technical debt (is is easy to just shuffle these 'not nice' tasks somewhere down in the backlog). I think the best way for every project is to use a basic kanban and then tailor it according to the developer and project needs to either go more into the agile way or more rigid (which is not a bad per se, especially for high-risk projects). Mind you, waterfall is not a dirty word like all these scrum coaches, masters and gurus would like us to believe.

  • This contains false statements about Agile and Scrum. Try reading the Scrum Guide, you will be pleasantly suprised that it doesnt advocate any of that. Also the name Waterfall was the word created as a strawman by Agile practitioners - so yes that is exactly what it means. – Dave Hillier Nov 16 at 12:53
  • 1
    @DaveHillier Your attribution of "waterfall" as an agile pejorative is incorrect. There's an actual model called the Waterfall Model, and had its roots back in 1956. Even the first formal use of the term long-predated the agile movement, although you're correct that even in its original form it was intended to describe a model with known flaws. – Todd A. Jacobs Nov 16 at 14:41
  • 1
    My comment to @DaveHillier aside, he's correct that this "answer" is largely an unsubstantiated opinion piece. While "buzzword agile" can certainly be a problem, Scrum is not inherently a buzzword-only framework. Most Scrum failures are implementation or misapplication failures, rather than framework deficiencies. As such, this answer currently does not actually answer the question as originally posed. – Todd A. Jacobs Nov 16 at 14:44
  • @ToddA.Jacobs wikipedia makes the following claim: "The first formal description of the waterfall model is often cited as a 1970 article by Winston W. Royce,[3][4] although Royce did not use the term waterfall in that article. Royce presented this model as an example of a flawed, non-working model; which is how the term is generally used in writing about software development—to describe a critical view of a commonly used software development practice.[5]" I perhaps would have been better saying it is used as a strawman. – Dave Hillier Nov 17 at 0:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.