1

Say you have a piece of software that you know can be improved on various quantifiable metrics (time to boot, CPU and memory usage, latency for various actions, accuracy of recognition tasks, etc.), but you don't know by how much - how would you write User Stories?

Is it better to guesstimate an improvement? (ideally with the team) To just write "improve such-and-such metric", without specifying by how much?

(Our organisation is new to Agile and we're trying out Scrum, and I've found we spend a bit too much time discussing these details and the meta topic of how to formulate them, so I'd like to know how others would cover this; we don't have specific formal requirements from our client on those, but know (from UX testing etc.) that those improvements are needed for a better user experience)

  • YAGNI. If you don't have a concrete customer need, then you don't have a viable project target. Ergo, you shouldn't be doing these sorts of vague stories at all. – Todd A. Jacobs Aug 19 '14 at 11:28
  • CodeGnome: My question is precisely about how to make good stories. Say you're making a game, and starting a level takes two minutes, which frustrates playtesters. The user need is concrete ("goddamit I want it to load faster!") but vague. – Emile Aug 19 '14 at 14:30
4

Performance improvements are often an exercise in cost-benefit trade-off. The executive who "just wants it to be faster" might find adding one server to boost speed by 50% an acceptable cost, but not two more servers for the next 20% boost. Also, there are usually many possible technical approaches to boost performance -- code refactoring, infrastructure changes, switching out components. So some degree of research and exploration is often required to discover the optimal solution for your current technical and business situation.

I would suggest you start with a time-boxed spike to brainstorm, research and possibly experiment with some possible approaches. (Sometimes this can even reveal some no-brainer quick fixes.) Then report back to stakeholders about the options, each with rough estimates of cost (in time or money) and expected benefit. Then a decision can be made on which to implement. At that point, the stories can essentially be in the form "do option B." Make sure to benchmark the metric before and after to determine if the expected costs and benefits were on the mark. Then a decision can be made about whether to proceed with additional steps. 

2

TL;DR

You can't invent meaningful targets out of whole cloth. Pragmatically speaking, both business analysis and user experience requirements all require forming a hypothesis, testing the hypothesis, and then drawing an actionable conclusion (e.g. your specifications) from the results.

Your team (and particularly your Product Owner) don't appear to have the right processes in place to gather meaningful specifications about non-functional requirements related to the user experience.

Gathering Requirements is a Real Job

My question is precisely about how to make good stories. Say you're making a game, and starting a level takes two minutes, which frustrates playtesters. The user need is concrete ("goddamit I want it to load faster!") but vague.

Good requirements gathering often entails experience in performing multiple drill-downs until you get to actionable specifications. "The load time is too slow" is not an actionable statement. However, a management target based on industry research (e.g. "load times must be less than 8.3 seconds to maintain engagement") or actual experimental data can certainly be turned into user stories.

A Product Owner should be working with Business Analysts (BAs) and User Experience (UX) designers to construct tests. For example, the UX team might develop a minimum viable product with load times of 5 seconds, 10 seconds, and 20 seconds. A good UX team would then gather data in the form of opinions or testable human behaviors to determine how "sticky" the product is at different load times to see what level of load time leads to actual disengagement or lack of interest in the product.

User Stories Are Not a Substitute for Understanding the Knowledge Domain

User stories are conversational placeholders between the development team and the Product Owner or end users. They are not specification documents, nor are they a substitute for essential business skills. A great cross-functional team may include UX expertise within the team, but most often requirements-management of this kind is really external to the team, and proxied by the Product Owner.

In other words, your problem isn't that you are having trouble crafting the user stories. The real problem is that your team (and particularly your Product Owner) doesn't have the right processes in place to gather meaningful specifications about non-functional requirements related to the user experience. Whether internal or external to the team, the organization needs to ensure that those processes exist and are properly hooked into the product development process.

  • Thanks for the details about how could work. I'm not sure it's just a question of process though, there's also a question of Value of Information; i.e. if the UX or BA people come up with detailed numbers, it's not always clear that they will make a difference - to stick to the game analogy, if your game is supposed to be ready to ship in two months and the loading time can't realistically go below 30 seconds without a huge rewrite, UX tests aren't going to be a very useful guide to action. Especially if everybody knows Fred is going to be optimizing loading anyway in the next months. – Emile Aug 20 '14 at 17:02
  • I guess my original concern was a case where the exact value doesn't seem that important to the team, but we're wondering whether we should put a detailed value anyway (it makes estimation and validation simpler). What you describe seems like how to handle a case where the exact value is actually super important (and I agree that it happens!). – Emile Aug 20 '14 at 17:07
  • I think this answer is helpful when thinking of the ideal situation, but in practice many people use agile within companies that do not have the resources to perform a level of analysis that involves testing on several functioning prototypes. That said, lots of data exists out there about acceptable limits based on human attention spans. At my organization, when possible we set a target based on that data and then in discussions we can estimate the story points for achieving all or part of that goal. – Holly Aug 21 '14 at 14:29
1

Often these types of stories come from high level stakeholders - in our case it was the CEO complaining the site is too slow. He was asking for sub-second performance, which was not realistic due to the data processing involved.

We used vague stories such as 'Improve performance of X', which gave us enough of a goal to identify some different technical stories that were all likely to improve performance. We then prioritised these based on effort and likely impact. We then started at the top, until we got to a point that our product owner was satisfied we had made enough of a positive impact, and the remaining effort outweighed the benefits.

I don't like placing a concrete target on these sorts of things, as you may later find out you can get 90% of the way with 10% of the effort, and that is good enough. If the work you have done hasn't made enough of a difference it will definitely be raised as an issue again and you have confirmation you need to do further work.

  • OK thanks; I'm not sure how "technical stories" would fit in the "proper" way of doing things but it sounds like it might be what's needed, though it goes against the advice I've heard of phrasing things in terms of a user benefit... – Emile Aug 19 '14 at 14:46
  • An additional question tho: for those technical stories (that don't have specific numbers attached), do you have some kind of acceptance criteria? Who validates it as "done"? How do you estimate it's size (if at all)? – Emile Aug 19 '14 at 14:48
  • We break away from the norm for these stories as they sit under a clear story with a benefit. ie "Improve search engine optimisation" (terrible example, I know) has real value, and can be represented with sub-stories that are more like tasks "Replace the doohickey with a doodad" which have a clear end point, but an uncertainty around how much difference they will make. Once this is done, you can see if you need to do more of these stories to get to a satisfactory point. I think common sense can outweigh process - the need is clear, the exact end point is not. – SpoonerNZ Aug 20 '14 at 11:53
  • OK, thanks, it seems to go against "how we were told to do stories in the scrum training" but I'm all for pragmatism. – Emile Aug 20 '14 at 17:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.