Should there be testing-only user stories?

Question

We estimate stories as 1, 2, 4, 6, or 8 points. Consider a story with less development work but huge testing effort. Let's say we estimate that story as 8 points.

We see the following options:

1. Let the story be 8 points, and be ready for the story testing to slip to the next sprint. This way development and testing are included in one story, but the team doesn't get any points for the work done in the current sprint. They get all 8 points in the 2nd sprint, or when the testing ends.

2. Split the story for testing purpose, i.e. development and some minor testing in one story, which can be completed in one sprint, then carve out the second story just for testing purposes (a testing-only story). This way the team gets some points in both the sprints.

In my previous experience, we have been opting for option 1, but in my current project people are more keen on option 2. I'm not sure if it is a correct way to split stories.

Answer 1

TL;DR

Neither of your stated options are truly agile. You are misusing points in an attempt to represent progress or to "hold people accountable." Neither is appropriate within the Scrum framework.

Points are an estimating tool. They are only meaningful in the aggregate, and are primarily needed for estimating team capacity during Sprint Planning. Using them as a productivity metric is misleading, and continued misuse of the story-point system will eventually harm both the team and the project.

Points are Not Rewards

This way the team gets some points in both the sprints.

In Scrum, a story is either done or not-done; it is never partially-done. You can certainly rig the system by artificially decomposing stories so that you can "earn" points in a Sprint by completing certain stories even when the real underlying feature isn't complete, but this doesn't serve any practical purpose other than to inflate velocity. Inflated velocity will ultimately skew your estimates, hurt the team, and damage your project. Don't do that.

Points are also not rewards. They aren't lollipops that you hand out for a job well done; they are simply a work-effort metric. Specifically, story points:

1. Are converted to a velocity metric by aggregating over an historical window, tracking the team's capacity for work over time.
2. Are used as a sanity-check to determine if a story can deliver its intended value within a single iteration.

Points Measure Capacity

Points are both an estimating tool and a means of tracking team capacity over time. When estimating, story points tell you:

1. How much effort the team thinks a story will take.
2. Whether or not the story (in its entirety) can fit within the current iteration.
3. Whether the team should accept the story into the sprint as-is, or whether the team needs to work with the Product Owner to refine, clarify, decompose, discard, or re-scope the story to achieve the current Sprint Goal.

Points also act as a proxy for team capacity over time. Earning points adds no value to the project, but being able to say that the team can usually complete an average of 12-16 points in a typical iteration helps the team estimate its capacity to accept stories during Sprint Planning. This helps the team avoid over-committing, prevents setting the team up for failure by creating an unsustainable work-load, and avoids planning a sprint that will not meet the stated Sprint Goal for the iteration.

Stories Should Fit Within One Sprint

If your stories can't fit within a single sprint, then you either have an epic that must be decomposed, or your sprint length has not been optimized for your project. There is always a trade-off involved in determining the optimal sprint length for a project, but the challenge of writing stories that fit into a finite time-box remains a constant challenge.

In Scrum, a story should be a full-stack, vertical slice of functionality that engages the whole cross-functional team. Testing is never a separate story; rather, it is an integral task required to ensure that a given feature meets the formal "definition of done" as defined by the project.

Stories that are expected to extend past the end of an iteration indicate a process problem that must be fixed during Backlog Grooming or Sprint Planning. Likewise, stories which don't meet the definition of done are also process problems that the team urgently needs to address.

Answer 2

When looking at problematic stories, the best advice is to use INVEST principles and in particular:

Stories deliver value

If QA is an activity that must be performed in order to deliver at all, then you can't split it off. Did you have a "definition of done" meeting? Does "done" include QA?

If your story doesn't deliver value, it's not a story.

Stories are small

All stories must be less than one sprint in length in order to be planned. Split your story by delivering a subset of the features (less value, but still some value).

Answer 3

Option 2 only seems valid if you view testing as a service, and not as a responsibility of the team. If that's the case, I'd question how agile the team is and do you really need to squeeze 2 separate teams (dev & qa) into one sprint. If your goal is to be agile, it might be worth not to distinguish between dev and test and consider a story "done" when it is properly tested. That way you'll end up estimating the overall work needed to be done as a team, and not just sub-parts of it.

Answer 4

Alright, please don't shoot me but I'll argue against what most people recommend here, and that is to split certain test activities into their own stories. Just to be clear, with test activities I am specifically referring to authoring automated test cases. Manual/exploratory testing is a bonus.

Let's start with some theory for which I'll use my go-to reference for (microservice) testing strategies I think we will all agree that the test pyramid makes sense, that there will be a LOT more unit tests than (automated) component tests, and even less (automated) end-to-end tests. Let's disregard for a moment who does what... I typically observe the following pattern in Agile/Scrum/whatever:

• A feature is made up of several stories (avg. 3-5)
• Each story is typically an individual deliverable that can be demoed and which (hopefully) provides some value to the end user.
• Each story includes the source code, unit tests, possibly some integration tests (see definition of integration tests here), documentation, etc. No discussion here that these types of tests need to be part of a story, as these types of tests need to be written by the story's implementer anyway.
• For each feature, there is only a handful of E2E tests written, and those E2E tests typically don't even test each story individually but test the contributions of multiple stories in one test case.
• It is difficult to write E2E tests while the functionality under test is not yet fully available in a DEV or TEST environment.
• A related observation is that the tools used for unit and integration testing are usually different from those used for E2E testing.
• Due to the different tooling, and also for historical reasons, even in days of Agile everywhere the specialization into developers and testers/QA engineers persists.

The question now is how to handle E2E tests: Include them in each story, or create separate stories? In my opinion, when I take all the influences listed above into consideration, the conclusion is to split them out. On my projects we typically end up adding one component/E2E test story to each feature. Authoring of test cases typically completes one sprint after the last story implementation completes. Features cannot be set to completed unless the associated E2E test story is complete, a clean and simple control mechanism.

I will argue that this does not violate the spirit of delivering stories, as each story still is fully demonstrable, and comes with a full set of unit and integration tests and therefore can be assumed high quality. With the split into developer and tester roles, this has proven to be the most efficient and systematic way for me. Else E2E test authoring activities will constantly "delay" the completion of individual stories, especially of those for which "development" work has been completed near the end of a sprint.

Now, in an ideal world there probably would be no specialization into developer and tester/QA engineer. Then I could see that component and E2E tests could be rolled into each story (still subject to inefficiencies due to the mismatch between fine-grained stories and coarse-grained component/E2E tests). But in the organizations I have worked in it was already difficult enough to establish an Agile mindset, lest purging the concept of dedicated tester/QA roles.

Maybe not the purest approach but the most practical and practicable I could come up with.

Answer 5

Be careful when splitting stories

If you find yourself splitting a user story ask yourself "does this story I am creating add value on it's own?". If the answer is no, then it should be a task as part of a larger user story.

Answer 6

Right answer is fully depends on your goals.

If the goal is to meet scrum requirements: you shouldn't split story into, let's call it tasks as there won't be a value inside each of them.

Nevertheless if you see an option to make more transparency and predictability within your development process: splitting tasks into small solid items will bring you a lot of insights (e.g. which part of your delivery pipeline is a bottleneck, etc)

Answer 7

Here are a few more abstract thoughts: (opinions)

Don't let "Points" get in the way: Points are a very abstract concept – who doesn't want to win more of them? The pursuit of them can distract from what you are really trying to accomplish. For this reason I don't use them.

"User" Stories only go so far: Many of the things that have to be done in building a software system are never explicitly visible to the user. Many of these things intersect many "user stories or activities."

Users are not Programmers, and their "stories" cannot fully inform programmers: They get in the car and drive it – but they never lift the hood, and they don't really know what the gearshift is connected to. They are never aware of what must be accomplished to let them drive safely down the road. They have ideas of what it takes to build a car, but they really don't know. "And, why should they? They're drivers, not auto mechanics!"

"User Stories" are fundamentally [IMHO ...] a communication device: They keep you firmly in contact with what the user will see, and to know that what you are producing will actually be useful for them, and they give both parties a common frame-of-reference that does not demand "programming knowledge." All of this is very good!! ... Very valuable!! ... But there is more that the team must generate for itself, addressing those issues that "users" know nothing of but which are essential in giving them what they require. As a project manager and as a design team, "user stories" are inputs. They are not implementation plans – but they help to drive those plans.

"Scrum is [IMHO ...] a metaphor: A very, very useful one. Embrace it for what it gives, and know when to let it go. It isn't a religion, and it never was intended to be. It is a valuable process and an excellent way of thinking – as far as it goes. It will not write your implementation plan for you, nor should you ever (of course ...) expect it to.