Test only stories in the Sprint?

Question

We realized that we have an urgent requirement to complete a series of automated testing stories in a sprint. Therefore we took in automation only stories. Some manual testers took in a few other urgent stories, therefore in the Doing column we have a bunch of stories waiting to be tested.

Obviously this will continue going forward because there will be a backlog of stories to be tested, whereas development will continue (we can't halt development because of this).

Any suggestions to get around this issue? Throw in more testers? Btw, our developers (and management) don't think that developers are good testers because they don't have the tester's eye, so they're more likely to make mistakes in their testing.

Answer 1

Stop the Line; Fix the Process

Obviously this will continue going forward because there will be a backlog of stories to be tested, whereas development will continue (we can't halt development because of this).

In other words: "Help, we have a broken process! How can we keep doing what we're doing, and fix it without fixing it?" You're trying to put lipstick on a pig rather than stopping the line to fix the underlying process problem. This is an agile anti-pattern.

An agile story is "done" or "not done." Nothing is stopping you from removing successful test completion from your team's Definition of Done, but this is generally a process smell found in organizations that value the illusion of progress towards arbitrary management targets over working software. The fact that there are deadlines is immaterial to the fact that your system has constraints.

As a metaphor, consider that it still takes typically takes nine months to have a baby, regardless of how many people you throw at it, or how many columns you put on a Kanban board. The biological system imposes constraints that can't be wished away.

Why Your Process is Broken

Your process is fundamentally broken because:

1. Your developers and your testers aren't actively collaborating.

   Development and testing are a type of collaborative ping-ping. Even if you don't follow a test-first methodology, developers and testers need to operate in a very tight feedback loop for agility. This means:

   • Being inside the same team, rather than treated as an externality or a drag on delivery.
   • Working together in as close to real-time as possible. Post hoc testing in a separate iteration isn't even close to this fundamental requirement.
   • Treating only verified, working code as "delivered." Anything else is wish-fulfillment.

2. Your developers are writing code without tests to define success.

   • Writing code before the tests is like building a house without an architect's plans or before pouring the foundation. Good luck insuring that house!
   • You're practically begging for bugs to be filed when you write code that may or may not meet executable specifications (pronounced "tests") that are developed separately.
   • Features, tests, bug reporting, analysis, patching, and retesting all take non-zero time from both developers and testers. Trying to sweep everything but features out of the way of "development" to meet management targets is fundamentally unsustainable.

How to Fix It

You need to factor testing into your team's capacity for getting work done, as well your Definition of Done. Doing this may slow your apparent pace of development, but it will not affect your actual capacity for delivering working software. Once you know your real, sustainable cadence for delivering finished increments, you can adjust your project schedule or scope based on your then-current system constraints.

Answer 2

If you have both 'developers' (development-centric developers) and 'testers' (test-centric developers) in your Development Team, then they should be working together, in order to improve your regular developers' abilities to QA, as well as to decrease the workload of the testers.

Also, it's unclear in your question whether the automation-testing is done before or after a story's development. This should be done before. If it's not, consider changing this - in addition to Test Driven Development's other benefits, this will help mitigate the bottleneck of things waiting for testing after development is done. User Acceptance Testing should be done afterwards, but there's no reason to put automated-testing off.

Finally, regarding your comment of:

we can't halt development because of this

Remember that in Scrum, a story which is not 'Done' constitutes zero (or even negative) value. If your process is such that the developers are just creating a bunch of non-finished stories, you'd actually be better off having them stop development and help testing catch up however they can.

Answer 3

Test only sprints, yeah! Developers not testing, bleh...

I firmly believe part of being a good software developer is being a good tester.

Testers should teach developers to test, it has nothing todo with having a testers eye.

Your questions conflict with your tags. You are describing a Waterfall process while thinking your doing Scrum/Agile. Testing is a parallel activity and quality is a team responsibility, not that of a testers as an after thought. If your management loves Waterfall, please do waterfall and don't call it Agile/Scrum.

Also read an similar answer I gave here with details about modern Agile/Scrum testing practises, it contains a good suggested read list: https://pm.stackexchange.com/a/20560/8528

Answer 4

Sorry, but it looks like some of the answers above address the wrong question.

Yes, manual validation is a must for acceptance, and yes, it usually should be done by people qualified to do it, and ideally working in the same Scrum team. However, the original question was not about that, but specifically about how to treat QA automation under the Scrum framework assuming resource contention. This is much more specific, and something that I also tried solving via different means.

It often happens that automation is an epic in itself, as it requires setting up an environment, developing the framework, code for the tests, the tests themselves, and then hooking up to CI. Automation can also be expensive, and putting it as success criterion for all development stories can easily grind software delivery to a halt. On the contrary, it is perfectly legitimate to release feature manually tested. Hence, I don't see any issue with having automation as a separate entity in Scrum.

So, it can be prioritized against other backlog tasks, assuming that the original feature was tested manually, and that the business is happy to pay the price of manual regression testing until automation gets in place. Backlog priorities would depend on how liable the feature is to be modified in the future (and hence require retesting), and how soon will automation provide ROI compared to recurring manual tests.

Bottom-line, this is more of a resourcing question. If you find manual regression price tag constantly growing, and/or defects escaping due to lack of automation, you have a good business case to present to the management: for example to slow down development to catch up on automation, to hire a contractor for a one-off boost to your automation gap, or any other option that the budget can afford.

For example, it is more and more common to find the Software Engineer in Test skillset profile, i.e. people who are both automation specialists and can also pick up development tasks. Having these engineers in the team gives the flexibility to put more emphasis on automation if required. There are other tactics, but as with all real-life situations, there's no silver bullet. For a more detailed answer, I'd need to understand your business's situation a bit better.