4

We realized that we have an urgent requirement to complete a series of automated testing stories in a sprint. Therefore we took in automation only stories. Some manual testers took in a few other urgent stories, therefore in the Doing column we have a bunch of stories waiting to be tested.

Obviously this will continue going forward because there will be a backlog of stories to be tested, whereas development will continue (we can't halt development because of this).

Any suggestions to get around this issue? Throw in more testers? Btw, our developers (and management) don't think that developers are good testers because they don't have the tester's eye, so they're more likely to make mistakes in their testing.

7

Stop the Line; Fix the Process

Obviously this will continue going forward because there will be a backlog of stories to be tested, whereas development will continue (we can't halt development because of this).

In other words: "Help, we have a broken process! How can we keep doing what we're doing, and fix it without fixing it?" You're trying to put lipstick on a pig rather than stopping the line to fix the underlying process problem. This is an agile anti-pattern.

An agile story is "done" or "not done." Nothing is stopping you from removing successful test completion from your team's Definition of Done, but this is generally a process smell found in organizations that value the illusion of progress towards arbitrary management targets over working software. The fact that there are deadlines is immaterial to the fact that your system has constraints.

As a metaphor, consider that it still takes typically takes nine months to have a baby, regardless of how many people you throw at it, or how many columns you put on a Kanban board. The biological system imposes constraints that can't be wished away.

Why Your Process is Broken

Your process is fundamentally broken because:

  1. Your developers and your testers aren't actively collaborating.

    Development and testing are a type of collaborative ping-ping. Even if you don't follow a test-first methodology, developers and testers need to operate in a very tight feedback loop for agility. This means:

    • Being inside the same team, rather than treated as an externality or a drag on delivery.
    • Working together in as close to real-time as possible. Post hoc testing in a separate iteration isn't even close to this fundamental requirement.
    • Treating only verified, working code as "delivered." Anything else is wish-fulfillment.
  2. Your developers are writing code without tests to define success.

    • Writing code before the tests is like building a house without an architect's plans or before pouring the foundation. Good luck insuring that house!
    • You're practically begging for bugs to be filed when you write code that may or may not meet executable specifications (pronounced "tests") that are developed separately.
    • Features, tests, bug reporting, analysis, patching, and retesting all take non-zero time from both developers and testers. Trying to sweep everything but features out of the way of "development" to meet management targets is fundamentally unsustainable.

How to Fix It

You need to factor testing into your team's capacity for getting work done, as well your Definition of Done. Doing this may slow your apparent pace of development, but it will not affect your actual capacity for delivering working software. Once you know your real, sustainable cadence for delivering finished increments, you can adjust your project schedule or scope based on your then-current system constraints.

  • Developers have to learn to test. Not only will it make for better code, it addresses your "critical path" or "Work in Progress Limit" of not enough testers. In fact it is becoming rather common to do away with dedicated testers and move to a development only environment where test and coding are done by everyone. Testers move towards being developers and Developers move towards being testers. – Joel Bancroft-Connors Nov 22 '16 at 20:22
  • 2
    Love the 'architect' comparison, probably going to quote that here and there. +1 :) – Niels van Reijmersdal Nov 22 '16 at 20:57
2

If you have both 'developers' (development-centric developers) and 'testers' (test-centric developers) in your Development Team, then they should be working together, in order to improve your regular developers' abilities to QA, as well as to decrease the workload of the testers.

Also, it's unclear in your question whether the automation-testing is done before or after a story's development. This should be done before. If it's not, consider changing this - in addition to Test Driven Development's other benefits, this will help mitigate the bottleneck of things waiting for testing after development is done. User Acceptance Testing should be done afterwards, but there's no reason to put automated-testing off.

Finally, regarding your comment of:

we can't halt development because of this

Remember that in Scrum, a story which is not 'Done' constitutes zero (or even negative) value. If your process is such that the developers are just creating a bunch of non-finished stories, you'd actually be better off having them stop development and help testing catch up however they can.

0

Test only sprints, yeah! Developers not testing, bleh...

I firmly believe part of being a good software developer is being a good tester.

Testers should teach developers to test, it has nothing todo with having a testers eye.

Your questions conflict with your tags. You are describing a Waterfall process while thinking your doing Scrum/Agile. Testing is a parallel activity and quality is a team responsibility, not that of a testers as an after thought. If your management loves Waterfall, please do waterfall and don't call it Agile/Scrum.

Also read an similar answer I gave here with details about modern Agile/Scrum testing practises, it contains a good suggested read list: https://pm.stackexchange.com/a/20560/8528

-1

Sorry, but it looks like some of the answers above address the wrong question.

Yes, manual validation is a must for acceptance, and yes, it usually should be done by people qualified to do it, and ideally working in the same Scrum team. However, the original question was not about that, but specifically about how to treat QA automation under the Scrum framework assuming resource contention. This is much more specific, and something that I also tried solving via different means.

It often happens that automation is an epic in itself, as it requires setting up an environment, developing the framework, code for the tests, the tests themselves, and then hooking up to CI. Automation can also be expensive, and putting it as success criterion for all development stories can easily grind software delivery to a halt. On the contrary, it is perfectly legitimate to release feature manually tested. Hence, I don't see any issue with having automation as a separate entity in Scrum.

So, it can be prioritized against other backlog tasks, assuming that the original feature was tested manually, and that the business is happy to pay the price of manual regression testing until automation gets in place. Backlog priorities would depend on how liable the feature is to be modified in the future (and hence require retesting), and how soon will automation provide ROI compared to recurring manual tests.

Bottom-line, this is more of a resourcing question. If you find manual regression price tag constantly growing, and/or defects escaping due to lack of automation, you have a good business case to present to the management: for example to slow down development to catch up on automation, to hire a contractor for a one-off boost to your automation gap, or any other option that the budget can afford.

For example, it is more and more common to find the Software Engineer in Test skillset profile, i.e. people who are both automation specialists and can also pick up development tasks. Having these engineers in the team gives the flexibility to put more emphasis on automation if required. There are other tactics, but as with all real-life situations, there's no silver bullet. For a more detailed answer, I'd need to understand your business's situation a bit better.

  • This sounds like suggesting "we will automate testing when we have the time" which is just as bad as hacking up some code and saying "we will clean it up when we have the time". Experience learns you will nearly never make time for these tasks later. If you want to keep going fast you need clean code and thats means more then just code that reads as well written prose, it also means code that is tested with automation. Suggested read less.works/less/technical-excellence/test-automation.html I truly think Agile teams should try to automate all testing and not postpone it. – Niels van Reijmersdal Nov 28 '16 at 21:52
  • The comment was more substantiated than "we will automate when we have the time". Realities of practical software development is that sometimes automation does not provide the right ROI, and there are market forces or budget constraints that push us in a certain direction. This is borne on experience, though I do agree that the ideal should be "automated as early as possible". Having said that, it is very easy to bring real-world examples where the suggestions in the linked article would be incorrect. – RomanK Nov 29 '16 at 4:54
  • Reality is that ROI of automated testing is hard to prove, until it is to late and the product has a messy code base and without tests it is impossible to refactor. Slowing a product to a grinding halt. I think Project Managers do not give this enough thought. As early as possible should be from the start in Agile projects. Unless someone is going to eat your lunch and it is a do or die situation, which for most projects it really isn't, although CEO's would like to tell you it is :) – Niels van Reijmersdal Nov 29 '16 at 7:54
  • I agree - 80% of the way. It goes beyond a chat, but it would be easy to pull out a few exceptions from my career where skipping or postponing an automated test was justified. Though there are more examples when I came onto a codebase which suffered from long neglect of automation tests, so people tend to err more on the side of omitting them. – RomanK Nov 29 '16 at 14:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.