7

We continue to struggle with how to improve our QA and Testing processes in the sprint. We run a two week sprint and find that the QA and testing gets left late in the process and ends up rushed with the beginning of the next sprint spent fixing bugs. Not ideal.

One solution we have tried is mandating all stories must be complete by day 8 of 10, but this has proven too rigid and really didn't change things much anyway.

We have 6 developers and don't have a QA position (we have the rest of the company help test on the staging server each sprint). I don't see us adding a dedicated QA position any time soon.

Given this scenario, what can we try to improve the quality and reliability of what we are releasing?

Improve this question

4 Answers 4

Reset to default
7

I would first ask a question why it happens so that all stories are completed late. Is it possible that all stories need 9-10 days to be completed? If not than probably developers are focusing on starting building another story avoiding mundane task of testing.

Limiting work in progress can be something which can help to change the focus of the team. This is one of concepts of Kanban. It basically is a limit which tells you that you shouldn't have more than x stories on a specific stage.

In your case it can be not more than 6 stories being developed at the same time. Having such limit enforces people either to push stories further down the development process, e.g. start testing instead of building another story from the backlog, or help other developers to finish their stories.

To make a simple example: if every single one of your developers is developing a story and you have a limit of 6 on development stage, the situation when someone starts building a new story before the old one is finished shouldn't happen. What more, if you set one limit of 6 for both "under development" and "development done" columns your developers will have to start testing finished features in the first place before they are able to work on a new story.

Of course limit shouldn't be arbitrary set on the number of people in the team. It will vary depending on a process, team, the way you work (e.g. if you pair program you effectively have 3 "work units," not 6), etc. Try to experiment to find right limits.

Improve this answer
4

Basics first. Do you have acceptance tests attached to your requirements? If not then how are you testing?

How, on a 2 week sprint, do you define done for an item on the sprint backlog? If testing all the way through QA isn't part of that "done" then you need to step back and rethink your process.

2 week long sprints are fine but why the rush? I realize as a startup you're trying to get stuff done and out the door. However if your team is ending up doing a lot of bug fixing than in reality you're not saving any time at all.

So from the top: get somebody to at least come up with acceptance tests, get an automated build server in place, get you devs to do rigorous unit testing. And then, track it all.

Managment might balk, but point out that delivering buggy code and spending time to fix it is going to cost them more. It always does.

Improve this answer
6
  • We do have acceptance criteria specified for each user story and we write unit tests and integration tests. The thing about those is they only protect you from what you think of and unfortunately users think very differently from developers. 2 week sprints already feel long to me. We don't always release everything each sprint and are pretty good about breaking features up into stories but that doesn't solve the QA question. Slowing down is an expensive answer - it might turn out to be the right answer, but we want to explore other ideas first.
    – chrishomer
    Commented Aug 10, 2011 at 15:21
  • Interesting... how is testing beging done? Ad hoc or do you have formal scripts? To be anal, acceptance criteria are not the same as acceptance tests. To me a test specifies at least the basic steps the user is expected to go through. Criteria generally just explain the expected end state. Sounds like either you as pm or some other non dev resource needs to come up with a repeatable testing process. The reason I say this is with ad hoc testing across a varied user base, are you really getting bugs or subtle feature changes. Also are the bugs being prioritized vs. the new features?
    – edgaralgernon
    Commented Aug 10, 2011 at 15:56
  • I would call it scheduled ad hoc. We develop the test scripts along with the features and try to keep them vague so the 20 people doing user tests, interpret and follow more varied paths through the features. I've found that using steps to specify acceptance tests leads to missing user paths that people don't consider. In a non-linear flow, you can't cover all the alternatives, right?
    – chrishomer
    Commented Aug 10, 2011 at 17:30
  • 1
    o.k. bounced this off a QA buddy. Though open ended tests are good in finding a variety of paths, if you don't use repeatable tests (so scripted paths) it will be hard to actually know if a bug has been resolved. Also testing time roughly equals dev time, so if it's 8 days of dev it should be 8 days of testing. I'm guessing a lot of churn is coming from 1) too much test variation 2) Product owner needs to prioritize defects vs features (and is everything being reported a real defect, or a new feature?) I wouldn't shorten the sprint length, might even add a week. just a thought.
    – edgaralgernon
    Commented Aug 10, 2011 at 18:45
  • @ChrisH Not scripting steps for acceptance tests seems a sane approach to me. Attempting to script manual acceptance tests in detail is a colossal waste of time - and you've already recognised that it tends to lead to inattentional blindness. Check out kaner.com/pdfs/ValueOfChecklists.pdf for more detail. If you want repeatability, automate. If you want accurate bug repro steps - then don't waste time writing out test steps for every test, YAGNI. Focus on getting people to write out detailed bug repro where they need to.
    – testerab
    Commented Aug 20, 2011 at 19:24
3

I think your team needs a slight adjustment in their "Agile mindset". It's a cultural shift that needs to happen early phases of Agile adoption.

Testing is not necessarily an activity that can only happen after the implementation is completed. Testers don't need to be only involved after the 'programmers' are done. It is important that testers get involved as early as possible, possibly bring part of design discussions, User Interface and usability finalizations, doing reality checks on UI and possibly design and architecture. This not only helps them in getting better understanding of the stories/features being worked on in the sprint but also provide useful input during the actual development.

This obviously is part of their effort - they can spend the time during the implementation time to understand the Acceptance tests, preparing testing environment etc. After a few Sprints, however, teams 'stabilize' and the testers would always have some stories to test or bugs to verify while others are coming in the pipeline.

The core issue at heart is not scheduling (IMHO) - it's changing the traditional testing approach and mindset.

Improve this answer
2

On a side note, if "all" of your stories are taking 8-10 days to complete, IMO your stories are too large. I realize it is not always possible to slice some stories into smaller chunks of deliverable business value but it potentially could be worth the time to ask the question of "why" do our stories take 8-10 days to complete? Has this been brought up within any retrospectives?

I would also recommend trying out a few one week sprints. I've found a shorter sprint helps teams focus on tasks at hand & ultimately slice stories differently than they would in a two week sprint.

The last point / question I would make is, how are you testing that bugs are fixed? Are automated regression tests built which follow the same pathway that found the bug? If there aren't I wouldn't accept that the bug has been "fixed".

Improve this answer
2
  • 1
    +1 for decreasing the sprint length. Shorter sprints means less WIP.
    – Andrew Clear
    Commented Mar 3, 2013 at 22:28
  • I recently reduced my team's Sprint periods from 2 weeks to 1 week and the results are immediate. The development teams are much more focused on their tasks but even more, we are able to slice up stories in smaller chunks than we did with 2-week sprints. Both accountability and estimates are much better this way and we can better engage to our plans.
    – Mike
    Commented Nov 18, 2015 at 14:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.