How to Record Bugs Following User Story Completion - in Large Projects

Question

We run scrum agile methodology and have a development environment for development. At the end of each sprint, we close off all the tasks/stories that have been completed and promote the code/system to the user test environment.

As bugs are found during testing, they are put into the sprint backlog.

Ordinarily, bugs are quite small defects, relative to the development and we raise these as bugs.

But, if it is found that the original user story wasn't really completed correctly, should the original user story be reopened?

-- Edit --

Following some very worthwhile comments/answers, which have got me thinking, I think I would clarify the situation, drawing together my comments.

The project is the delivery of a Tb data warehouse project consolidating data from 8 source systems, with about 30 developers and a dozen testers. We have defined user stories but, specifically due the difficult UAT process, the Definition of Done for a user story, is to pass unit testing and pass some fairly superficial system and integration testing. This may be an incorrect thing to do, and is the crux of my question now!

The UAT process is quite long winded and takes

• several days to create the environment
• 24 hours of continual elapsed time to loading the data from the source systems into the data warehouse. This time could be sped up, with lots of ££ spent on hardware, which the project doesn't have.
• 3 days to perform the actual tests. Some are automated using a front end portal, others are execution of data from the source systems and comparing to the data warehouse. This time could be sped up to 2 days with some more staff.

So within a 3 week sprint, this is not really feasible to fit in, as there needs to be a code freeze of all the developers at a point in the week which is then promoted to the UAT environment, then the above process is performed. We would roughly lose one third of our sprint in testing, where the developers would not be doing anything, because if they did, then their code would need UAT and couldn't be delivered in the current sprint.

Currently we are reopening user stories that have been closed when bugs are found, inserting a new "bug" subtask and then putting that into the backlog.

I would be interested particularly in hearing from someone who has engaged with a large project; that is not to invalidate people with good theory or small project knowledge.

Is this the correct way to do things?

Answer 1

When you have a story, it has a Definition of Done that includes all of the testing and quality assurance activities. I would recommend that a defect in a user story doesn't exist unless it was found after the story has been called Done. Anything that you find before this point isn't a defect, but part of the normal iterative development process.

Based on your comment, the fact that testing in your clean, non-development environment isn't part of your Definition of Done is the problem. I can understand the need for a development environment with development tools (compilers, test harnesses, debug configurations and builds, analysis tools, a complete and real data set, etc) that don't exist in the deployment environment. However, if you keep running into issues that are only found in this environment, then someone testing in that environment should be part of your definition of done. Your team performing a deployment to your clean environment and executing the tests should be part of your acceptance criteria and Definition of Done. This deployment to your clean test environment should happen throughout your sprint so you can be reasonably sure that when you deploy to the real environment, you won't keep having these issues.

Answer 2

Separating out development from testing is almost always a bad thing. The reasons for this are:

• It is difficult to get a true measure of progress. When stories have been developed but not tested it is difficult to know how much effort still remains. You can give the impression of good progress ("we have developed 6 stories this sprint") but what if when they are tested some serious bugs are found?
• Typically the longer the time between the development work and the bug being raised the longer it takes to fix the bug. When a developer is asked to fix a bug on something they worked on an hour ago they often do it quickly. Ask a developer to fix a bug on something they worked on weeks ago and it is much more difficult and usually takes longer.
• Separating out development from testing tends to encourage developers to ignore quality.
• Separating out development from testing tends to encourage a silo mentality.

In the Scrum Guide is says the following:

"The heart of Scrum is a Sprint, a time-box of one month or less during which a 'Done', useable, and potentially releasable product Increment is created."

There are often technical challenges to developing and testing in the same sprint. But it is better to work to overcome those challenges than to risk the problems listed above.

Answer 3

So... you discover bugs after your development team declares its work 'done' and then have to go back and fix them. Whilst new code has already been developed by the time you discover the defects.

Because testing in your Sprint takes too much time.

A sarcastastic person might conclude 'you have no time to do it right the first time, but somehow you do have time to do it again'.

You have found an impediment (group of impediments) that hinder your team to finish the stories. A few suggestions to deal with the impediment: You can start loading a test system in the 3rd week of the Sprint. Perhaps load the system on Friday evening so it loads over the weekend and your team is ready to test on Monday.

Your developers can invest time in automating the deployment and setup of the test-environment.

Your developers can start automating testcases.

Ideally, you'd start a deployment on Friday evening. It would run through the weekend and runs tests that ensure that basic functionality still works, and preferably run functional regression tests.

During the user-test the last Sprint-week, the developers spend their time fixing the bugs and automating more of the tests the users perform.

Results: Deployments go faster, freeing time from developers. Regression-tests are already done by the time Users start testing, so less time is wasted on bugs in software that should have worked.

Eventually you might even deploy more often during a Sprint or shorten your Sprints.

You may need some additional hardware to create a test-environment for the Dev-team, but hardware is cheaper than hours.

You may be tempted to go to 1 month Sprints although I wouldn't. I'd prefer not to put too so functionality in a Sprint that large bugs can hide in it.

Answer 4

A User Story is completed if it meets your Definition of Done. The story should be accepted by the Product Owner in the live environment. If the acceptance criteria are met then the user value is delivered and the story is done. So for me there is no benefit from re-opening stories if there are defects found.

My advice would be to focus on defining a meaningful Definition of Done, defining clear acceptance criteria and regarding the acceptance of stories as a serious ceremony.

Answer 5

In one of our projects, where we used JIRA, we defined another state aside of Open, In progress and done: resolved.

Once a feature is developed by developer it is set to resolved and tester knows s/he can start to test.

We did it to support "instant contact" since our team was distributed to different team zones.

Example: Lets say that developer finishes work, but tester sleeps on the other side of the world. Developer then sets the status to resolved. When tester comes to work (developer already home) s/he sees what has been done and starts to test. Each bug s/he founds links to the user story and until all linked bugs are resolved, the user story in not put to done