3

We run scrum agile methodology and have a development environment for development. At the end of each sprint, we close off all the tasks/stories that have been completed and promote the code/system to the user test environment.

As bugs are found during testing, they are put into the sprint backlog.

Ordinarily, bugs are quite small defects, relative to the development and we raise these as bugs.

But, if it is found that the original user story wasn't really completed correctly, should the original user story be reopened?

-- Edit --

Following some very worthwhile comments/answers, which have got me thinking, I think I would clarify the situation, drawing together my comments.

The project is the delivery of a Tb data warehouse project consolidating data from 8 source systems, with about 30 developers and a dozen testers. We have defined user stories but, specifically due the difficult UAT process, the Definition of Done for a user story, is to pass unit testing and pass some fairly superficial system and integration testing. This may be an incorrect thing to do, and is the crux of my question now!

The UAT process is quite long winded and takes

  • several days to create the environment
  • 24 hours of continual elapsed time to loading the data from the source systems into the data warehouse. This time could be sped up, with lots of ££ spent on hardware, which the project doesn't have.
  • 3 days to perform the actual tests. Some are automated using a front end portal, others are execution of data from the source systems and comparing to the data warehouse. This time could be sped up to 2 days with some more staff.

So within a 3 week sprint, this is not really feasible to fit in, as there needs to be a code freeze of all the developers at a point in the week which is then promoted to the UAT environment, then the above process is performed. We would roughly lose one third of our sprint in testing, where the developers would not be doing anything, because if they did, then their code would need UAT and couldn't be delivered in the current sprint.

Currently we are reopening user stories that have been closed when bugs are found, inserting a new "bug" subtask and then putting that into the backlog.

I would be interested particularly in hearing from someone who has engaged with a large project; that is not to invalidate people with good theory or small project knowledge.

Is this the correct way to do things?

1
  • 2
    The answer to your final question is 'no'. If the original user story was not completed correctly then your acceptance criteria was defined well enough, your testing process failed at some point and/or the Product Owner should not have accepted the story as 'Done'. You should invest time in the retrospective understanding why you cannot complete the build and test of a single user story inside a Sprint and why the Definition of Done is allowing defects to occur in production-ready software. I have raised this as a comment though so you can challenge my thinking. May 24, 2016 at 8:41

5 Answers 5

3

When you have a story, it has a Definition of Done that includes all of the testing and quality assurance activities. I would recommend that a defect in a user story doesn't exist unless it was found after the story has been called Done. Anything that you find before this point isn't a defect, but part of the normal iterative development process.

Based on your comment, the fact that testing in your clean, non-development environment isn't part of your Definition of Done is the problem. I can understand the need for a development environment with development tools (compilers, test harnesses, debug configurations and builds, analysis tools, a complete and real data set, etc) that don't exist in the deployment environment. However, if you keep running into issues that are only found in this environment, then someone testing in that environment should be part of your definition of done. Your team performing a deployment to your clean environment and executing the tests should be part of your acceptance criteria and Definition of Done. This deployment to your clean test environment should happen throughout your sprint so you can be reasonably sure that when you deploy to the real environment, you won't keep having these issues.

1
  • Thanks. Our project is pretty complex, involving about 30 devs and perhaps a dozen testers. The work is combination front-end and database work, and we have a CI/AT suite running in dev on a limited set of data, but actually this doesn't test the 1 Tb set of data moving through the system and the nuances of combinations of data in the front end. So we have resorted to running a sprint and delivering user stories for that sprint, then in the following sprint, the user stories delivered in the previous sprint will be run through in a test 'sprint' and defects raised.
    – Marcus D
    May 23, 2016 at 14:39
3

Separating out development from testing is almost always a bad thing. The reasons for this are:

  • It is difficult to get a true measure of progress. When stories have been developed but not tested it is difficult to know how much effort still remains. You can give the impression of good progress ("we have developed 6 stories this sprint") but what if when they are tested some serious bugs are found?
  • Typically the longer the time between the development work and the bug being raised the longer it takes to fix the bug. When a developer is asked to fix a bug on something they worked on an hour ago they often do it quickly. Ask a developer to fix a bug on something they worked on weeks ago and it is much more difficult and usually takes longer.
  • Separating out development from testing tends to encourage developers to ignore quality.
  • Separating out development from testing tends to encourage a silo mentality.

In the Scrum Guide is says the following:

"The heart of Scrum is a Sprint, a time-box of one month or less during which a 'Done', useable, and potentially releasable product Increment is created."

There are often technical challenges to developing and testing in the same sprint. But it is better to work to overcome those challenges than to risk the problems listed above.

2

So... you discover bugs after your development team declares its work 'done' and then have to go back and fix them. Whilst new code has already been developed by the time you discover the defects.

Because testing in your Sprint takes too much time.

A sarcastastic person might conclude 'you have no time to do it right the first time, but somehow you do have time to do it again'.

You have found an impediment (group of impediments) that hinder your team to finish the stories. A few suggestions to deal with the impediment: You can start loading a test system in the 3rd week of the Sprint. Perhaps load the system on Friday evening so it loads over the weekend and your team is ready to test on Monday.

Your developers can invest time in automating the deployment and setup of the test-environment.

Your developers can start automating testcases.

Ideally, you'd start a deployment on Friday evening. It would run through the weekend and runs tests that ensure that basic functionality still works, and preferably run functional regression tests.

During the user-test the last Sprint-week, the developers spend their time fixing the bugs and automating more of the tests the users perform.

Results: Deployments go faster, freeing time from developers. Regression-tests are already done by the time Users start testing, so less time is wasted on bugs in software that should have worked.

Eventually you might even deploy more often during a Sprint or shorten your Sprints.

You may need some additional hardware to create a test-environment for the Dev-team, but hardware is cheaper than hours.

You may be tempted to go to 1 month Sprints although I wouldn't. I'd prefer not to put too so functionality in a Sprint that large bugs can hide in it.

1

A User Story is completed if it meets your Definition of Done. The story should be accepted by the Product Owner in the live environment. If the acceptance criteria are met then the user value is delivered and the story is done. So for me there is no benefit from re-opening stories if there are defects found.

My advice would be to focus on defining a meaningful Definition of Done, defining clear acceptance criteria and regarding the acceptance of stories as a serious ceremony.

7
  • I understand. We don't have user testing tasks in our user stories. Our sprints are 3 weeks long and we need a clean deployment to another environment to test some story functionality completely, then with your methodology, you would keep a story open until it has passed User Testing in sprint (n+1) - potentially 3 weeks later.
    – Marcus D
    May 23, 2016 at 14:15
  • @MarcusD - why can you not complete a story including Testing within a single Sprint? What is blocking you from accomplishing that? May 24, 2016 at 8:38
  • @Venture2099, We perform unit testing, and simple system testing, but to perform user testing, we need to perform a data load which takes about 24 hours to load end-to-end, and the testing process of a set of sample tests takes up to 3 days. So putting that into a 3 week sprint, burns quite a lot of developer time, thus we chose to separate the dev from the user testing.
    – Marcus D
    May 24, 2016 at 9:20
  • @MarcusD... in true 5 why's tradition; why does it take 24 hours to load? Why does 24 hours = 3 days? I am not saying it can be improved, just asking why these constraints exist. May 24, 2016 at 14:07
  • @Venture2099 ... 24 hours = 1 continuous day. 8 source systems into staging then into a data warehouse of about 1 Tb. then with extracts / reports run on top of DWH. One suggestion which will happen is to create a logical subset of data that we push through ... challenging, but possible.
    – Marcus D
    May 24, 2016 at 18:03
1

In one of our projects, where we used JIRA, we defined another state aside of Open, In progress and done: resolved.

Once a feature is developed by developer it is set to resolved and tester knows s/he can start to test.

We did it to support "instant contact" since our team was distributed to different team zones.

Example: Lets say that developer finishes work, but tester sleeps on the other side of the world. Developer then sets the status to resolved. When tester comes to work (developer already home) s/he sees what has been done and starts to test. Each bug s/he founds links to the user story and until all linked bugs are resolved, the user story in not put to done

1
  • Very interesting and an aspect of that might assist us...
    – Marcus D
    May 26, 2016 at 11:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.