Project stuck in endless testing cycle

Question

I am involved in a project where for some time now the project has stuck in an endless testing, UAT, bug fixing cycle.

The issues that caused this are, lack of initial requirements, new requirements creeping up during testing, and just bad coding from the developers' side of things.

So, at the moment, we are stuck in the final cycle in which the developer gives a release, the release is tested (prechecked), business users are called for a UAT, the users find bugs that haven't been discovered before, the problems are sent to the developer for resolution, and the same cycle starts all over again.

How do you get out of this endless cycle?

Answer 1

I would hope that by now you would have frozen scope as you have cited new requirements entering at the testing stage as a problem. That has to stop otherwise you are in an endless cycle.

Once you have stopped the scope creep there are a number of areas you need to look at. But the one that leaps out at me is System Testing. If the users are finding issues in UAT that presumably means you didn't find them in system testing? No matter where you find them, the bugs are there, but it is preferable to find them (and fix them) before going into UAT. I would start measuring the number of issues found during System Testing and during UAT so that you have a quantitative picture. Then strengthen your System Testing; do more of it, use more (or better) testers, work from proper test scripts, make sure you have the right amount of test coverage (i.e. you can't test everything so make sure you are testing the areas that are right for your users in as much depth as required to assure quality in those areas).

Make sure that in each cycle of UAT all applicable test scripts are exercised. Don't let the users just stop testing just because they found an issue in one area. Make sure they test the other areas.

Introduce Regression testing into your system testing to some extent- For every release go back and re-test work that has already be validated and do it before the users see it.

Track the issues situation on a daily basis. Log everything and make sure you have prioritised the issues list (on a daily basis if necessary) so that the developers are focussing on the most important issues.

Grit your teeth, put your head down, and just get through it :) This can be hard, but if issues exist in the system you just have to keep going until everyone is happy that the release can go out (i.e. not necessarily when all the defects are fixed!). But be sure to make sure you are not making it worse by changing code during testing, other than to fix issues.

Lastly, after the dust has settled, take a long hard look at the process to try and figure out how and why the output was not of sufficient quality and try, as an organisation, to learn those lessons for next time.

Good luck!

Answer 2

I'm going to assume that there's at least one senior stakeholder involved in this process who has the power to say that the project is ready to be released. If there isn't then you have a big problem and you need to identify someone in the organisation who can step in to this role.

Now you need to get that person on-side.

Look at the current scope of the project, based on the requirements you've agreed to from the start up until your latest round of UAT. Review that list with your senior stakeholder and decide which of the requirements are actually critical to the deployment of your product. There will always be new requests on a project but your job as PM (and as a team more generally) is to work out which of those are absolutely essential and which can be defined as 'nice to haves' or as candidates for later version of the product. Once you've been through this process (which may be painful and long-winded - don't run any more UAT during this) you'll have your complete 'Phase 1' requirements and senior stakeholder buy in for them.

Next you actually need to deliver these requirements. If you're having problems with code quality and with UAT a quick(ish) way to tackle the problem would be to assign each requirement a 'definition of done'. If you can track the requirement back to the original source then speak to that person about what exactly that requirement looks like when it's finished. This will give your developers clearer outputs to work towards. At the same time, communicate to all stakeholder (the people who keep changing their requirements in particular) that not everything that everybody wants to include can make it in to the first release. But make sure it's clear that these additional requirements have been captured and will be reviewed/worked on next (or whatever you have capacity to agree to).

There's a lot of good advice in the answers already provided about best practice for project management in general. I think your project sounds like it's a bit late in the day to 'fix' by introducing radical methodological shifts. I'd concentrate on quick wins that will move you past your current impasse (something senior management types normally have a knack for) but make sure that there's time for a proper review of what went wrong so that you can avoid making the same mistakes on future projects. If you're not already keeping one start a 'Lessons learned' log for this purpose. If nothing else it's a useful place to vent your frustrations...

Answer 3

You have a couple of different issues:

• Poor planning. This covers the lack of initial requirements and the resulting scope creep. Your best approach is to stop all work and work out with your customer what business capabilities need to be delivered. Do not restart work until you have requirements to an adequate level of detail needed.
• Poor change management. This covers the scope creep. Identify a single, strong leader who has to approve any and all future changes to requirements. Do not allow your team to work on new requirements until you go through this formal change process.
• Poor quality systems. This covers finding too many bugs during testing. Review your internal testing processes. Divorce the developers from any and all testing, they will bring too much bias to the process. Give your team time enough to do the job right before it goes to UAT.

Answer 4

Have had to manage it as starting R&D manager in the past. Successful have been:

• Involve management; establish necessary steps to break out of negative cycle and get feedback.
• Management or sales should involve customer in problem and sketch situation: it is going bad, but this is the best way to get out of problems instead of even bigger problems to be discovered later.
• Get rid of non-motivated and non-productive members of team.
• Strictly follow plan and follow up on daily basis together with key team members.
• Maintain issues list and priorities, for instance using MoSCoW.
• Keep interface to management small; involve the sponsor only if possible.
• Buy paint for your hair when it starts fading gray.

Good luck!

Answer 5

You should consider starting a code review process. Just the thought of it will make the developers think before submitting a piece of code in to the repository. You should also start some automated test processes for previously encountered problems.

There are some tools that can help you. I do not know the development environment of yours, but Jenkins can help you with some of the automated stuff.

Also, Gerrit code review tool will make sure that submitted code will be checked by peers before going into the repository.

If you use these tools back to back, it should improve your bug rates.

Answer 6

You should probably make a proper plan and define the process before starting the project.

1. You are stuck in endless testing cycles. Define the testing process. When should they test? Give them deadlines. Give proper time for bug fixing.
2. If your requirements are not clear in initial stages, work only on the requirements which are clear. If you start working on requirements which are not clear, you will end up reworking them and wasting time. If you are not clear with requirements, call for a meeting with the client to make the requirements clear and start work.
3. Manage change requests. Once you get change requests, don't just start working on it. Prioritize and then start working on it. In this way, you will have a proper road map and tracking of all requirements.
4. If you feel your developers coding is bad then you should focus in having code review at regular intervals. In this way you can avoid bad code and prevent rework at later stages.
5. And your problem regarding stuck up in testing cycles, you should have a clearly defined process and clear targets for completing each process.

I think first you should study the process which you follow, identify gaps in your process, and try to solve the fundamental problems.

Answer 7

I ran into a very similar problems about two years ago. In my case, after we had hammered out the scope and had finalized the requirements, the endless cycle kept continuining due to lack of a formal bug tracking system.

As are result, after UAT, when the business users would provide their list, there was a large overhead keeping it in sync with our tickets (our bug tracking system was not client-facing). Example: Users provides bugs 1, 2, and 3. On our side, it would be tickets ABC, DEF, and GHI. There were spreadsheets that kept a track of the users' bugs, our corresponding tickets, and then tickets we opened for issues found internally.

Long story short, the moment we implemented a consolidated bug tracking method (which meant pausing the project for a while), things picked up very quickly and we were able to wrap up the project in 4 weeks.

To not freeze the project would have been a classic case of, "the lumberjack not sharpening his axe because there were too many trees and he didn't have enough time".