Project Management Stack Exchange is a question and answer site for project managers. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

One particular client has a very traditional, conservative approach to UAT and deployment. When the development team completes the items in the sprint, this is all packaged into a release, which must wind its way through a UAT/Staging environment, where it is fully tested for new functionality, then fully regression tested, then finally deployed to Production. The reason for this approach is that the Staging environment contains a copy of the production data, which is too sensitive for developers, and the company is accustomed to a waterfall-style approach. Note that this cannot be changed, so please do not suggest some kind of organizational overhaul. The UAT testers are also not part of the Scrum team. This also will not change.

Even though unit testing, regression testing, and other forms of testing are incorporate into the sprint, and the product owner is very involved, there is no way around this lengthy UAT process at the end. Naturally, this results in many failures reported by UAT, and the items get rejected and sent back to development. The typical UAT cycle is about 1 week.

I'm looking for some good ideas for how to structure Scrum sprints in that type of climate/environment. Again, please do not suggest that the company change, because that is outside the scope of this discussion. This is more of a "work in the real world with clients" question. They love agile practices, backlog management, refinement, sprint planning, release planning, and all of the other Scrum goodies. We just need to tack this 1 week of UAT and rejection onto the end, and I'm looking for ways to accommodate this.

Switching to a different methodology is also an option, if you have some suggestions.

share|improve this question
    
I'm not sure what exactly you're trying to improve here. Interruptions? Scrum Guide compliance? I'm not also sure what are the reasons for failures that are being reported after UAT. Are they related to the quality of staging environment? If you share more details, I'll update my answer. – Bartek Kobyłecki Feb 28 at 22:48
    
Thank you. The objective would be to have less risk to delivery. So, ideally, it would incorporate the UAT cycle into the sprint, but find things for developers to do to avoid slack when the release is a low-defect version. The reason UAT is a challenge is because the project is an extension to a legacy system that has many inter-dependencies. A determination must then be made as to whether the issue was pre-existing, caused by new code, or caused somewhere downstream because A invoked B, then A invoked C, but B directly or indirectly altered the state of C, in a less than obvious way. – Pittsburgh DBA Feb 29 at 5:41
    
Scrum gives most ROI when there is no technical debt, such as unpredictable legacy systems. In such cases, if Scrum is believed to be the best approach, I would advice to make it more reliable first. At the same time you wrote that Scrum is not necessity, so my answer was more about solving a problem, not solving a problem within Scrum framework. – Bartek Kobyłecki Feb 29 at 9:36
up vote 3 down vote accepted

TL;DR

You can't put lipstick on a pig and expect it to be the belle of the ball. When faced with a process impediment that can't be changed, such as the one you're describing, you must make the process issues a visible cost to the project. As you have almost no control over the impediment and can do little to control for it, your key goal should be transparency.

Make your process visible and transparent to the team, your stakeholders, management, and the client. It is then up to them to identify the value (or lack thereof) in external processes. Your internal process will clearly identify the value that's being added by the team, and the lead time and process overhead added by any externalities.

It is senior management's job to control for (or accept) process externalities. As long as everyone keeps that in mind, things will run much more smoothly.

Externalize the UAT Feedback Loop

Naturally, this results in many failures reported by UAT, and the items get rejected and sent back to development.

This is a problem, but it is an externality. You have an external process that is adding process overhead (and therefore costs) to your bottom line. The correct way to handle this is to make those costs visible to the stakeholders.

In your specific case, I would remove UAT from the Definition of Done. Work is completed for the Sprint when it passes unit testing, continuous integration, peer review, or whatever else is in your Definition of Done. The work done is then presented as a completed increment to the customer at the end of the Sprint.

The customer can then spend your next Sprint performing as much or as little UAT as they like while you work on the next increment, and any problems they may uncover are slotted into the Product Backlog by the Product Owner as new work for a future Sprint. This additional lead time will be a visible artifact of their chosen process, and visibility has great merit. This is really the right way to handle this situation, and fits the central tenets of the iterative development model.

Extend Sprint for UAT with Visible Costs

As an alternative, you can simply extend your current Sprint length by two or more weeks to accomodate their UAT cycle. Part of your Definition of Done will be passing the mid-Sprint product to UAT, and then waiting for the results to come back.

The cost of idle developer time is imposed by the external process. Your job isn't to create busy work, or to perpetuate the "100% utilization fallacy." The team can use that week to update the CI server, patch workstations, do in-service training, or just read a good book. The point is that this UAT cycle has a development cost (whether in non-product work or in iterative work) that is part of the organizational framework. Just make that cost fully visible.

The main risk to this approach is that when UAT is completed, any changes or new work may not have been accounted for in the team's estimates. As a result, you may or may not have enough time to address the changes or rework within the Sprint. As a result, the visible cost to the project will be more failed Sprints (e.g. Sprints that don't meet the Sprint Goal), incomplete work which must then be returned to the Product Backlog, or Early Termination with a return to Sprint Planning.

These are acceptable outcomes in the sense that they ensure that the process remains transparent, and that the cost to the project for these processes is visible to the team, the stakeholders, and the client. Just remember that transparency is the core goal, and that there should never be any invisible work, ever.

share|improve this answer
1  
Thank you for this in-depth analysis. – Pittsburgh DBA Mar 2 at 3:50

For the record: when the Development Team is unable to deliver a potentially releasable increment at the end of the Sprint, that's a serious Scrum deviation. Let's put that issue on a side now.

What you could do:

  • involve as much of end-users into a Sprint as possible. Don't limit yourself to physical presence. Think out of the box. Example: you can record video clips presenting new features, share them as soon as possible and ask users if that's what they expected

  • if UAT take 1 week to complete think about extending Sprint length by 1 week and include them into Sprint. BUT don't wait with UAT do the last week of the sprint. Don't treat this as an opportunity for Dev Team to do even more new features, because it'll strike back with double force. Gains? Less Work in Progress and task switching. (I assume that currently Sprint is being often interrupted by those end-users' requests)

  • coach Product Owner to represent the end-users' view at the highest possible level
share|improve this answer
    
I like the extension. If you get lucky and not much gets kicked back, the debs get some time to clean house in prep for the next iteration. – RubberDuck Feb 29 at 0:22
    
This was my first inclination. The next issue would be how to explain the 3rd week to management. They would want to make sure to have those developers mostly allocated to assigned work. What do you suggest if the release is defect-free? Spikes on bugs? Additional stories would be too risky, because they would likely need the full UAT cycle (the 3rd sprint week), so there would have to be some kind of WIP limit on that 3rd week, in terms of what it could emit and get through UAT. – Pittsburgh DBA Feb 29 at 5:22
    
@PittsburghDBA I deserve a self slap for what I wrote, actually :) I was not clear enough, so let me correct myself. By including UAT tests into Sprint I didn't mean to do them all at the end, in the last week. End users (or whoever perform them) should be included as soon as possible in the process in any way possible. – Bartek Kobyłecki Feb 29 at 9:21
    
@PittsburghDBA Slack time of your developers could be spent on improving your staging environment. Did they have time to deep dive into that issue? From what you wrote I have an impression, that failures revealed during UAT are symptom of development environment, rather than misunderstandings regarding product backlog items. – Bartek Kobyłecki Feb 29 at 9:27

The reality of your situation is pretty common and has both things to do now, and stuff for the maybe distant future.

Do now

Focus on the output from the Development Team being potentially shippable. That is the new increment of software that they produce each sprint heads off to UAT and should always pass. (If it always passes then the business will see less of a value in UAT.)

1) Create a Definition of Done that represents the quality level that gets you to potentially releasable.

2) Empower your Development Team to refuse to accept backlog items that don't have adequate acceptance criteria.

This will move you towards the values and principals of Scrum without interfering with the wider organisation.

The medium term

Once you have quality and consistency you can start to focus the UAT senario into more of a feedback session. Maybe you need to bypass the rest of the chain and get real users into your sprint review, but that is where you will start to accelerate. If you can not build stuff at all that is wrong, and build more of what the users need you will build more trust.

3) Bring end users and stakeholders into your Sprint Review and adapt the backlog accordingly.

4) Invite UAT to your sprint review.

5) Insert telemetry gathering tools into your code, like Application Insights, to really understand your users behaviours.

The Long Goal

As you demonstrate ability as a Development Team to deliver working software that meets the needs of the business then the UAT and staging processes will be increasingly seen as a waste. You will the. Have both your Customers/stakeholders in your corner fighting to give the Development Teams more control and responsibility.

share|improve this answer
    
This is insightful. I don't know why, but I am always surprised, at least a little, by the pace of organizational change. – Pittsburgh DBA Mar 2 at 3:51
    
The definition of done hits what I was going to say in a separate answer: if there's an expected impediment that will be there every single sprint, it might be worth adapting your definition-of-done to this reality. Scrum doesn't require your potentially shippable increment to actually be shipped! Your definition of done might say that it has to be developed, tested and found to be ready for UAT and that might be where your process stops. The feedback from the UAT then goes into the backlog for the next sprint and when an increment passes UAT successfully, you take the time to release. – Cronax Mar 3 at 11:02

Here is what I'd suggest. When UAT cycle completes, the result of it is added to the product back log. Before the next spring planning session the product owner grooms and (re)prioritizes the product back log. During the next spring planning session the dev team together with the product owner select which product backlog items are taken into the next spring. Some issues could be critical to the PO to fix, some could be left in the product backlog longer.

share|improve this answer
    
This would be the most natural way to go, but most of the time, the policy requires an immediate fix to the release candidate story, so we have overflow in terms of sprint planning. The release would then commence as soon as possible, with new fixes, having passed UAT again. It disrupts the normal scrum pattern. – Pittsburgh DBA Feb 29 at 5:33
    
@PittsburghDBA, "the policy requires an immediate fix to the release candidate story", what does such a policy aim to achieve? I.e. what is the rationale for that? Clearly it disrupts your process/success, so there must be an upside? Same as you, I live in the real world, with real customers and their requirements. While more often than not we have to accept what they ask of us, we non-the-less should strife to explore their motivations and problems with them as thoroughly as we can, so that we can suggest a solution. So if you do not know why this policy is in place you should ask. – zespri Feb 29 at 6:04

If the productivity of the team is being strongly hampered by work being rejected a week later due to a failed UAT, then it may help to account for this in story planning. Try to restructure your stories to include this real world issue, such that if you fail UAT, you still can demonstrate valuable progress to the customer. Assuming you do succeed at the UAT from time to time, can you show your customer that the work is getting done, there just may be an extra round trip involved?

If the customer is being affected by these delays, then you do need to address the issue of your team failing to make a product which meets the needs of the customer. My next step would be to treat the UAT testers as customers. You said they aren't part of the SCRUM team, but it may be possible for them to be customers. No testing team likes failing software, maybe you can work with them to find ways to pick up the things they care about earlier in the process. If they are a customer, involved in the process, they may get ideas from this. They may say "you know, that tool you used to ensure your software worked before we tested it... think we could use that?" Then you've provided value they never expected to gain.

It all really depends on what the customer actually wants. Perhaps the issue may be handled with version control. Perhaps you have two tracks, one track adding new features, and the other track ensuring you can produce something that passes UAT using the previous set of features.

share|improve this answer
    
Thanks for this. I was also thinking of increasing the release frequency, so that they can perform UAT on completed stories ASAP, and then toward the end of the sprint, the UAT would be focused more on the integration tests and system tests. – Pittsburgh DBA Feb 29 at 5:27

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.