Client with conservative UAT policy

Question

One particular client has a very traditional, conservative approach to UAT and deployment. When the development team completes the items in the sprint, this is all packaged into a release, which must wind its way through a UAT/Staging environment, where it is fully tested for new functionality, then fully regression tested, then finally deployed to Production. The reason for this approach is that the Staging environment contains a copy of the production data, which is too sensitive for developers, and the company is accustomed to a waterfall-style approach. Note that this cannot be changed, so please do not suggest some kind of organizational overhaul. The UAT testers are also not part of the Scrum team. This also will not change.

Even though unit testing, regression testing, and other forms of testing are incorporate into the sprint, and the product owner is very involved, there is no way around this lengthy UAT process at the end. Naturally, this results in many failures reported by UAT, and the items get rejected and sent back to development. The typical UAT cycle is about 1 week.

I'm looking for some good ideas for how to structure Scrum sprints in that type of climate/environment. Again, please do not suggest that the company change, because that is outside the scope of this discussion. This is more of a "work in the real world with clients" question. They love agile practices, backlog management, refinement, sprint planning, release planning, and all of the other Scrum goodies. We just need to tack this 1 week of UAT and rejection onto the end, and I'm looking for ways to accommodate this.

Switching to a different methodology is also an option, if you have some suggestions.

Answer 1

TL;DR

You can't put lipstick on a pig and expect it to be the belle of the ball. When faced with a process impediment that can't be changed, such as the one you're describing, you must make the process issues a visible cost to the project. As you have almost no control over the impediment and can do little to control for it, your key goal should be transparency.

Make your process visible and transparent to the team, your stakeholders, management, and the client. It is then up to them to identify the value (or lack thereof) in external processes. Your internal process will clearly identify the value that's being added by the team, and the lead time and process overhead added by any externalities.

It is senior management's job to control for (or accept) process externalities. As long as everyone keeps that in mind, things will run much more smoothly.

Externalize the UAT Feedback Loop

Naturally, this results in many failures reported by UAT, and the items get rejected and sent back to development.

This is a problem, but it is an externality. You have an external process that is adding process overhead (and therefore costs) to your bottom line. The correct way to handle this is to make those costs visible to the stakeholders.

In your specific case, I would remove UAT from the Definition of Done. Work is completed for the Sprint when it passes unit testing, continuous integration, peer review, or whatever else is in your Definition of Done. The work done is then presented as a completed increment to the customer at the end of the Sprint.

The customer can then spend your next Sprint performing as much or as little UAT as they like while you work on the next increment, and any problems they may uncover are slotted into the Product Backlog by the Product Owner as new work for a future Sprint. This additional lead time will be a visible artifact of their chosen process, and visibility has great merit. This is really the right way to handle this situation, and fits the central tenets of the iterative development model.

Extend Sprint for UAT with Visible Costs

As an alternative, you can simply extend your current Sprint length by two or more weeks to accomodate their UAT cycle. Part of your Definition of Done will be passing the mid-Sprint product to UAT, and then waiting for the results to come back.

The cost of idle developer time is imposed by the external process. Your job isn't to create busy work, or to perpetuate the "100% utilization fallacy." The team can use that week to update the CI server, patch workstations, do in-service training, or just read a good book. The point is that this UAT cycle has a development cost (whether in non-product work or in iterative work) that is part of the organizational framework. Just make that cost fully visible.

The main risk to this approach is that when UAT is completed, any changes or new work may not have been accounted for in the team's estimates. As a result, you may or may not have enough time to address the changes or rework within the Sprint. As a result, the visible cost to the project will be more failed Sprints (e.g. Sprints that don't meet the Sprint Goal), incomplete work which must then be returned to the Product Backlog, or Early Termination with a return to Sprint Planning.

These are acceptable outcomes in the sense that they ensure that the process remains transparent, and that the cost to the project for these processes is visible to the team, the stakeholders, and the client. Just remember that transparency is the core goal, and that there should never be any invisible work, ever.

Answer 2

For the record: when the Development Team is unable to deliver a potentially releasable increment at the end of the Sprint, that's a serious Scrum deviation. Let's put that issue on a side now.

What you could do:

• involve as much of end-users into a Sprint as possible. Don't limit yourself to physical presence. Think out of the box. Example: you can record video clips presenting new features, share them as soon as possible and ask users if that's what they expected

• if UAT take 1 week to complete think about extending Sprint length by 1 week and include them into Sprint. BUT don't wait with UAT do the last week of the sprint. Don't treat this as an opportunity for Dev Team to do even more new features, because it'll strike back with double force. Gains? Less Work in Progress and task switching. (I assume that currently Sprint is being often interrupted by those end-users' requests)

• coach Product Owner to represent the end-users' view at the highest possible level

Answer 3

The reality of your situation is pretty common and has both things to do now, and stuff for the maybe distant future.

Do now

Focus on the output from the Development Team being potentially shippable. That is the new increment of software that they produce each sprint heads off to UAT and should always pass. (If it always passes then the business will see less of a value in UAT.)

1) Create a Definition of Done that represents the quality level that gets you to potentially releasable.

2) Empower your Development Team to refuse to accept backlog items that don't have adequate acceptance criteria.

This will move you towards the values and principals of Scrum without interfering with the wider organisation.

The medium term

Once you have quality and consistency you can start to focus the UAT senario into more of a feedback session. Maybe you need to bypass the rest of the chain and get real users into your sprint review, but that is where you will start to accelerate. If you can not build stuff at all that is wrong, and build more of what the users need you will build more trust.

3) Bring end users and stakeholders into your Sprint Review and adapt the backlog accordingly.

4) Invite UAT to your sprint review.

5) Insert telemetry gathering tools into your code, like Application Insights, to really understand your users behaviours.

The Long Goal

As you demonstrate ability as a Development Team to deliver working software that meets the needs of the business then the UAT and staging processes will be increasingly seen as a waste. You will the. Have both your Customers/stakeholders in your corner fighting to give the Development Teams more control and responsibility.

Answer 4

Here is what I'd suggest. When UAT cycle completes, the result of it is added to the product back log. Before the next spring planning session the product owner grooms and (re)prioritizes the product back log. During the next spring planning session the dev team together with the product owner select which product backlog items are taken into the next spring. Some issues could be critical to the PO to fix, some could be left in the product backlog longer.

Answer 5

If the productivity of the team is being strongly hampered by work being rejected a week later due to a failed UAT, then it may help to account for this in story planning. Try to restructure your stories to include this real world issue, such that if you fail UAT, you still can demonstrate valuable progress to the customer. Assuming you do succeed at the UAT from time to time, can you show your customer that the work is getting done, there just may be an extra round trip involved?

If the customer is being affected by these delays, then you do need to address the issue of your team failing to make a product which meets the needs of the customer. My next step would be to treat the UAT testers as customers. You said they aren't part of the SCRUM team, but it may be possible for them to be customers. No testing team likes failing software, maybe you can work with them to find ways to pick up the things they care about earlier in the process. If they are a customer, involved in the process, they may get ideas from this. They may say "you know, that tool you used to ensure your software worked before we tested it... think we could use that?" Then you've provided value they never expected to gain.

It all really depends on what the customer actually wants. Perhaps the issue may be handled with version control. Perhaps you have two tracks, one track adding new features, and the other track ensuring you can produce something that passes UAT using the previous set of features.