QA Code freeze and setting the expected delivery timeline

Question

We are in the healthcare industry, where bugs can have an impact on people health. As a result, we started having a code freeze on the staging environment to have the QA test each build. In addition:

• We don't deploy on Fridays so developers will not have to fix urgent bugs over the weekend
• We do 2 weeks sprints
• The QA asked to have the code on Thursday morning on staging for testing

As a result, code that wasn't completed by Thursday morning is not being released in that release. In other words, all the code developer code over Thursday-Friday isn't being delivered in the current sprint.

Because of that, I proposed to the stakeholders to consider separating the engineering sprint and the code delivery, setting the expected sprint delivery to a few days after the sprint ends. As expected, I got a lot of pushback from the stakeholders, telling me that they never heard of anything like it and that I break the Scrum process. Trouble is that I agree with them, but I can't think of any other way, apart from committing to less each sprint and having developers work on other things (support, next sprint tickets) in the last couple of days.

Anyone with a similar setup - how did you resolve it?

Answer 1

Having worked in two regulated environments (aerospace and pharmaceutical/healthcare), I've seen the problem of needing independent quality assurance in a deliverable.

A few things to consider:

Build QA time into your Sprint. My current organization used to run two-week Sprints. We began on a Wednesday afternoon (some teams begin on Thursday morning) with Sprint Planning and have development through the next week's Friday. Code freeze was Friday afternoon. QA has at least 2.5 days (Monday after code freeze until Wednesday afternoon) of testing exclusively on a release candidate. We recently modified this to begin Sprints with Sprint Planning on Wednesday afternoon or Thursday morning (depending on the team). For a given release, code freeze occurs with the Sprint Review on Wednesday. If there is a scheduled release (we don't release after every Sprint), this is planned as part of Sprint Planning with the fact that QA staff will be running manual test cases or ad hoc testing and automation team members will be reviewing and documenting automated acceptance and performance test cases. Any necessary capacity and support from the team is known going into the Sprint Planning.

Involve QA in every step of the process. When your product management team is starting to develop ideas and requirements (stories, backlog items), get QA involved so they can provide feedback on testability. When you refine these requirements, get QA involved to review. If you are using acceptance criteria, have QA help to write them to make sure they are complete. When you are doing estimation, including QA effort in the estimation. When you plan your work, include QA to ensure that they are confident that the planned work can be done (developed, tested appropriately by development, and go through an appropriate QA process) within the Sprint.

Involve developers in quality. In an environment where developers cannot test their work, they can still do valuable work after a code freeze. Bugs (or questions) may come up - the developers need to work with QA to fix issues and address concerns. Otherwise, they can improve automated test coverage to prevent regressions in future changes or improve developer tooling and infrastructure. If there's not much in improving quality, the developers can focus on risk reduction for upcoming work by learning and improving the skills or prototyping upcoming functionality.

You do have to make changes to Scrum to make it work in a regulated environment, but I would not break Scrum. Ensure that your increment is done (by development and quality assurance) at the end of the Sprint. Ensure that your Development Team includes all of the skills needed to deliver the work and everyone's estimation and effort are being considered.

Answer 2

The way I see it your problem comes down to the definition of done. Does your DoD contain the requirement that your code has passed QA?

If so, then however long your QA needs to vet the latest build automatically becomes slack time for your devs at the end of the sprint. That's not necessarily a bad thing. You can do backlog refinement, research, learning, refactoring, etc in that time (if no bugs turn up). You just can't push out new builds. You can still deploy on the friday in the middle of the sprint. Bugs would simply get fixed when the devs are back in on monday. If this is how you want to roll you should however do minimize the time QA needs for their work. Your feedback cycle should be as short as possible.

If your DoD does not contain the requirement for QA testing then your proposed solution is the correct one: Your team will push out new builds whenever they have them. The last build occuring in the last days of the sprint whenever you are done. Whatever issues QA finds are then treated like new backlog items. They will be included into sprints according to their priority just like any other task. If QA finds a particularly critical bug then the PO can renegotiate the scope of your current sprint to include that issue (generally at the expense of something else).

Answer 3

Analysis

When it comes to managing the release cadence, a business really only has three choices, regardless of industry or framework:

1. Fully integrate QA with development.
2. Accept that either developers or testers will be idle some part of each Sprint when practicing Scrummerfall.
3. Decouple releases from delivery or deployment (more on that below).

While high-compliance environments aren't orthogonal to agility, the demands of the compliance requirements can sometimes force a business to face the fact that you can't have disparate processes that move at different speeds run in lockstep. High-compliance environments also tend to fall prey to the 100% utilization fallacy, so the idea of having sufficient slack in the process to allow disparate processes to deploy on the same cadence is unrealistic.

Decoupling, when supported by the right engineering and process controls, is usually the sanity-preserving way out.

Differentiate "Delivery" from Deployments and Releases

While it's generally best to integrate QA directly into your Sprints, a lot of businesses overlook the value of decoupling delivery and deployment from release. While Scrum mandates that the output of a Sprint is a potentially-shippable increment, it doesn't actually require that the increment be released to end-users.

In a regulated environment, agile teams are still responsible for delivering increments that meet the Definition of Done. However, downstream processes such as third-party testing or production releases do not have to move in lock-step with the development team.

The main issues you are likely to face with decoupling deliveries from your other processes are: the need to negotiate bug-fixing with the business, and the assignment of responsibility for releases outside of the Scrum team.

In canonical Scrum, bugs found outside the development cycle must re-enter the Product Backlog. The business contracts with the Scrum team not to bypass the backlog and estimation process, although early terminations (costly as they are) certainly remain an option.

Furthermore, a truly decoupled release plan means QA or some other team will own the release. Whether the business tries (and likely fails) to have every delivered increment tested and released, or whether they more sensibly give the release teams the authority to batch up deliveries and set their own cadence for tagging and deploying releases is strictly a management decision.

There are certainly variations on these themes, but the core tenet is that each Sprint must maintain its integrity against unplanned work, and that any work moved outside the team must abide by that. So long as that's true, decoupling releases is the approach I advocate in healthcare and government settings.

Answer 4

Can't QA be integrated in your scrum team? This not only means you integrate one person that does QA, but you spread the QA knowledge in your team.

The main goal building a scrum team, is that the team is able to build (or even better deploy) a product increment on its own. Further there are no other roles in the team than developer. So your team can have a QA specialist, but all other developers should be able to support in QA. The other way around your QA specialist should be able to support development. Perhaps, to accomplish this, you have to manage the knowledge transfer.

This way you could integrate QA tasks completely in your sprint planning. Anyone in your team could handle or at least support any type of task, development and QA. At any time in the sprint you can do QA for finished features. At any time in the sprint you can react on QAs feedback on development.

Perhaps its also worthy to define what you mean by QA. Is it really quality assurance or more like user/stakeholder acceptance testing?