In an scaled Scrum with multiple teams, who should be responsible for writing automated system tests?

Question

We have three teams contributing to the same product. Currently our automated system tests are being written by two QA people that see what has been finished in the reviews and then write appropriate tests.

That has been working okay, but we are now pushing towards continuous deployment and need to ensure that the system tests are written during the Sprint so that we can immediately run regression tests. Since none of the teams fully own the feature right now we are unclear on who should be responsible for system testing.

Answer 1

The Scrum Guide states that a Scrum team should have all the skill necessary to deliver a product increment.

The idea is that a team will take one or more features and fully implement them by the end of a sprint. Scrum teams own the features they are working on and this will include system testing and deployment.

When you have multiple Scrum teams working on the same product then a common practice is to have each team working on a feature theme (for example, one team could be working on security features and another might be working on personalisation features).

Regardless of how the work is divided between the teams the approach is the same. Each team takes full responsibility for the delivery of a feature until it is potentially releasable.

In you situation I would suggest you consider:

• Having the system testers join your Scrum teams
• Having Scrum teams own features end-to-end
• Divide work between your teams based on a criteria such as feature themes (empower the teams to work out the best approach to dividing the work)

You may also want to reconsider the way you do your automated testing. A lot of automated testers I have worked with will start on the tests at the same time as development starts. They do not wait for development to finish before they start to create the automated tests.

A common approach is for the developers to quickly produce a stub release that mimics the feature they are about to develop. The testers then immediately get started on writing a test. Once the developers have finished development they swap out the stub and replace it with the real code.

Answer 2

Each Scrum Team Tests Its Own Work

A lot depends on your scaled framework, but in general the answer will be that each individual Scrum team is responsible for testing its own work. For example, when using Nexus:

• The Scrum Teams are responsible for developing Increments of potentially releasable software, as prescribed in Scrum. [NB: A completed increment should encompass the full Definition of Done, including testing the increment.]

• The Nexus Integration Team takes ownership of any integration issues. It is accountable for successful integration of all work by all Scrum Teams in a Nexus. Integration includes resolving any technical and non-technical cross-team constraints that may impede a Nexus’ ability to deliver a constantly Integrated Increment. [NB: Testing each Scrum team's increments for them is not a cross-team function that belongs to the Nexus team. The proper focus of a Nexus is the integration between teams.]

In Nexus, there might be members of the Nexus team who are responsible for testing the integration of the work between teams. However, the assumption is that each Scrum team is delivering functional, tested work to be integrated into the Nexus increment.

Don't Externalize Testing

If you're trying to shave costs or take some kind of operational shortcut by externalizing testing activities, you're not following a scaled Scrum approach. You're not even following agile best practices at that point. Instead, you're taking a step backwards from integrated, cross-functional teams towards a more traditional waterfall approach where development and QA are separate and sequential functions.

Don't do that.

Answer 3

Who should be responsible for writing the automated tests?

The developer who writes the code. and preferably, before the production code is written.

Call me a TDD zealot if you like, but I've been able to measure the difference in test coverage between code I wrote TDD and when I wrote the tests after the code was finished. Branch coverage is better by ~20% in my admittedly unscientific research (with an admitted sample size of 1 developer).