What does a QA team do during the development phase of a sprint in Agile Scrum?

Question

I understand that, in theory, QA should be working with dev "collaborating" right from the day 1 of a sprint. But how does it actually work in real life? Let me present two scenarios -

Scenario 1 where a story requires 3 days of development - Let's say it's a complicated report with time consuming database setup and domain logic etc and takes 3 days to code. Since the dev can't deliver anything meaningful to test before 3 days and assuming it takes only 4 hours to write all relevant test cases, what is QA expected to do until the report is ready for testing?

Scenario 2 where stories are tiny enough to be completed in hours - Let's consider a login/registration screen. There can be tiny stories for user being able to login, user being able to register, forgot password functionality etc. A dev can complete each one of these stories in hours and pass on to QA and move on to the next story but the next story might break the previous ones. For example, if dev finishes up login functionality and QA starts testing it and then dev starts working on forgot password functionality, it might the break login in unexpected ways because the functionalities are indeed related. If QA waits for all of the related stories to be completed then we end up with scenario 1 above.

In a perfect world, QA can be expected to do nothing if there is indeed nothing to do and accept it as part of the cost. But in a real world, PMO and other groups keeping track of resource utilization will certainly point it out as poor project management and worse. So, how does it all work in real life scenarios? How best to apply Scrum in these types of scenarios?

Answer 1

Test-First Practices Engage Testers/QA Throughout the Process

You're falling into a utilization fallacy by treating development and QA as separate activities. On an agile team, practices such as test-driven development (TDD), behavior-driven development (BDD), or acceptance-test driven development (ATDD) ensure that quality is baked in by writing the tests first.

In other words, the red/green/refactor pattern means that QA should be involved before any code is written at all! Even on teams where individuals are I-shaped rather than T-shaped, developers and testers should be working hand-in-hand the entire time through pair programming and continuous integration.

Collaboration and Utilization

Developers and testers should be actively collaborating, rather than working in parallel or in sequence. Even if your organization falls prey to the 100% utilization fallacy, there are always things that testers need to do on an agile team including:

1. Refactoring tests.
2. Updating test fixtures and harnesses.
3. Analyzing code coverage and other continuous integration (CI) metrics.
4. Replacing fixtures with factories.
5. Exploratory and manual testing.
6. Working with stakeholders on building executable acceptance tests (e.g. Cucumber scenarios).
7. Identifying boundary conditions.
8. Linting and style checking.
9. Fuzzing.
10. All the other cool and important stuff that QA people do.

I'm not in any way recommending that you stovepipe your processes. I'm simply pointing out that there's always plenty of testing and quality assurance work to do, even absent new code or features.

Cross-Train Your Team

It's also smart money for the project to invest in cross-training developers and testers. Many developers can benefit from learning about testing techniques, so that they can write more testable code. Likewise, testers can benefit from learning more about development, which makes them better testers and more well-rounded resources.

Answer 2

Along with the "agilisation" of projects, the role of QA is "evolving" considerably.

I see Todd's scenario (+1!) as the target state for a mature QA team. Hope your team can make it to it directly. There are cases, however, where the QA team is less mature and requires a few maturity steps in between.

In a perfect world, QA can be expected to do nothing if there is indeed nothing to do and accept it as part of the cost.

There's no ideal world where someone is doing absolutely nothing.

Abraham Lincoln is quoted as having said

“If I had five minutes to chop down a tree, I’d spend the first three sharpening my axe.”

That's what any QA, regardless of how mature a process or a team is, should be doing while not doing... well, QA.

---

So, how does it all work in real life scenarios? How best to apply Scrum in these types of scenarios?

Step 1: Mindset change.

QA is no longer a team responsible for executing test cases. QA is responsible to understand in dept how the application works and foresee the potential use cases that could fail and that the developer may not be aware of. In the past, this was heavily done by (functional) analysts. This may no longer be the case. In some aspects, there's an expectation that QA will become the reference for functional knowledge. it's debateable, but it's important to consider this.

Step 2: Sharpen up your skills.

Python is becoming one of the most useful languages nowadays. Developers are using more Python for data analysis than web development. You know who could also benefit for knowing the basics on data analysis? Yes, QA.

Step 3: Target the future.

Applying steps one and two, you might be halfway through with a team capable of fit into Todd's QA team.

Answer 3

You asked what the collaboration looks like and that is the important question. In Scrum, there doesn't need to be a QA phase. Practices like 3 Amigos meetings, Test Driven development, Behavior Driven Development, and pairing mean that the QA team members can be collaborating with the team the whole time. To whatever degree we can, we want to use the Lean principle of Build Quality In rather than checking for it afterwards and for that, we need the QA perspective through the whole process.

Answer 4

Agile mindset and way of working is very different from traditional mindset and way of working. This sentence shows "PMO and other groups keeping track of resource utilization will certainly point it out as poor project management and worse." that your company has long way until you can call yourself agile. I suggest you to read Agile Manifesto as a team activity and to find out what it means for you as a team.

Another thing what seems you are missing: Testing and QA is not the same. Testing is way how we analyse software with aim to find errors we already build in. QA stands for all activities we set with aim to prevent software from built in bugs - no bugs, no need for fixing them! I wrote a piece on it.

In agile team tester can take both roles - finding errors (testing) and setting rules which protect software (QA). For testing you need a software, for QA you don't need a software.

In real life agile testers day could have activities like this:

1. questioning user story,
2. analysing acceptance criteria,
3. working on examples,
4. writing ATDD,
5. pair programming,
6. helping to write unit tests (the hardest part in writing unit test is to decide what to test),
7. static testing,
8. writing automated e2e tests,
9. generating test data,
10. preparing exploratory testing charters,
11. educating team in testing techniques and approaches.

I hope it helps.

Answer 5

Can they get things ready to test so when stories/bugs are completed by developers they will be ready to test? Setting up servers, databases or data so they can hit the ground running when it's time to test.

We have multiple clients all with their own database. Our QA team knows which stories/bugs affect certain clients when the sprint starts. So our QA team creates database backups for those specific clients and gets testing databases ready in our testing environment.

I found this article interesting: https://www.352inc.com/blog/what-does-qa-do-on-the-first-day-of-a-sprint/

Encourage your QA person to learn new skills to help move stories to QA. Options > include:

• Learn to program

• Do some design work

• Perform some user tests

• Server setup and management

Answer 6

All of the points you make in your question are valid, but these problems are solvable by careful collaboration within the Scrum team.

An example of a conversation during sprint planning:

"Some of these stories will take a few days to develop before they are ready for testing. But a few of the other stories are quite quick to do. Why don't we do a few of the smaller stories first, so that the testers have something to get on with? Then they can work on the first big story when they are ready."

Also:

"I am worried that a lot of our stories are dependent on each other. That is going to make planning testing really tricky. Shall we refactor the stories to make them independent so that testing runs smoother?"

And:

"This sprint is going to be quite light on testing. Why don't we spend any extra time available expanding our automated regression coverage?"

As the Scrum team matures it may well adopt practices like test driven development and behaviour driven development. You may also get cross-training between developers and testers so that the team members have t-shaped skills profiles.

With more experience and improved working practices the team will become better and better at balancing development and testing. Eventually there will be little distinction between what is considered coding and what is considered testing.

Answer 7

Milestones vs. User Stories

The very-critical thing that I meant to say, at the beginging of the above answer, was this:

If your developers need to embark on a multi-step development advance that is of some complexity such that they wish to break it down into milestones apart from "user stories," then I suggest that they should create tests as they are developing the code. (As well as any necessary "stubs" and so on.) The milestone is reached when all of their tests are passed, and they move on from that point while constantly checking that the tests continue to pass, so they know immediately if they just broke something. From time to time, they may retire one of their tests. I present these as separate from the activites of a QA team which is focused on "what will finally be visible out-the-door." All of the foregoing – "Test-Driven Development" – might well be the purview of the development manager, not the higher-level project manager, and seen as being parallel to ("user acceptance" oriented ...) QA.

There are two tiers of "assurance" going on here at the same time – both very similar, yet very different. But the project manager must be at all times aware of what is going on at both levels. And, the two levels should be distinct!

In my opinion, then – the two levels are not (yet) distinct. If there were, then a "release-candidate release process from the development team" would very clearly be in place, and there would be no question as to what the "code to be QA-approved" consisted of.

The specific reason why "the two levels are not (yet) distinct" is exactly as the edited-out paragraph stated: in a software project of any size, there must be two distinct tiers. The QA-team, operating entirely at level-two, should not be involved in – nor, in any way impacted by – the internal (but very necessary) processes that might be happening downstairs at level-one. But the development teams should also never be releasing "candidates" up to level-one QA until they are sure.

My suspicion is that the development team is "racing ahead," from one milestone known only to itself, to another, somehow expecting the QA team to evaluate a target that is moving under its feet. Obviously this is nonsense.

Task Breakdowns and QA Workflows

In addition, I see from your description that the functionality which your QA team is supposed to test is not on a stable version-control branch that will not be affected by future development activities. Developed material should be added to QA branch(es) and tagged so that QA is always testing against a non-moving target.

But also: when you say that "a future story breaks an earlier one," then I would say that you have a much larger task-breakdown problem. "Stories" are supposed to represent what a hypothetical user sees, but they also should reflect the behavior of the to-be released application, not an intermediate step.

"Stories" do have to be composed into what will be the actual workflows and milestones four your development teams – they might not exactly coincide because "story-tellers don't necessarily know how it works." What actually drives your development schedule should be a sometimes-parallel pursuit of completed features, which can then be meaningfully tested such that "regression" from that tests-passed state is not to be expected. It is possible that a single development-workflow increment might complete "part of a story," or "part of several stories." In practice, the architecture of the application – particularly that of a legacy application – dictates this.