Should a Product Owner start testing while an item is in code review

Question

We have a Product Owner (PO) who starts testing while an item is in code review and gives feedback to developers. I see, from the point of view of the PO, that this allows for quicker feedback. However, I don't think she should start testing while code is still in the review stage, but rather wait until the item is ready for testing because we have testers.

I've got feedback from the Team that "Tasks' turn-around time is slower (testing seems to be slow at times)"

Any ideas on this topic?

My idea/solution is:

• Ask the PO to wait until the item is in the testing phase, keep to the Sprint timebox, and don't become a bottle neck.

I know this is an obvious question but I would like to hear if anyone has encountered something similar and what did they do to fix it.

Answer 1

The answer depends on whether this slows down or speeds up the process.

How many of the bugs she found were discovered during the Code Review phase?

If they were all discovered during the Code Review then maybe she's wasting her (and everybody's time). Though we can still ask:

How many of the bugs she found were fixed before the Code Review stage?

If they all were, then she is saving you time by having you review corrected code with fewer bugs.

And then there's the Type Of Bugs she finds that makes a difference.

A Product Owner would typically check the product for UI issues, wording, typos and aesthetics.

These "bugs" will not be picked up by Code Reviews or the QA department, as they are mainly subjective. (With the exception of typos.) So the sooner she fixes these issues the less work (of retesting) for all involved.

If, on the other hand, she's only finding hard to find bugs that a good Code Review would have picked up in much less time, then maybe it's worth pointing out to her that she's wasting her time.

One last thought: If she's simply taking the product for a quick test-drive and (since she found them) reporting all the trouble she finds, then maybe your coders need better work ethics; if they tested it for a minute before releasing it they would also find and fix these obvious bugs.

Answer 2

As a general rule I would say that the earlier the Product Owner starts testing the better.

The reason for this is that a lot of bugs happen as a result of misunderstanding of requirements. The Product Owner is uniquely well placed to review and give feedback to a team.

Ask PO to wait until the item is in the testing phase and keep to sprint timebox and don't become a bottle neck

If the Product Owner discovers an issue in the testing phase, re-work may be required and a repeat of the code review. This could be considered waste if it was possible for the Product Owner to discover the issue earlier in the development cycle.

Some Scrum teams I have worked with got to the point of demonstrating features to the Product Owner during development, let alone waiting for the testing cycle to begin.

If the team currently finds it disruptive to receive early feedback from the Product Owner then I would suggest looking at ways to improve the development process to make this less of a problem.

My recommendation would be to build your development process around a flexible and early feedback cycle. One approach I have seen taken is to have an environment specifically for the Product Owner. When the development team feels they have a stable build they release it to the Product Owner's environment and let her know that there are new features to review. If there are known issues then they should be clearly communicated and it is the responsibility of the Product Owner to not report duplicates.

Communication and teamwork are critical to making this work.

Answer 3

In favor of eliminating waste I'd suggest waiting, too.

Assuming a PO's time is more expensive/valuable than a tester's or dev's, simply because of supply and demand as there is only one PO, but multiple devs/testers, the following would be a waste of good PO time:

• Reporting errors that would have been found by a developer anyway
• Reporting errors that would have been found by testers anyway
• Testing the same things again after scheduled changes (Code Review) have been made

In addition, developers might be a little miffed if their work is criticized before they declared it "done". You don't tell a cook their dish is not tasty dragging it out of the oven when the cook insists it's only half-cooked and needs anther 20 minutes. You don't tell people they park like idiot's when they are still in the street moving. In other words, don't tell people they made mistakes when they are not finished yet. It can only be a "bug" if it should work in the first place.

So filing bug tickets will generate wrong metrics afterwards. Many bug tickets means the developers need to work better. But in this case, they weren't even done yet. So your metrics are skewed.

But as always in Scrum: discuss it with the team, see what everybody has to say and find a solution that works for everybody.

Answer 4

Acceptance Testing Isn't the Product Owner's Role

It's a good thing that you have a Product Owner that wants to be involved in Product Development and the QA/UAT process. However, while collaboration is great, testing is not part of the Product Owner role in Scrum.

The Scrum Guide says:

The Product Owner is the sole person responsible for managing the Product Backlog. Product Backlog management includes:

• Clearly expressing Product Backlog items;
• Ordering the items in the Product Backlog to best achieve goals and missions;
• Optimizing the value of the work the Development Team performs;
• Ensuring that the Product Backlog is visible, transparent, and clear to all, and shows what the Scrum Team will work on next; and,
• Ensuring the Development Team understands items in the Product Backlog to the level needed.

What your Product Owner is doing is not intrinsically doing any of these things, and isn't leveraging framework processes and practices to ensure quality.

What to Do Instead

The Product Owner should certainly collaborate with the Development Team. This includes helping the Scrum Team craft the Definition of Done, set Sprint Goals, and participate in Sprint Reviews and Retrospectives. However, the Product Owner's focus is on defining the work and optimizing value, not performing QA or acceptance tests.

As a Product Owner, especially when part of an immature Scrum Team, I may ask the Development Team to demo critical features that meet the Definition of Done throughout the Sprint rather than waiting until the very end. Or, I may ask them to do a "prep demo" within the team ahead of the formal Sprint Review in order to make sure we're all on the same page.

As the Product Owner, I don't get to approve the work. Instead, I work through the Product Backlog to define work, and through the Definition of Done to set expectations about quality. Most importantly, I work through the Sprint Goal to set expectations about what qualifies as a delivered or non-delivered increment of work.

If my Sprint Goal has been met in accordance with the Definition of Done, then the work increment has been successfully delivered. If the Sprint Goal has not been met, then the work increment has not been successfully delivered. That doesn't mean work hasn't been done, or that various bits of work haven't been done properly; it just means that the iteration has not achieved the scope and value planned for it.

As the Scrum Master, part of your job is to educate the Product Owner on the responsibilities of the role. Help the Product Owner use the tools of the framework properly, and to collaborate effectively with the team.

Building Trust

If the Product Owner is routinely testing the product or its features through each Sprint, this may be an indication that the Product Owner perceives an ongoing quality issue or lacks trust in the Development Team. This is an opportunity to improve communications, re-evaluate the Definition of Done, or otherwise inspect-and-adapt the process to increase trust between the key team roles.

Likewise, if the Product Owner is routinely providing feedback at the implementation level (rather than at the system or feature level) within the Sprint, this is a framework implementation smell that may indicate poor Backlog Refinement, poor Sprint Goals, user stories that lack adequate context, or user stories that don't meet INVEST criteria. If that's the case, then working with the Product Owner and the Development Team to improve the contents of the Product and Sprint Backlogs, and to improve adherence to the INVEST criteria (especially the Testable requirement), will often yield significant process improvement.

In addition, non-developer inspection of an increment other than at framework inflection points can also lead to scope creep. For example, if the Product Owner is inspecting a feature that meets the Definition of Done but requests in-Sprint changes, this may represent a change of scope or design that wasn't part of the current iteration's Sprint Planning. Developers should actively collaborate with the Product Owner to define scope and design issues, but any feedback that changes scope or puts the Sprint Goal at risk needs to be carefully negotiated. An obsolete Sprint Goal, or significant changes in scope, may necessitate cancelling the Sprint and returning to Sprint Planning.

In each of these examples, the Scrum Master's key objective is to help the entire Scrum Team build confidence and trust. This means:

• The Product Owner needs to be confident that the Development Team's process will successfully deliver the Sprint Goal.
• The Development Team needs to be confident that they're being given a clear problem with adequate context, and that the information is sufficient to identify and implement a solution within a single iteration.
• The Development Team needs elbow room to find the simplest design and implementation that will meet the Sprint Goal.

Your goal as the Scrum Master is facilitate trust, which is built through visibility, transparency, and open communications. Leverage the Sprint Retrospective and other framework elements as much as possible, but always ensure that you are solving for any real underlying trust and communications issues, rather than just addressing the symptoms.

Answer 5

I don't see how this is a problem in the first place.

If the problem is that the PO is not available to give feedback or answer questions then this is an entirely different problem. If the PO is testing things and finding bugs then they're saving your team time and effort.
So if they're still available to do all the things you need them to do let them do whatever testing they want and be grateful for their input.

Answer 6

My idea is to advise the product owner to start testing the user experience and functionality once the coding and testing is completed by both developers and testers. The primary responsibility of the PO is to find out any difference in the functionality as per the user story. So the precious time of PO can be utilized efficiently.

Answer 7

Todd A. Jacobs nailed it. I find that if a PO does not trust in the development and testing teams, there is a deeper issue there that needs to be sorted. This could lead to moral issues among personnel, as well as the PO mismanaging their own functions, and the project as a whole.