How can we continuously integrate new features when the PO only determines whether a feature is "done" at the end of each Sprint?

Question

My team is having an issue with integrating software product features. Specifically, when do features get pull requested into our integration branch of code?

Currently our understanding is:

• The Product Owner is the only person who can say that a feature or User Story is "done".
• The Product Owner makes his decision on whether a User Story is "done" at the end of sprint, during the Sprint Review Meeting.

With the points above, it appears that pull requests for our "done" features or User Stories should occur at the end of our sprints. But how can this be the case?

If we merge several features together after we get the Product Owner's approval, it may be the case that the features conflict or break each other. Now the features are no longer "done", additional work to resolve conflicts must be done, etc.

How can we continuously integrate our new features (allowing us to do realistic integration testing) while retaining the fact that the Product Owner is the final say of whether or not a feature is "done"?

Answer 1

The team says the story is done. Think of this on a storyboard where the team marks the story as ready to pull (done or completed) into accepted status.

The Product Owner accepts the story, (by pulling it into accepted) but their acceptance only indicates the story is done in terms of business functionality.

So acceptance from a product owner, is only partial to saying the story is done. Done goes beyond PO acceptance because it encompasses the non-functional portions of finishing a story as well. QA, tech debt, etc.

One model for handling integration and done stories is to land these stories into a staging environment. Staging is a clone or close replica of your production environment. Its also where integrating teams would go to consume any code. Your Product Owner uses this environment to validate that the functional requirements of the story are met and accept the story. This doesn't need to happen during the review ceremony. It can happen anytime throughout the iteration, but ideally is completed soon after the iteration finishes (otherwise you hold your stage environment hostage and can't check in new features).

In order to get to staging, all your done criteria (functional and non-functional) must already be completed.

I'm simplifying a bit with this model as there are done criteria that can happen in production in some situations. But the majority of done criteria must be met by the time code goes to stage.

Answer 2

From Scrum.Org:

The DoD is usually a clear and concise list of requirements that a software Increment must adhere to for the team to call it complete. Until this list is satisfied, a product Increment is not done. During the Sprint Planning meeting, the Scrum Team develops or reconfirms its DoD, which enables the Development Team to know how much work to select for a given Sprint.

The key sentence there for me is that the Scrum Team (especially the Product Owner) develops and confirms the DoD. It is an agreed standard prior to the Sprint work commencing.

If the Product Owner does not consider the Definition of Done to have been met then technically the Scrum Team have not accomplished the Definition of Done and the PO reserves the right to reject the User Story(s) as not done.

Edit to Add

The Scrum Alliance have a handy chart for ensuring that the Definition of Done is simple, transparent and consistent. Please note the two boxes stating

• Product Owner Approval
• Acceptance Testing

I know some in the community will respond with anger at my strict siding with the Product Owner but in my opinion it is absurd to think that the Developers are allowed to decide when to take work, what constitutes completion of that work and then force that work into the done column regardless of the Product Owner expectations.

The Product Owner manages the stakeholders, the business requirements and assumes the risk on behalf of the business entity. Of course, they absolutely must own the Definition of Done.

As a balancing act the Scrum Team own the Definition of Ready which allows them to reject User Stories from the Product Owner due to unclear specifications, technical problems, dependency work or any other valid concern.

Ultimately, if we allow Scrum Teams to mark their own homework and push products to a business without Product Owner due diligence the results could be terrible.

In addition, that does not scale to Scrum of Scrums otherwise each Scrum team would simply proclaim their dependency was 'Done' without a Product Owner to consider that they whole product was progressing.

Personal Approaches

As an aside I thought it would be helpful to illustrate how I control this issue as the Scrum Master.

• When a Back End Developer user story has been Tested it goes into "Being Verified" where we discuss it with the PO.
• If the PO disagrees then I ask for the Business Specifications or Requirements that demonstrate that it is not 'done'
• 95% of the time I side with the Developer and reason the Product Owner to either produce a new card for the Product Backlog or accept the current completion.
• Business Intelligence Front End Development in our environment is trickier since much more testing is required including UX and data reconciliation between old and new outputs
• At this point the PO represents the End User and acts a customer
• We give the Product Owner much more capability to reject reports at this stage

Like all things Scrum (and Agile) a balance must be struck to ensure team harmony, trust and a high functioning team. I take the parts of Scrum I need to achieve that and adapt the rest.

Some might say I am not doing Scrum anymore but the truth is, I don't need the label, what I need is a highly productive Business Intelligence Centre.

To the OP, I say this -

Does it suit your needs to have the Product Owner reject User Stories? Be honest. If he/she is truly skilled and good at their role, their rejection of User Stories may be the best thing that happens to your delivery saving the team a higher amount of rework in the long run.

Answer 3

The main source of your conundrum is the Product Owner waiting until end of sprint to accept stories. The function of the Sprint Review is not for the Product Owner to review and accept the work of the Team. It is for the Team to review itself and present what was done to other stakeholders. Time for Product Owner review, feedback and iteration should be built into your sprint schedule.

With that in mind, let's answer your main question "when do features get pull requested into our integration branch of code?"

When to integrate can be a tricky decision. It's common to have a staging environment that serves as the environment both for integration and acceptance testing. But merging a feature can be hard to turn back from. If the Product Owner finds significant issues, now you have a potential blocker to pushing to production, and an untrustworthy branch for other features to branch from.

If possible, the Product Owner should review and approve each feature in isolation before it is merged with other features. Acceptance of the feature at this stage means that it has been realized as envisioned, although more work may be needed to make it shippable. Product Owner approval at this point should be the green light to proceed with completing the work of integrating and deploying the feature.

Here's one way integration and merge conflicts can be handled:

Let's say two features, A and B, are being developed simultaneously. Feature A becomes reviewable first and is accepted by the Product Owner. It can now be merged into the integration branch. All other features, as they approach reviewability, should be paying attention to the progress of the integration branch and updating themselves from the integration branch. This is the point where merge conflicts can arise and be resolved. Now the onus is on Feature B's developer to resolve any conflicts. This may even require modifications to Feature A. Once this is done, Feature B, along with the modifications to Feature A, can now be reviewed by the Product Owner. Once accepted, it too can be merged to the integration branch.

Answer 4

TL;DR

Your Scrum implementation has some misconceptions, which appear to be based around how to implement the "Definition of Done" in a properly-collaborative fashion. If you address this communications failure, your continuous integration efforts should become much smoother.

In addition, it will help your team a great deal to focus on the iterative nature of agile methodologies; agility stresses refinement of the product over time, rather than on perfect execution of "specifications" within each increment. The Product Backlog is a living document, and stories on the backlog are added, changed, or removed at least every iteration. It is not a one-pass set of requirements, and your team needs to leverage that flexibility.

When is Something Done?

In Scrum, a user story or increment is considered done when it has achieved its stated goal and met any existing criteria to which the Scrum Team as a whole (which includes the Product Owner) has previously agreed.

Consider the following user story:

As a web-browsing customer,
I want to see everything on the page in 300-point type
so that I don't suffer from eye strain.

This story would be done when:

1. The web site's default font had been set to a ridiculous size.
2. The team agrees that the implementation meets the goal of avoiding eye strain.

3. The current "Definition of Done," which applies to all stories in the project, has been met. This definition varies from team to team, but might include unit tests, user acceptance tests, pushing to a continuous integration server, or slapping each other with wet mackerel. Whatever.

   NB: The point here is that the team (including the Product Owner) agrees to the definition beforehand. There is no post-facto retconning of "Done."

4. Someone on the team marks the story "done" on the Sprint Backlog.

That's it. The story is now done. It may or may not be done right, but it's considered complete for the purposes of Scrum. The Product Owner should have been involved throughout the Sprint, so if the feature has completely slid off the rails, the PO's lack of involvement is likely the problem and should be addressed during the next Sprint Retrospective.

What About Stories That Aren't Done Right?

At the end of the Sprint, the team collects points for all stories that the whole team (including the Product Owner) agree were completed according to the Definition of Done. If the stories were thus completed, but are unsatisfactory in some way, that's grist for the mill during the Sprint Retrospective and for other inspect-and-adapt meetings. It doesn't change the fact that the stories were done in an agreed-upon way—an agreement to which the Product Owner was an active party—and so there's no "acceptance" of the stories to be done.

However, during the Sprint Review (my exposition in bold):

• The Product Owner explains [to stakeholders] what Product Backlog items have been “Done” and what has not been “Done”. This is a status report, not an opportunity to accept or decline stories.
• The Development Team demonstrates the work that it has “Done” and answers questions about the Increment. This is often a great place to explain misfeatures that are nevertheless "Done," and to solicit stakeholder feedback about how to improve them in a future iteration.
• The Product Owner discusses the Product Backlog as it stands. He or she projects likely completion dates based on progress to date (if needed). This is another inflection point where stakeholders can decide if an imperfect feature is "good enough," or if it needs to be reworked or added back to the Product Backlog as-is.
• The entire group collaborates on what to do next, so that the Sprint Review provides valuable input to subsequent Sprint Planning. This is where some limited discussion of misfeatures could take place, but most process problems should be addressed in the Sprint Retrospective.

Then, during Backlog Refinement or Sprint Planning, the Product Owner can take any stories that were completed but didn't deliver the desired value and:

• Write new stories to address any shortcomings.
• Put stories back onto the Product Backlog, hopefully with more detail or a better understanding with the team of the story's goals and objectives.
• Decide that the features, as delivered, are "good enough" or don't need to be iterated over again.

Conclusion

To sum up, the Product Owner doesn't get to single-handedly declare whether or not a story is done. After all, the "definition of done" is a collaboration between all the members of the Scrum Team.

However, the Product Owner (as the sole custodian of the Product Backlog) does get to determine if a story can be marked completed on the Product Backlog or if elements of it need to be done again in another Sprint. Simply throwing the original story back onto the top of the Product Backlog without addressing the underlying problems with the story, the Scrum process, or the Scrum Team's communication is probably a bad idea though, so the Scrum Master should definitely assist at that point.

Answer 5

This is a good question, which many teams struggle with.

The Product Owner makes his decision on whether a User Story is "done" at the end of sprint, during the Sprint Review Meeting.

Nope. The Product Owner accepts "done" User Stories during the Sprint.

So the whole Story lifecycle (To do - In progress - Done - Pushed to master) happens before the Sprint Review. And once Story is pushed to master branch, other branches should pull it in and fix conflicts if any.

User Story acceptance may take couple of days, so the Sprint Review meeting is not held to accept User Stories. It's held to compare planned vs actual and assess if the Sprint goal was met.