Specifically, if we assume that the team spend a day on Sprint Planning using the 4-8 hours of planning for each week of our sprint formula then that doesn't really seem like enough time for the team to flesh out implementation details. And should the entire delivery team really be involved in that discussion? At a sprint planning meeting we will have the Product Owner, Scrum Master, Developers and Testers (Maybe even a Business Analyst or 2). Do we really want to go deep on Dev with all of these other team members burning hours? Or should this be discussed in a separate developers only meeting? It just doesn't appear that Scrum provides for this? Am I missing something? How is your team dealing with this?
TL;DR: Scrum doesn't provide for a lot of things, you need to figure out a way that works for your team and project.
During the planning meeting (from the Scrum guide)
The Development Team usually starts by designing the system and the work needed to convert the Product Backlog into a working product Increment.
What designing the system means could be open for discussion. Personally I think the planning meeting should result in an achievable Sprint backlog. This means that each story should be clear enough to be estimated and the team pickups only so much stories that they can complete in the cycle. To let the team estimate they should have a short discussion about how to technically implement it, but not 100% of the details have to be clear at that moment, just enough to estimate it. We try to minimise waste for non-technical roles, but still discuss enough so they have an idea how to team will try to tackle the task. The goal is knowledge sharing and shared focus, not a 100% detailed Sprint plan.
So to answer your first question: Yes, everyone should be involved in the discussion. Everyone who will do a part of the work should estimate the whole story.
More details during the Sprint:
During the Sprint our teams starts each story with an "architecture and design" task, this is a session which results in a design and technical sub-tasks. Including design, coding, but also testing tasks. This is the moment the actual detailed design becomes clear based on the higher-level acceptance criteria noted down in the Sprint planning. After this session we swarm to complete the user story as quick as possible. The sessions length depend on complexity of the story at hand. I would say this is a dev-team group session, but sometimes if the complexity is low a single person could prepare everything with a short introduction to the rest of the team.
Now it is possible that during the Sprint in such a design session the team notices that they under-estimated. Challenge the team to build only what is really necessary, apply the YAGNI principle and prevent over-engineering. If this happens often, inspect and adapt, maybe you need to take more time during the Sprint planning meeting. Discuss this in a retrospective and let the team decide how to experiment with improving this.
One of the major Dutch web-shops with around 20 Scrum teams (in 2013) did experiments with the length of Sprint planning meetings to get and idea what level of detail is good enough. They told me that doing Sprint planning meetings in 15 minutes (yes just 15 minutes) lead to relative the same average estimations as 4 hours would. Now I think this will lead to a bit of more discussion during the Sprint, but it highlights that details are not the most important thing of a Sprint planning.
The reason Scrum does not detail everything for you is that there is not a single truth for process in Software development. It is different per product, team, company or industry. What the Scrum framework does is to help you inspect and adapt to find your fit the quickest.
The first half of Sprint Planning consists of selection of items from the Product Backlog that will fit within the current Sprint. Ideally, the backlog items meet INVEST criteria and are suitable for selection without a great deal of fine-grained planning during this portion of the meeting.
The second half of Sprint Planning is decomposition of Product Backlog Items into tasks for the Sprint Backlog. This is where the team is expected to do "just enough" planning to build a roadmap for the current Sprint.
Let's work an example. For instance, let's imagine a story that says:
As a web site user
I want to be able to maintain a user profile between visits
so that I don't have to re-enter preferences each time I log in.
When this story is selected from the Product Backlog, the team then needs to identify the dependencies and agree on a general implementation that can be broken down into tasks. These tasks should typically be no more than ½ to two days in length to make it easier to coordinate tasks and track them as done or not-done rather than as percent-complete.
In this example, the team should probably agree on a few things, like:
- Where will the user profile be stored?
- What preferences will be stored in the profile?
- How will the team test that the functionality meets the Definition of Done?
The team does not have to over-constrain the technical implementation during Sprint Planning. Instead, Sprint Backlog items are created, and can be discussed and refined further by the development team during the course of the Sprint itself. The goal is to do just enough planning to set a direction for the team, so that the work increment can begin in a coordinated and collaborative fashion.
Iterative development is an emergent-design process. The goal is not to produce detailed up-front specifications, but rather to generate consensus on a "good enough" approach that the team can then collaborate on throughout the iteration. To that end, Sprint Planning (and the Sprint Backlog in particular) are about defining what is to be worked on, not how it will be implemented.
By definition, any working increment that:
- supports the Sprint Goal,
- meets the what and why of the Product Backlog Item, and
- fulfills the Definition of Done
is sufficient for a Sprint. Over-specification, over-engineering, and over-planning are agile process smells. Obviously some planning is needed, but the goal is always to do just enough planning to deliver a working increment.
Support Just Enough Planning
I like lists. Maybe you do, too. Some ways to support this type of planning include:
- Ensure the Product Backlog is well-maintained, and that the top of the backlog is filled with items ready for selection into a Sprint.
- Ensure the entire team participates in Backlog Refinement so that the what and why of the items are at least implicit in each selectable backlog item.
- Ensure the team only selects actionable stories that fit within a single Sprint during Sprint Planning. Decompose stories or swap out items that won't fit during the Sprint Planning process.
- Ensure that the team creates a Sprint Backlog with tasks during Sprint Planning. Some very mature teams don't always do this when the Product Backlog items are small and well-defined enough, but in general skipping this step is where teams miss out on just-in-time planning.
- Focus on what and why in Sprint Planning. Leave specific implementation details (the how) up to the team members doing the work during the Sprint.
- Leave plenty of slack in your Sprint to allow for collaboration, exploration (e.g. micro story spikes), and small meetings focused on "just enough" planning to coordinate the current day's work increment.
If you find yourself in need of complex, detailed planning exercises, chances are your Product Backlog or Sprint Backlog items aren't decomposed well enough, or don't meet the INVEST criteria. It may also mean that you're over-constraining the team by limiting their opportunities to collectively collaborate on implementations. Back up, fix the process issues, and then turn the team loose to implement the functionality in the simplest way that can possibly work.