Within the Scrum framework, work selected for the Sprint during Spring Planning must meet a number of criteria. To be a bit reductionist, when combined with other sections of the 2020 Scrum Guide such as the definition of the Product Backlog it is clear that work selected for the Sprint must be completable within a single iteration.
To do that, you'll need to avoid skipping over essential framework activities such as Backlog Refinement and setting Product and Sprint Goals. It is extremely common for new Scrum implementations to overlook the importance of these activities, and that is most likely where your difficulties start.
Analysis and Recommendations
Understanding Sprint Planning and Sprint Goals
In Scrum, Sprint Planning is an activity where the Developers select Product Backlog items (PBIs) that have been refined by the Scrum Team sufficiently to fit within a single Sprint.
Sprints enable predictability by ensuring inspection and adaptation of progress toward a Product Goal...When a Sprint’s horizon is too long the Sprint Goal may become invalid, complexity may rise, and risk may increase.
There are certainly other criteria at work, but in general, the flow is:
Product Goal defined on the Product Backlog
The Product Goal describes a future state of the product which can serve as a target for the Scrum Team to plan against...The Product Goal is the long-term objective for the Scrum Team. They must fulfill (or abandon) one objective before taking on the next.
A singular Sprint Goal is defined by the Scrum Team in order to make progress towards the current Product Goal through the delivery of one or more Increments. The guide says (emphasis mine):
The Sprint Goal is the single objective for the Sprint...[that] provides flexibility in terms of the exact work needed to achieve it. The Sprint Goal also creates coherence and focus, encouraging the Scrum Team to work together rather than on separate initiatives.
Note that "[a]n Increment is a concrete stepping stone toward the Product Goal" so you can have more than one Increment per Sprint, but only one Sprint Goal. This is the most common thing people get wrong: not having a well-defined Sprint Goal that can reasonably be delivered by the end of the Sprint.
Developers perform PBI selection from the Product Backlog, using the Product Goal, Sprint Goal, and team capacity as guides. Team capacity is the primary use of velocity, but "velocity" is not a required metric for the Scrum framework.
Product Backlog items that can be Done by the Scrum Team within one Sprint are deemed ready for selection in a Sprint Planning event.
Through discussion with the Product Owner, the Developers select items from the Product Backlog to include in the current Sprint. The Scrum Team may refine these items during this process, which increases understanding and confidence.
Selected work is further refined by the Developers for the Sprint Backlog. While not mandated by Scrum, the work is often decomposed into user stories or vertical slices using the INVEST mnemonic.
The Sprint Backlog is composed of the Sprint Goal (why), the set of Product Backlog items selected for the Sprint (what), as well as an actionable plan for delivering the Increment (how)...[that provides] a highly visible, real-time picture of the work that the Developers plan to accomplish during the Sprint in order to achieve the Sprint Goal.
All PBIs not completed by the end of the Sprint are returned to the Product Backlog for future consideration. This last item has been hotly contested by people over the years, but the 2020 Scrum Guide (with emphasis mine) makes it very clear:
Work cannot be considered part of an Increment unless it meets the Definition of Done.
The Definition of Done is a formal description of the state of the Increment when it meets the quality measures required for the product.
If a Product Backlog item does not meet the Definition of Done...it returns to the Product Backlog for future consideration.
This is necessary because agile frameworks embrace changing requirements, so incomplete work may or may not still have value. Even if it potentially has value, a given item may not remain aligned with the Product Goal or Sprint Goal for the next Sprint. Even when in scope, the PBI may need additional refinement and re-estimation based on the current state of the project and its objectives rather than on historical estimates. In short, the priority, objective, and level-of-effort for the incomplete PBI must be re-evaluated rather than simply "rolled over" to the next Sprint.
Confidence Intervals, Cadence, and Sizing
An effective Sprint delivers Increments of value. Ideally each Increment is a vertical slice of functionality, but can be anything the team determines can move the ball downfield towards the Product Goal. The objective of a Sprint is not to complete all the items on the Sprint Backlog, but to meet a unifying Sprint Goal.
Generally, the success of a Sprint has nothing to do with how many backlog items you perform. Instead, it's based on whether or not the Scrum Team collectively met its commitment to the Sprint Goal. This won't always happen, but if you're missing your Sprint Goals more than 20% of the time then you haven't developed a reliable delivery cadence.
Velocity with appropriate fudge factors can be a useful metric for determining whether the Scrum Team has enough capacity to meet the work needed for the current Sprint Goal. However, velocity is just a probability range based on various factors, and is not a money-back guarantee.
There are other estimation techniques that can be more useful when evaluating the overall size of the Sprint Backlog. Two of my favorites are:
- TFB: "too freakin' big" to fit into one iteration; such work should be refined further using INVEST or similar techniques
- 1: likely to fit within a single iteration
- NFC: "no freakin' clue" means the work isn't sufficiently decomposed, doesn't meet the team's Definition of Ready, or is otherwise inestimable and needs to be clarified or refined further
Regardless of how the work was selected or sized, the Developers or the whole Scrum Team hold a "confidence vote" to determine how confident they are that the plan will successfully meet the Sprint Goal. This is usually done by polling the team, and on the count each team member holds up 0-5 fingers. The average is usually taken as the team's collective level of confidence in the Sprint Plan, which should be discussed or revised if the confidence level is too low.
Utilization rates or volume are not the objectives; instead, you are aiming for a reasonably reliable, sustainable delivery cadence for the team. If your Sprint Plan doesn't have a clear Product Goal or Sprint Goal, or if the Sprint Plan has insufficient slack to allow for normal variances, then you will continue to struggle with creating a sense of predictability for both the Scrum Team and other stakeholders.
The Product Backlog should be routinely refined to ensure that PBIs are ready for selection. This is another area where your team is likely struggling.
Product Backlog items that can be Done by the Scrum Team within one Sprint are deemed ready for selection in a Sprint Planning event. They usually acquire this degree of transparency after refining activities. Product Backlog refinement is the act of breaking down and further defining Product Backlog items into smaller more precise items.
If your Scrum Team is entering Sprint Planning without sufficiently-refined PBIs that can fit within a single Sprint then you need to spend more time on Backlog Refinement in order to meet the "small" and "estimable" elements of INVEST. Even if you don't strictly follow INVEST, you must still ensure that the team has sufficient PBIs that are ready for selection, align with the current Product Goal, and form a coherent Sprint Goal. This can be hard at first, but gets easier with practice and when the team leverages the Sprint Retrospective to identify and improve on the processes that are leading to incomplete Sprint Goals.
Scrum is based on empiricism and validated learning, so there's definitely a learning curve involved in building the team's estimation and planning muscles. However, the framework provides a lot of inspect-and-adapt opportunities, and encourages continuous improvement. Just make sure you start with the essentials like refinement, Definitions of Ready & Done, and clear Product & Sprint Goals. Without those in place, you won't be gaining the benefits of the framework; you'll just be following your pre-Scrum processes and then wondering why things aren't fitting within the time boxes or incremental delivery model that Scrum requires for success.