I cannot speak to your particular situation but yes, it is pretty common that not everything is done.
This is in fact a logical consequence of the structure of the team, and you being a developer are in a prime position to appreciate it.
Load and delay
If I were to ask you “do you run a server at 100% load?” you would likely laugh in my face. “What sort of load is acceptable?” might garner a set of mixed responses; including simply “how do you define load?”, but I think we could settle on the idea that more or less no matter what the definitions are you would not like server loads above, say, 10-25% of capacity: certainly not much above 50%. And were I to ask you “why not?” you would very likely first give the Wrong Answer (that you’re worried about hardware failure—it’s a fair point but usually if you’re doing it right you’re already guarding against that risk) and then second give the Right Answer: that you’re worried about latency spikes.
The core problem is that a sprint model typically assumes one way or another that there is 100% developer load, and developers respond by padding out their estimates with safety buffer so that they can hopefully only operate under 50% load or so; then you have natural problems dealing with the same two problems: hardware failure (employee burnout) and latency spikes (hey, Legal said one week ago that Floozels must only be awarded for the Flotsam that is also Jetsam—why isn’t this fixed now?!). In practice nobody wants to accept the latency spikes and so they simply throw story points of developer time out of the current sprint in order to keep latencies low with their bosses.
In fact the problem is somewhat larger than this because safety factor in estimates is much much bigger than even the typical advice to “always multiply estimates by N” suggests. The reason is that the private estimate for completion completes the goal with, say, a 90-95% probability, which for realistic situations might already add a factor of 2 of safety versus the 50/50 chance of completion that we should idealistically be able to target; furthermore a PM then buffers those estimates with some additional padding anywhere from 20-100% when communicating with the business as a whole. Take any software project, and something like three-quarters or more is safety buffer. We devs should be completely scandalized at the idea that any project would ever run over-time or over-budget if we understood how much safety buffer we have to spare to complete it. The typical excuses you hear, “oh we will be late because in our last week a hurricane took out our data center causing all of our systems to be down for several days, but the team worked together and pushed really hard and we got everything working and we’re only going to be late by three days” are laughable simply because given the safety buffer they should have delivered before that hurricane hit.
Waste and purpose
The solution to the paradox is waste; we must be wasting all of that precious safety buffer. This happens in many different forms and enumerating all of them would make this answer very long. However I think we need to point the finger squarely at management simply because the developers lied in the first place; they padded their estimates first from 50% to 95% and then they added another factor-of-two on top of that, and this means that management has not created a space where they feel safe in being totally honest with their estimates.
I can also give a quick sort of aspirational vision for how to solve it. In theory there is a dependency graph of tasks-to-be-done for your Sprint Goal or any other release you have, and it typically involves some real stinkers that nobody wants to take on, like “figure out why the QA server is crashing even though we have implemented workarounds to make it not crash so that QA is unblocked, we have no idea what’s causing it, good luck, don’t break anything.”
If you think about commits on this sprint to your develop
or master
branch, you see that even though the dependencies are in many cases not enough to structure the thing completely, the tasks will ultimately form a directed acyclic graph and there is some critical path from sprint-start to sprint completion which takes the longest amount of time, while there are several “side branches”—other devs working on important things which are not on this critical path—which does not.
In the ideal project there is at any given time exactly one developer who is in a state of total and complete panic. They are working on the critical path, and they know it. The critical path is something of a baton or hot potato that they are looking to pass off to the next person as soon as they can; and they are given considerable power to loop in other developers to help them with their task, or to compromise scope of their feature to be good enough to pass the baton on. Facilitators have several roles:
- They prevent burnout by ensuring that the baton keeps being passed from person to person (as others are given a great deal more latitude to rest).
- They make sure that others do not waste enough of the safety buffer on these side-paths that it would lead to a change in the critical path where someone (let’s call her Alice) on the critical path says “I can’t do this, I am waiting on Bob’s latest changes.” Or when the critical path has shifted definitively this way, a facilitator tells Bob, “hey, you don’t need to be in a state of panic anymore, by the time you pass this off to Carol she will still be waiting on Alice, we’re giving Alice the hot-potato.”
- They lubricate these jumps. When Bob finished and Carol starts, the facilitator is ultimately asking Carol to completely drop everything that she is doing: this will be disastrous if she does not have advance warning. So they are asking the panicking Bob for progress updates, but they do not care if he is at his deadline or not. Remember, that deadline was only estimated at being 50% chance of completion; it is therefore rightly meaningless for holding anyone accountable and there is no shame in missing it. So Bob needs to be approached with gentleness and good faith and asked how soon he thinks he can be done with his task, purely so that the facilitator can tell Carol “hey in around two days or so I am going to ask you to drop everything else and work on this one specific task, you will have the baton, please clear your schedule for that and start getting ready to sprint with it.”
This is not a universal structure of all business; it is a derivation of some general principles for a specific context where everything being produced is fundamentally different and only is produced by some big complicated cooperative effort. Your software teams may be working in a different business sector which does not work this way: for example Valve no longer makes big software games to be bought for a big price tag but instead makes skins and user customizations sold on Steam to get a lot of micro-sales; one can imagine that the Valve Before would have strongly benefited from this sort of structure whereas the Valve After probably would not see those sorts of returns. But those sorts of patterns are pretty common.