There is an alternative method to track velocity that is especially useful for new teams or teams still trying to master the estimation exercise.
And that is to simply count how many stories that team can complete in a standard iteration.
Not points but whole stories.
In theory the story size should be taken into account when measuring velocity but in ...
The purpose of velocity, and story points, is to aid the development team in answering one question:
Given our current understanding and knowledge, what is the scope of work that we can confidently deliver during this Sprint?
Scrum, in particular, is meant to be a framework for developing a product in a complex environment, meaning one where the work ...
You are quite right that estimates are hard to get right and that the real amount of work will include an error component.
However, there are a couple of factors that work to reduce the impact of these errors:
The team is estimating on lots of backlog items so errors should have a tendency to cancel out (e.g. high estimates will cancel out low estimates).
Velocity measures how much work the team can complete per sprint, given a stable team and a common understanding of the complexity of the items that they work on.
It's usually measured in number of story points, or sometimes, if PBIs can be split into similar sizes, you can even count number of PBIs completed per sprint and use that as your velocity.