Per the “planning fallacy” Wikipedia entry:
The planning fallacy...is a phenomenon in which predictions about how much time will be needed to complete a future task display an optimism bias and underestimate the time needed.
Different agile frameworks use different metrics, but the commonly-used velocity and cumulatative flow metrics are the typical guardrails for mitigating planning bias. There are certainly other metrics, but I'll talk briefly about velocity because I think it most closely addresses the concern.
How Velocity Mitigates Planning Bias
Velocity is a metric that looks at historical rates of delivery, and uses a moving average or other statistical functions to smooth perturbations. The velocity metric is probabilistic, and while it often works best with a baseline reference class to anchor the metric, in pragmatic usage it's actually less sensitive to that than one might think. The real determinant of how reliable velocity is as a predictive metric is the consistency of the estimation process.
The key to velocity's widely-accepted success in agile planning is a consistent estimation process. It actually matters very little whether the estimation process is optimistic, pessimistic, or ignores the reference class altogether, so long as the estimation process is applied similarly across iterations. Over a sufficient time window, a consistent estimation process will trend towards the sustainable rate of delivery.
While a consistent estimation process is essential to reliable forecasting, that doesn't mean your project's estimation process can't change or improve. The notable caveat here is that iterative or incremental improvements to the estimation process will create small perturbations in delivery rates, but will generally be small enough to be swallowed by the smoothing function.
As a related caveat, it's also possible to make radical changes (hopefully improvements!) to the estimation process. When that happens, you either need to discard historical data or apply a "fudge factor" to account for the changes in the methodology. This may lead to less forecasting reliability from the metric in the short term, but can often be worthwhile if it leads to higher sustained reliability in meeting forecasts.