This seems to be a common question theme lately. I recently answered similar questions in this thread and in this thread.
There are three main issues swirling around you.
Estimations are just guesses: In the 80' and 90's we somehow forgot the concept of "[Cone of Uncertainty][3]" in estimating. When you first estimate a project, you are lucky if you are even 50% accurate. And if you are not, then it will always be later. You will almost never over estimate the work to be done thanks to Hofstadter's Law, which basically says "We always under estimate, even when we know we always do."
The only true way to know when (scope driven) or what (schedule driven) you will deliver for a project, is to start doing it and then use velocity to predict either when or what.
Scope vs. Schedule: I recommend my, short, slideshare presentation on this. I have full speaker notes embedded to explain each slide. It's a great way to visually explain Scope vs. Schedule so customers and teams understand.
My personal recommendation, based on years of Agile development, is to always go with a Schedule Driven release with an MVP that is no more than 80% of what you think your Release Line is.
Release Lines vs. MVP Lines: We're all familiar with the term of Minimum Viable (or valuable) Product. However I see a lot of variation in what people think the definition is. I usually go with this "If we don't reach MVP, then we would throw away all the code and not release the product at all" and "MVP is what does the customer need for the product to be useful?".
I usually cite the original iPhone. It didn't do BlueTooth, it didn't cut and paste, it had no real apps to speak of. And yet it exactly met the customer need (A dirt simple phone, with idiot proof UI that would allow phone calls, emails and basic web surfing).
Release Line is a term we don't hear enough of and is what causes us to run into so many problems with MVP defined projects. We'll say "these are the MVP features" of the project and "We must ship by August", however we rarely then look at what we think engineering can actually do. In the Schedule section of my presentation I show the worst case scenario of the team delivering less than the MVP because the product owner defined the MVP as exactly what engineering thought they could deliver.
Never do that. Always under commit and over deliver. If you think you can ship 10 features (engineering estimate), in the schedule, don't commit to more than 6. This allows for two very real things. 1- You under estimated the amount of work to be done. 2- Your customer/ product owner changes their mind and adds new features during development.
Finally, patience and Agile Value 3: We are trying to turn around thirty years of bad project planning and expectations that an estimate is a commitment. We won't change it over night.
So always keep in mind Agile Value 3- Customer Collaboration over Contract Negotiation. Keep in constantly in communication with your customer and be totally truthful and transparent. If you think there is even a chance of delay, tell them right away. In the last seven years I've used this model. My customers (internal and external) love the total transparency and are a lot more tolerant of issues because I'm communicating as often as daily. When you can see the sausage being made, you are less concerned about eating the final product.
