It's common for business to have some level of distrust towards IT, especially if they were disappointed by IT before. If you gain their trust the situation will most likely change and they'll get off your back. For that you need to:
- Understand them and their pain points, speak their language, feel for them. If they recognize that you're one of them they'll trust you more. For that you need to study the domain, learn their workflows.
- Deliver and excel their expectations. Good UX skills together with strong dev team are necessary for this.
At first you'll have to play by their rules and provide estimates. But once you start to deliver and gain their respect things will change.
And when providing estimates don't forget to mention that after initial implementation you'll be ready to hear out their feedback. And if they change their minds about something or will have suggestions - you'll be happy to address them. They'll get their estimate and you'll get your agility.
This all depends on the organization of course. Some companies (especially large ones) may have strict policies and even if business wants to be agile they don't necessarily have means and authority to change the rules.