TL;DR
Whatever your team is doing, it isn't Scrum. None of the roles, events, or theory defined by the framework are being followed properly, and even basic agile values like customer collaboration are being ignored. Scrum has no answer to how to fix a non-Scrum process, but you can certainly fix most of the problems you're seeing by actually following the framework as it was designed to operate.
The Scrum Implementation Was Broken from the Start
The Product Owner created a Product Backlog. The Scrum Master created the tasks from the Product Backlog and is leading the Development Team. Everything started okay, but now the Scrum Master is reporting plenty of errors on the formulas that the client sent at the beginning of the project.
None of what you describe here is really Scrum. While the Product Owner is responsible for building and maintaining the Product Backlog, doing so through large, complex specification documents rather than through incremental and iterative collaboration with the customer is a bad way to start. Doing so without any input from those with subject-matter expertise is even worse.
In addition, a Product Backlog should define what is to be done, but only the people doing the actual work (e.g. the Developers, not the Product Owner or Scrum Master) should be estimating or planning the work. A "Scrum Master" who is turning Product Backlog items into tasks and dictating how the Developers should build the product is not fulfilling the role or empowering the team. How to deliver each increment is the responsibility of the Developers, not the Product Owner or Scrum Master.
Furthermore, you don't mention Sprint Planning or test-driven development at all. Apparently the Product Owner believes this is easy, uncomplicated work. In that case, the Product Owner is free to wave a magic wand and deliver the work with an abracadabera or two. Otherwise, the Scrum Team needs to plan small, testable increments of work that are within their capabilities rather than trying to build large and untested chunks of complex code that will magically be error-free.
Agile Values and Principles
Forget about Scrum for the moment. The team isn't even operating with agility. A core agile value is:
Customer collaboration over contract negotiation.
and a key underlying agile principle is:
Business people and developers must work together daily throughout the project.
The current process has replaced customer collaboration and whole-team collective ownership, as well as ongoing inspect-and-adapt reviews of the product and process, with a set of complex specification documents and top-down task assignments. This is neither agile nor effective, especially if the product is complex or outside the scope of what the team has experience delivering.
Next Steps
First, before reaching out to the customer, the whole team including the Product Owner, Scrum Master, and Developers need to sit down together and collectively agree that:
- The process is deeply broken.
- Fault is irrelevant. What matters now is how to get the project back on track.
- The whole team needs to work together to build a better process.
- Test-first development or some form of quality testing needs to be made part of the Definition of Done for each increment.
- The project most likely needs to be reviewed and re-estimated based on what the team has learned so far.
- The customer should be actively engaged throughout the process.
- The customer should be aligned with an expectation that the product can be refined or improved iteratively rather than delivered in monolithic chunks that aren't small and testable.
- Someone at the company with sufficient authority needs to renegotiate timelines, budget, or scope with the customer based on the current reality, not wishful thinking.
The bottom line here is that the product can't be effectively delivered at the pace or quality promised within your current process. Whoever owns the customer relationship needs to level with the customer about that, and reset everyone's expectations accordingly.
Meanwhile, the team needs to work together to build a more reliable and more realistic plan for the work. If time permits, bring in a more experienced Scrum Master or agile coach to help the team. Otherwise, just commit to doing better than you have, with the understanding that fixing the process will take time, money, and resources whether you do it now or later. If you have an understanding customer, you might be able to restructure the project and the team's work now; otherwise, pull together to do the best you can, but ensure that your leadership team understands the long-term costs of ignoring the underlying problems in favor of short-term delivery.
You must to fix your current process either way. Half-measures might get your current project across the finish line eventually—ideally without creating a death march for the Scrum Team—but the process issues you're experiencing now will likely bite you again on the next project. Deciding how best to handle the current problem is mostly a business decision for which senior leadership is ultimately responsible, but fixing a poorly implemented sales-and-delivery process is an organizational problem that requires everyone in the company to contribute to a realistic and sustainable solution, regardless of the project management framework you're using.