Refactoring in Agile development

Question

In an app development project, the full scope of the project was not considered in the app design. So, the frequent refactoring of codes to accommodate changes is negatively impacting the project delivery timeline, which has become necessary during a sprint. How best can this be controlled to minimize the risk of overshooting the agreed date of project delivery, since stakeholders are not willing to compromise?

Answer 1

TL;DR

Part of any project management framework, but especially agile frameworks like Scrum, is the necessity of continuously managing stakeholder expectations. People want what they want when they want it, but a big part of project management is explaining to people what they can actually have within the various constraints impacting the project. As the project evolves, so should the understanding of what is currently feasible within then-current constraints, especially when what's feasible diverges from what was originally planned.

Rework is Expected

Refactoring changes implementation without changing external behavior. In all likelihood, you're not really refactoring; you're paying down technical debt, or performing rework and integration, which are not intrinsically the same things.

Frameworks like Scrum treat various types of work (including refactoring and rework) as an acceptable trade-off for empirical control and design. This is handled in the estimation process, and (correctly) shows up as a drag on velocity as tech debt, complexity, or the pace of change increases. This is both expected and desirable in an iterative framework, because each inspect-and-adapt loop provides an opportunity to change scope, prioritization, or approach.

Rework is the cost of empirical control. You can't really avoid it in complex projects such as software development. Scrum just makes the rework visible to the team and to stakeholders as the cost of making necessary or desirable changes.

Fixed-Everything Isn't Possible

The "iron triangle" posits that you can adjust a project by controlling for budget, scope, or schedule, with quality an implicit fourth dimension impacted by the other three sliders. The following graphic depicts this.

Agile frameworks like Scrum generally treat scope, and especially project scope, as the main slider by asking:

How much scope can we fit into the time box of a single Sprint, or fit within our agile release plan?

Currently, your project is attempting to fix both scope and schedule. That only leaves budget as your available slider. If you attempt to lock that down too, then either your project will fail, or quality will suffer, or both. You simply can't reliably do fixed price, fixed scope, and fixed schedule in any project framework while maintaining quality; Scrum simply makes it more obvious, and makes the trade-offs more visible.

The point of Scrum isn't to magically make the iron triangle disappear. Instead, it forces the project (as well as the stakeholders and sponsors) to comes to terms with the inherent trade-offs. Scrum enables stakeholders to determine whether something is "good enough" to ship at the end of each Sprint, and the Product Owner gets the opportunity to adjust scope and prioritization during Backlog Refinement and Sprint Planning. The stakeholders need to leverage these events as a way to ensure they get the most value feasible within the release plan; it is never a money-back guarantee that the project will be either 100% complete or fit for purpose at the end of a fixed series of Sprints.

Decrease Sunk Costs

Another aspect of agile development is the opportunity to "fail early." When a project is likely to fail for whatever reason, agile frameworks like Scrum give the Product Owner the opportunity to save money by cancelling a Sprint and planning a new one based on changed expectations, or allowing stakeholders to cancel the remainder of the project if it will not serve their purposes. The latter will not make the project succeed, but it will reduce the sunk costs associated with the project.

Nothing anyone does can guarantee that a project will succeed. The best you can do is control for it, and kill off a doomed project as early as possible when success is no longer an option.

What You Can Do Now

First, acknowledge that you are on the path to a death march. If you won't face that truth, nothing else you do will matter.

Next, figure out whether the problem is design, tech debt, unreasonable expectations, or something else. It doesn't matter what the root cause or causes are; you just have to make them visible so the team and the stakeholders can collaboratively address them.

Finally, communicate the current status clearly. If you are on pace for a full-featured release a few cycles past the originally-projected delivery date, discuss whether the project can adjust the "time" vertex. If you can't or won't adjust the schedule, then you can still meet the fixed deadline by increasing costs or reducing scope. Would the stakeholders trade on-time delivery for cutting some time-consuming bit of functionality? You won't know unless you ask.

If the project can't or won't do any of the things above, then prepare for failure. Brush up your resume, and hope that you've learned valuable lessons that you can use on your next project or next job. Most of all, hope that you've learned to communicate early and often about assumptions, challenges, framework requirements and trade-offs, and internalized the importance of continuously-managing stakeholder expectations.

Answer 2

Are you producing a deployable increment of product each sprint? If so then agreed delivery dates are already being met and that is the best way to minimise risk and give the customer confidence in what you are doing. The stakeholders (via the product owner) should get to decide which things they want and when, so what's most important is that they continue to see you delivering value and that you get their feedback about improvements, changes and fixes needed to the product. Getting their feedback is key here. Customers might understandably be concerned if the timeline extends with no new functional scope but as soon as you start including the latest improvements that they see as priority they will be more likely to appreciate the need for flexibility.