Agile project management prevents coding for the future. Our client asks some reports which can be coded withing the schema of existing database structure, but the best practices strongly suggest that the client really aims for some kind of data warehousing solution. Neither our company nor our client is ready for upfront investments and additional work that will bring rewards in future only. What to do? 1) Be satisfied with the present status quo and go with suboptimal design of reports with low maintainability? 2) strongly press for warehousing solution. Who makes such decisions? Developers see the problem, client and management does not see the problem and don't understand the concerns of developers. What should be done in this situation, what behaviour are expected by the best practices of project management methodologies?
Over-Engineering is an Agile Anti-Pattern
Our client asks some reports which can be coded withing the schema of existing database structure, but the best practices strongly suggest that the client really aims for some kind of data warehousing solution. Neither our company nor our client is ready for upfront investments and additional work that will bring rewards in future only.
There are several agile principles you need to follow:
- CodeGnome's Law of Transparency: "No invisible work, ever!"
- YAGNI: "You Ain't Gonna Need It"
- Visualize the work.
- Surface technical debt.
Basically, agile frameworks are not a silver bullet that make work magically easier or more efficient. When done right, agile principles make business choices and their associated costs visible to the organization.
By making the process that the team is following to generate reports transparent, and by making the costs of that work visible, you empower clients and management to make informed choices. That does not mean that implementing a forward-looking solution is the right thing, either. In some cases, the business cost of a sub-optimal technical solution may be the better choice if:
- The labor costs for the sub-optimal solution are lower than the replacement costs of a newer system.
- The current solution is "good enough" for the expected lifetime of the product or solution in the marketplace.
- Business priorities or market conditions make the current solution a better strategic choice than investing in a different long-term solution.
- Anything else that causes the senior executives, when fully informed, to decide that limited resources or budgets should be expended elsewhere.
Your job as a project manager, Scrum Master, Product Owner, or agile practitioner is not to make business decisions for the organization. Your job is to provide viable options for incremental/iterative delivery within the Definition of Done, and to make all work and all project costs visible.
The rest is up to senior management. If they break the process, they get to keep both halves.
If you see the problem, but someone else does not, the only thing you can do is keep trying to make them understand the problem. This isn't just related to Agile; in general it's your job to make other (non-tech) people understand the consequences of their choices as best as you can.
Keep making the reports, keep being honest about the time required to make them, the maintenance costs that are being paid (and I mean actually paid, not potentially down the line) and keep trying to explain that if the client expects to be doing more of this in the future, it might help to move over to to better architecture.
Remember that Agile is a big fan of being transparent towards your client and collaborating with them. So don't try to secretly build things they're not asking for, and don't hide the truth from them. If you are right, then one of two things will happen:
A) they will learn of and accept your concerns after a while and think about the future and the investments they need to make B) they ignore you and they will suffer the pain of doing so
These are the exact same outcomes that would happen with any other methodology, by the way.
"Because we're agile, we want to do a large up-front investment that won't bring immediate results."
Umm... does not compute.
The point of agile is to get something working in front of the client asap, in order to get feedback so you can change that previous something.
Clients don't know what they need until you first give them what they ask for and they realize it's not what they want. And, sometimes (rarely, in my experience, but it does happen), clients actually do know what they want - at least, more than the developers do. Maybe they expect this report to only be used for a few months? That would make a large up-front investment a poor choice.
The only way you can know is by getting feedback.
So. Go for the quick solution that gets early feedback. But build it in such a way that's easy to change in the future. Take a look at the Repository Pattern of Domain-Driven Design that facilitates the Dependency Inversion Principle.