In most traditional project management mindsets, people are more concerned with their budgets and schedules for a project than turning out a usable product. They want to know what something is going to cost, how long it will take, and what they will get for their investment, all before any actual work gets done. It also assumes that the people asking for the product actually know what they want before they have anything to see or try. This leads to lots of documentation, no or minimal feedback mid process, and a lack of flexibility. There is an over emphasis on status reports and making sure everything matches the plan as originally laid out, and insufficient allowances for uncertainties and ambiguities.
Scrum, and agile in general, recognizes that the real world isn’t that structured and ridged, and that the people drawing up requirements won’t know what they really want until it is in their hands. It takes the approach of focusing on the most urgent thing of the moment, get it done, and move onto the next thing. It also relies on showing completed work to the stakeholders and end users as soon as possible, soliciting their feedback, and adjusting the plan accordingly. The overall goal is to always have the most useful product for the effort invested to date.
This does not play well with the traditional mindset because we do not have fixed timelines and rarely have a firm plan for anything that is more than a couple weeks out. We can report on the things that have been completed, what we are currently working on, and what we plan to start on next. But we can’t tell you when the project will be finished, what order features will come out in, or even guarantee that our “what is next” plan won’t change before we get to it.
To give a bit of a long-winded example, I once worked for a government agency, first as a developer, later as a team lead and manager. I am also a Certified Scrum Maser and Agile Coach, and I have worn BA and PM hats a fair amount in the past.
There was a phase where we were trying to “do agile” without any real support from our administration or stakeholders. Essentially the developers adopted some “agile” ceremonies, but everyone else was still turning out 30+ page requirements documents without involving the developers, demanding estimates for projects months in advance of when we would actually work on them, expecting us to adhere to timelines that were set without our input, pointing back to the documentation when we raised questions, and throwing fits anytime the project outcomes deviated from the requirements, timeline, or budget, regardless of the reason.
At its worst, what should have been a six month project took almost 2 years, and had to be rewritten twice because stakeholders changed, which changed the requirements, and even after the first rewrite, what made it into the requirements documents was still not what the end users actually needed.
Shortly after I took over as manager, I convinced people to let me run a project without interference. At first I was handed was one of those 30 page documents, with an expectation that we give an upfront estimate for cost and completion time so that it could go through the approval process. Normally this would have involved the manager(me) and possibly a team lead or senior dev going through the document in a day or so, and handing back some numbers that were 2 or 3 times what we thought they should be to account for the things we didn’t know and didn’t have time to figure out.
Instead I pulled the entire team in to review the document, break it down into features of a size that we could give meaningful estimates on, and plug it into our tracking system. I had to fight our PMO and other management repeatedly because we were taking too long and using too many resources to come up with said estimate (several 1-2 hour meetings over a couple weeks). However, in the end, we had a rough map of the work that was needed, every developer was in the loop on the direction and decisions we had made, and all had the opportunity to give input on the estimates. When we were done we presented a timeline of about 10 months with a cost that was calculated based on which developers and other resources we expected to work on it, and for what percentage of their time.
When said project was greenlighted a few months later (there was never any real question that we would do it) the first thing we did was dust off our previous breakdown, decide where we need to start to have anything to show for it, verify our estimates for that block of work, and toss the requirements document aside. I then set the team up with a near scrum approach, we ran 1 week sprints, with a release every 4 weeks. We had a stakeholder meeting every 2 weeks.
When the stakeholder meeting aligned with our release, we presented the new features, solicited initial feedback, and laid out our objectives for the next release. For the off cycle meetings, we were soliciting and discussing bug reports, and adjusting the release plan based on feedback from the end user testing. Again I made a point of having no less than three developers in the room for each stakeholder meeting, with the expectation that they would ask and answer questions as needed, but primarily so the team had more than one perspective on what was said. And again I got flack for it because I was “wasting” the budget by having developers sit in on meetings instead of coding.
At first it was a bit of a struggle to get the stakeholders to actually show up to our meetings, to the point where we suggested that the project be shelved since it apparently was not important enough to demand their attention. But after we got them into three or four meetings, where they could see the progress being made, and see their feedback being incorporated, they started being more willing to block out time for us. After about the second time where someone came out saying “this is cool, but what we really need is” about a feature, and saw the issue corrected in the next release or so, they were sold.
During the entire project I also was fighting a constant battle with the PMO over my status reports. They wanted to see the same types of project schedules and reports they were accustomed to, with features being checked off in a preset order and on schedule, and be able to match our expenditures to their budget projections. Instead I was giving them a report on story points that had been completed and accepted, estimates on points in progress and when they should be done, and revisions on the original estimates and timelines based on issues that had come up during development or testing.
They would throw a fit every time we re-estimated a story based on new information, added stories to track bugs, or broke things down into more granular work segments, especially if the change ended up changing the number of points in the project, complaining that we were changing scope. They also hated that my estimates were in points rather than hours, and wanted to know how to translate between the two. Eventually I put together a tool that linked task hours to stories to releases, and gave them a running average for points per release and hours per point, but they still didn’t like that it was not a fixed number and we refused to commit to a hard completion date until we were entering our last couple of release cycles, and it was obvious when things were going to wrap up.
In the end we finished the project in 11 months, were about 20% over our original budget, and had people actively using the product on release day, without a huge list of defects to delay the roll out. It was by far the most successful project ever completed for that agency in terms of meeting timeline, budget and usable features. To give some perspective, the norm for that department prior to this project was to be over on time by as much as 50%, at least double the budget, and have low initial adoption because the end product didn’t meet the actual needs of the organization or users.
In this case, I as the direct manager had bought in, and had enough pull and support from my supervisory chain to shelter my team and get away with bullying people from other silos into playing along, but the organization as a whole was not on board. Over the course of the project we brought most of our business side stakeholders around to supporting the methodology, but the PMO still only tolerated us as opposed to appreciating what we were doing.