Agile in a reactive organisation?

Question

I am working in an organization as a Scrum Master that is extremely reactive by nature, where the ‘agile’ teams are being given extremely aggressive ‘non-negotiable’ deadlines by senior executives, which have a fixed scope of work.

Where in a recent project my team were unable to push back and it lead to their velocity not being respected - to deliver the project, required them to work x2, x3 their velocity and scope creep during sprint cycles. When I’ve tried to push back I’ve faced resistance in the form of ‘we have to do this - it is what Snr Execs want’.

I am now in a dilemma where on one hand the org is expecting us to follow agile best practices - we use Scrum but on the other hand not respecting first principles to enable us to implement it properly meaning agility has to be compromised from teams having to end up working in a waterfall way.

What is the best way to implement agile in these circumstances?

Answer 1

You ask:

What is the best way to implement agile in these circumstances?

Before thinking about this question you should think about how to bring in some common sense into the situation. Having people work overtime isn't sustainable in the long run and with time people start to make mistakes because they are tired and stressed. They end up working more and producing less because they have to fix the mess they create while going too fast than what's reasonable.

You also mention forced scope with scope creep. This is another source of problems. You have to do something and it's non negotiable, but then apparently it wasn't the right thing to do because changes are demanded. This causes rework and frustration which adds to the already negative atmosphere people work in with overtime and pressure.

Only bad managers and executives fix project issues by putting pressure on people and asking for a lot of overtime. You are calling this a reactive organization, when it's in fact a dysfunctional organization. No matter what labels you put on what's going on, reactive, Agile or Waterfall, what you have here is a death march plain and simple.

You are not doing Agile and the chances of implementing good Agile practices are slim to none in this situation you find yourself in. So before thinking about that, you should try to step down a gear.

You need to find some time to look at things and think about the best course of action, and you can't do that if you are constantly busy trying to deliver something that was imposed on you. There needs to be some slack in the system in order to inspect and adapt, but it seems your upper management has decided to go not just for 100% resource utilization but 200% or 300%.

So first thing to figure out is if this death march is an exception or if this is just management expects from people. Sometimes there are good reasons to ask for overtime and demand a big effort from people (regulation, emergencies, etc) but it must be something that should happen rarely. In all other situations people should use good project management practices to keep things moving. So ask about the "why". Why is scope forced on team? Why are the deadlines aggressive and non-negotiable? Where are these decisions coming from and why? What happens if you miss the deadlines?

Once you find out what's going on then you are better equipped to try to fix it. At that point you can explain that this way of working isn't sustainable and suggest solutions to fix it, solutions that will need to involve collaboration from higher management, mainly to negotiate realistic scope, deadlines and capacity. Depending on why this is happening and upper management willingness and openness to have a true conversation, you will know what chances of success you will have in later trying to find the best way to implement Agile.

I really hope people will be opened to having this conversation, although in many circumstances upper management just replaces Waterfall with Scrum and think that they will get twice the work in half the time from just the fact of labeling things with a different name.

Answer 2

What is the best way to implement agile in these circumstances?

You cannot. Agile is not a grass roots movement you can do "against" or without the respect of the higher ups. It's all about team work and if part of the team does not want team work, then you cannot make them. Especially not if they are your bosses.

It's up to you to decide whether this is a one time occurrence and you can live with it or if it's a recurring scheme. If it's recurring, and there is no sign that your bosses will be replaced any time soon, the only option is to "fire your boss". Also known as quitting and getting a better job.

---

Just to give you a perspective on how ridiculous this is: Did you ever consider going to your bosses saying "I signed a contract just yesterday for a brand new car. It's great. Turns out that my estimation of how much I make per month was a little optimistic, so you will need to pay me double or maybe triple, I don't know yet whether I will take the extra tire package. It's a business need for me now, because I already signed." That is what they did. Sign some kind of deal that requires you to work double or triple for their gain. If you had done that, they would probably have asked if you are on drugs, it's that absurd. But they think it's okay if they do it. There is no way you will get any respect or any of the other agile values from them. People do not change at their core. If you want that, you need to find people that are team players.

---

And another remark concerning your choice of words. That company isn't reactive. Reactive means you react in an orderly fashion. With a plan based on reality. The things you describe can happen once in a while because we all make mistakes, even in a team, even on a higher pay scale, nobody is perfect. But if this is a pattern, it's not "reactive", it's chaotic and exploitative.

Answer 3

I would recommend rethinking why you want to transform to agile methods. Agile methods are designed to deal with unclear, ambiguous, or changing requirements. If you have a fixed-scope body of work with a fixed-date delivery deadline, many of the techniques associated with agile methods will add overhead.

In this situation, you can't just "implement agile". You need to make fundamental changes to how the organization approaches work. In complex efforts, it's often not possible to specify the work up-front. By doing the work, more details about what is truly required will emerge. Agile methods are designed to handle continuous rescoping of work and incorporating feedback.

Until you make the fundamental changes, it doesn't make much sense to apply agile methods.

Answer 4

Managers need to manage

The truism of "good, fast or cheap - pick one" is always relevant. You can manage with that, so long as you know what's acceptable. If senior execs expect all three, managers MUST push back. That is literally the only reason for having middle managers. If it was as simple as just giving a task and a deadline to engineers, there's no reason for middle management to exist, except perhaps as a scapegoat for people higher up the tree.

Let's assume that you're not merely a scapegoat and you might have some agency, because otherwise all bets are off. In that case, the answer is to...

Make senior execs, individually or collectively, accountable for consequences

If you can code it in that time but you can't carry out testing, the answer back to management is "My team cannot guarantee that this will work. Do you accept full personal responsibility for every technical fault, and any resulting damages?" (Appropriately tactfully phrased, of course, but that's what it needs to amount to.)

If the execs want it fast anyway, the answer back is "My team will not work overtime for free, if you don't want them to jump ship to $Competitor. What bonuses can you offer them to work extra hours?"

Or there may be a compromise possible of "We can do it fast this time, but we have N months backlog of technical debt. Do you accept that my team get N months of no new feature requests, for any reason whatsoever?"

Or of course they may just say "we don't care". In which case your reply needs to be "I can tell my team to do this, but we will lose team members to $Competitor, and the time to do any work in future will be significantly worse. Do you accept this as a consequence of your strategy?"

And of course make sure all this is in emails and meeting minutes, with the senior execs having signed up to it. At some point down the line, the shit will hit the fan, and your job is making sure all the shit lands on the senior execs in question.

But this isn't specific to agile

Agile is merely how you produce a deliverable efficiently to the required quality with the available man-hours, in a way where you can track progress towards your goal. If you have lower quality standards or more man-hours, you tailor your sprints accordingly. What you need from the senior execs is just their buy-in on reducing quality or increasing man-hours.

Answer 5

I have worked in consultancies with a similar setup. They need to give the appearance of using agile methods to look cutting edge, but typically they only pay lip service to them.

What I have found is that you can use elements of agile frameworks like Scrum to show them just how bad things are.

For example, say the team is given a fixed-scope, fixed-deadline project which is unrealistic. Accept the work and don't argue about it. However, measure your teams rate of progress (for example by using user stories and velocity) and use that to get a more realistic idea of what is possible. Make this visible to everyone, execs included. Offer them options, for example work that could be cut out to make the deadlines more realistic. If they choose to ignore this information then that is fine, but at least they are now better informed about the true situation.

In parallel with this try and coach them about the value of an agile approach. Sell them on the benefits and how it could potentially make it easier to deliver projects and make money.

It will take some time, but it is possible that you can slowly win them around to working in a more productive way.

Answer 6

Arrange a meeting with the high-ups for a MoSCoW prioritization.

The bosses want you to do Agile? Well, then you're going to do Agile, and that means MoSCoW prioritization. Split the task you're doing into pieces, use Story Points to estimate the amount of effort it would take these pieces, and arrange a meeting with the bosses to prioritize those pieces into things you Must do, things you Should do (but aren't mandatory), things you Could do if time allows, and things you Won't do since they're outside of scope. If the bosses try to push you into doing everything as a Must, remind them that to be Agile, the amount of Musts can only be 50% of the total Story Points. If they're hesitant to book a meeting with you, then remind them that it's a part of the Agile process, and if they want you to work in a properly Agile fashion, then this meeting is a necessity.

Then, work on the tasks from the highest priority downwards, in order to deliver a Mininum Viable Product. If some of the Shoulds and Coulds don't get done, then that's fine! That's how Agile works; it's the tradeoff that they agreed to in the prioritization meeting by asking you to work Agile. If they're not happy about that, well, they were the ones who wanted to work in an Agile way - if they wanted 100% feature delivery, then they should have asked you to work in a Waterfall way, and accept that there are likely to be delays and cost overruns.

Answer 7

This sounds so far away from what Agile is supposed to be, it's almost unbelievable. Read https://agilemanifesto.org/principles.html and see if it sounds anything like the way your management is behaving, particularly "Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.". This company is not implementing agile (small 'a') practices, it is using Agile (capital 'A') as a new method to bully and overwork their employees. If you're interested in agile, leave this place immediately and find somewhere that really understands what agile is.

Answer 8

Realistically, the only way to deal with this is to be prepared to find a new job, and then push back with some non-negotiable standards your team will operate by. You have to realize that it isn't a matter of technique, but executives have put you and the team in a ridiculous situation so that no matter what you do they won't be satisfied.

The alternative is to try and feed BS to both sides at once and hope that you can get enough done while constantly being criticized. That works out in most cases, but no one is happy with it.

Maybe pushing back fiercely will give you and your team enough space to do what you can do.

Answer 9

Ex-Project Manager, ex-Line Manager, ex-Scrum Master, and ex-Product Owner here. 🤷‍♂️

A picture says more than a 1000 words, so give this to management and ask them if they want to risk business continuity planning by overloading your team beyond a division by 0:

i.e. Does upper management want to risk the entire / partial / ... business ¹ when a disaster happens and your team will not not be able to cope with extensive business fall-out???

• If not based in the US of A, mention e.g. "COVID-19" and the last major outage in your company.
• If based in the USA: mention 9/11 and companies having their Disaster Recovery site being in the other tower and going out of business because of that decision and your team being within one of the two towers!

Note 1: depending on the business criticality your application / framework / ... represents, ensure you make the number big! E.G. if you're a core framework team, make it about the total turn-over of the entire company!

What is the best way to implement agile in these circumstances?

As mentioned by other people already: You want to, but you cannot! 😢

Answer 10

Agile is a tool, not a solution. In this case, I would phrase agile as a tool which can be used to help leadership understand the company they are running. If leadership not only does not wish to understand their company, but would go further such that we can say that they wish to not understand their company, there really isn't anything that can be done from a tool perspective. This is now a 100% emotional intelligence problem. How do you teach a senior executive how to run their company? This is far beyond agile, and indeed may qualify you to run your own company (so have humility in the process... running a company like this is a heck of a challenge! If I were to work at SpaceX, I'd be hard pressed to argue against Elon Musk's approach of working people to the bone. Its how his company functions)

One approach I use is to try to analyze the problem from the senior executive's position. They are trying to navigate a business environment. These environments are often treacherous, with imperfect information. One of SCRUM's goals is transparency, but once you get up high enough, opacity is often a virtue. As a simple example, I know of very few companies that offer unlimited transparency to permit their customers to inspect their inner workings. In fact, I don't think they exist. There's always a level of opacity here, to protect business secrets, and a complicated trust relationship between customer and producer.

This relationship is probably the place to look. What does your company provide the customer, including what promises and contracts management signs, not just the final products. In long-lead-time production, these contracts themselves are often work products that are essential for making the final delivery happen. As much as I hate to say it, sometimes this is true even when the contracts are absurdly idealized. How your company goes about "correcting" their absurd promises is also a key element of this... the relationship between customer and producer is important!

Then, look at your relationship between the senior executive and the rest of the teams. Ask yourself how transparent that relationship should be. You may arrive at the conclusion that it should be transparent, which is the idealized answer. Or you may decide that this leadership is toxic enough that you actually need to insulate the work from them. I cannot recommend one way or the other, its your company with your individual people involved. But there should be some balance that you can strike.

With these in mind, you can start to formulate goals in the form "Support company product X through interacting with senior executive Y." These goals, once small enough, become product (or even sprint) goals for your agile team.

Then, use agile for what it's good for:

• If the senior executive (your "customer") changes direction often, use agile to respond to these changes.
• Use agile to protect the developers. You may not be able to protect them from the OT the senior leadership is calling for, but you can collect metrics which can be used to protect the reputation and performance reviews of the developers. (the OT may burn them out, but you can do your best to support them until then)
• Use agile to develop the people, not just the product. Jeff Southerland's TED talk explains how he uses SCRUM as a tool to develop better self-lead leaders. At the very least, when senior management's OT expectations burn the team our, they can go to their next job with a valuable skillset.
  • There's an interesting balance to be struck with letting the developers bend the "just get it done as fast as possible" mindset. Giving the developers some freedom to do things differently may not quite support the "transparency" ideal with your toxic leadership, but empowered developers are more willing to work long hours than cogs in the machine. You may actually be supporting your leadership's real expectations of OT by treating your developers like human beings able to chart their own path. Teach them how to plot it within the structure leadership provides -- this is a very useful life skill.
  • As an anecdote, I worked on a toxic team with bad leadership (not senior leadership, but senior enough for our small team). We chose to use SCRUM as our tool for planning, and between us and our direct leadership, we focused on transparency. We recognized the toxic environment above us, and literally treated the production as a training exercise in how to develop successfully in a real world environment (rather than the idealized utopias they teach in classess). Much of how I use SCRUM today leverages this experience, because I now have confidence that, should an environment turn toxic, the tools and planning elements I am putting in place will be sufficient to survive. (And that particular effort was so successful by management's selected metrics, that it got extended for two additional periods of performance...)
• Manage expectations and enable the team. This is the obvious stuff you already know, but I felt it was worth putting a bullet at the end, just to make sure it made the bulleted list. Use this as tools to show that your team is working as advertised. Use burndown charts with the real velocities (both actual velocity and scaled to 0% overtime). Get your team more resources. God I wish it was as easy as I make it sound when writing this bullet... but the bullet is here anyways! In other words, "go do the thing you know you were supposed to be doing." You already understand it, or you wouldn't be asking these questions on Stack Exchange.

Now I know there's a billion agile methods out there, at least one for every developer. Some use different terms. My environment uses the concepts of both stories and features. Features are specified by leadership (like your senior exec.) and they can be whatever they need in the product. The stories are developed by the team, for the team. I find this division to be very useful because it assigns responsibility for planning failures squarely in the right wheelhouse. If your team plans stories to complete a feature, and fails to complete them in a timely manner, that's on them (and you should work with your team to build up better estimation). However, if the issue is management putting unreasonable deadlines in place, that shows up in the breakout of a feature. The approach of "planning packages" from EVMS can be a useful tool here. Management may assign the size of planning packages, which can then be compared against the velocity data you have. The team has to eventually break this out, developing a set of stories to implement the feature that was planned. If they come up with a much larger number, then there's a negotiating phase where the scope may shift, and then there's a reckoning where the leadership is forced to admit that they handed a 1000 point planing package to the developers which, when realized, was 3500 points.

A particular chart I have found useful for this is a product burndown chart for the lifespan of the effort (since you have deadlines). On this chart, I draw work in planning packages in a different color from work in stories. This results in a nice visual pattern: everywhere a poorly done planning package is broken out, there's a simultaneous drop in the amount of planning-package color and a sudden jump in total scope. It is then up to you and your emotional intelligence to demonstrate to the senior leadership that all of your scope creeps happen in the feature/planning-package parts, and that your sprint planning is spot on.