How can a Scrum team work with a Waterfall team

Question

My Scrum team needs an API to retrieve data. It's not overly complicated, essentially its just one request that gets sent to the API and then the API returns some data. This is the end of the journey.

Our problem is that the team designing the API is using Waterfall. The waterfall team is also designing the API to cater for numerous other teams needing differing versions of the data returned.

They seem to have layers and layers of project managers and programme managers sitting on top of the actual development team and they are not located in the same city. They use strict waterfall, so they want ALL our requirements up front and they use legacy approaches, e.g. they don't really know how AWS (where our service sits) works etc.

How do you recommend my Scrum team works with them or tries to work with them?

We were thinking of trying to arrange a regular chat between their development team and ours but we not really sure the project managers would allow that. They are also strict. No changes to the requirements. They want to lock down the requirements.

So we are a little unsure as we are not experienced with this way of working.

Answer 1

Pretend the Waterfall team is an outside contractor

Since you're interfacing with the Waterfall team's code solely via an API, just pretend you're hiring an outside contractor to create that code for you.

The Scrum rules for how you write your code don't apply to them. You submit your requirements to them, they write up a formal Statement of Work for what they will do for you, which might go through some back-and-forth before both sides agree. Once everyone's signed off on it, they go implement their code.

Meanwhile, someone on your team should create a dummy that uses the contracted API, so that you can start writing code that interfaces with it. As soon as the Waterfall team's code is usable, substitute it.

Answer 2

First of all, make it known to your stakeholders that there is a project risk in your project. The waterfall team is asking for things (final, unchanging requirements) that you don't have at the start of your project.

To mitigate this risk, you can analyse your current backlog for items that may affect the data that you need to exchange with the system developed by the other team. Refine those stories to such a level that you can proactively write the requirements for what you might need in the future. In this process, also use the experience from the team to predict possible/likely future changes to the backlog that affect this.

In this process, you are probably asking for more than you will actually use, because some stories get dropped or predicted changes don't come about, but that is the cost of dealing with the waterfall team.

Additionally, you can try to negotiate a lenient and efficient change-request process with the waterfall team, so that it becomes easier to request changes after the initial requirements were baselined. If you have that in place, you don't have to analyse the backlog items as deeply and you can keep the less likely changes out of the initial requirements.

Count yourself lucky that you have a fairly narrow interface with the waterfall team (a single API call). Because of that, you can try to accommodate their process without sacrificing your process too much.

Answer 3

For what it's worth, APIs and protocols are an area you often do need things fixed. This has to be very deliberately and explicitly designed because if you need to change it later, every consumer will have to update their code. APIs which are often changing are a nightmare to work with especially if they implement backwards incompatibilities.

I do not see a reason that settling on an interface protocol cannot be done in a SCRUM setting, in several iterations even - but you need to be happy with it before sending that to the other party.

Of course you can make your interface extendable if you actually cannot decide all requirements, or you can design an interface that is flexible for future expansion without changes.

Answer 4

I have been in this situation a couple of times and it is very challenging.

Things that can help:

• Spend some time with the waterfall team explaining how you work. Even if you can't resolve the conflicts between the approaches it still helps if they understand what you are trying to do.
• Ask them to produce a mock/stub at the earliest opportunity to at least allow you to make progress without becoming totally blocked.
• Prioritise the work associated with the dependency to at least get it out of the way as quickly as possible.

Answer 5

When the API is not too complex, you could create your own mock API to experiment with which returns test data. When you got your requirements down, you can contact them to create a real API delivering real data.

Answer 6

They are also strict. No changes to the requirements. They want to lock down the requirements.

That is not really true though. They would like to restrict changes to the requirements, for sure, because that's the nature of waterfall development. In practise though it is never possible to avoid some changes to requirements, because issues will always occur during development where requirements need to be refined. It's also important to note that requirements probably will be locked down during development, so that developers can get their work done efficiently. After that, they'll look at any changes you ask for.

You may be using Scrum, but you should still have a representative set of use cases, shouldn't you? If you have your use cases, you should know what data you're likely to need from their API. That gives you the requirements to pass to them, or at least enough requirements that minor later refinements can be treated like any other bugfix.

Answer 7

Remember, Scrum can be applied to anything, not just software development. It can be applied to fleshing out API requirements too.

I assume your entire project does not revolve on this one API call. Just put a priority of fleshing out your API requirements.

In this you are your own product owner. Put the API-related feature / code at the top of your product backlog, write up your API requirements, and go. Sprint through it as if API existed, and then figure out what else your API needs, repeat until you are done with API-related stuff. Give the finished product (API requirements) to the waterfall team.

Answer 8

When you are designing an API which will be consumed by many business units (and maybe by external customers), a Waterfall approach with fixed and rigidly-stated requirements might very well be exactly what should be done. The business impact of such a layer of software is enormous. "This one scrum team" is only one of the many clients that this layer has.

I believe that even the Agile team must comply with their demand for fixed requirements. Build a mock-API to allow you to continue development on your project before the actual API is delivered. Design the software which consumes this API to be easily changed if when the implementation changes.