You lack coherent goals for the machine learning track. This is ultimately the problem you need to solve. Your underlying issue isn't really the type of work, or the framework you're using; it's that part of your project is currently missing clearly-defined goals and the ability to routinely measure progress (or the lack of it) towards those goals.
The one-sentence solution to your problem is: Treat machine learning as goal-directed, test-driven work. That's certainly an oversimplification, but I provide more details below that should help you and your teams design a workflow to do that more effectively.
Analysis and Recommendations
There are some recurring steps that need to be done over and over again:
- Upload test images to the app to run analyses
- Check if the results are correct (or at least acceptable)
While you could treat these as a rinse-and-repeat process, doing so is most likely an anti-pattern. Each iteration, the ML deliverable ought to have some sort of well-defined and achievable goal, even if the goal is just to learn something that helps follow-on planning (e.g. what many agilists call a "story spike"). Identify the ML goals and success criteria for each iteration, and then build your work plans accordingly.
As an example, it's unlikely you're uploading random images and that you have no Definition of Done or measurable metrics for the expected results. So, flip the script and define what "done" means for that iteration first.
Define Objectives and Key Results
For example, using a BDD framework like Cucumber:
NB: The following is simply a basic example for project management purposes, and not intended to be a detailed technical example. You might want to use Cucumber Scenario Outlines if you need more granular information about what the AI is getting right or wrong.
Feature: Identify Red Balloons in Images
With a sample set of 1,000 balloons and 4,000
other non-balloon images, the system will
correctly identify only the 100 red balloons
with >= 80% accuracy.
Rule: Correctly Identify Balloon Shapes
Example: Find at least 800 balloons in data set.
Given "data set 12345"
When the model ingests "images" from the data set
Then it should find at least "800 balloons"
# Make sure to add some failing examples, and
# test for both false positives and false
# negatives. A Cucumber Scenario Outline might
# be useful for defining multiple success/failure
# criteria including tests for boundary conditions.
Rule: Correctly Identify Only Red Balloons
Example: Find at least 80 red balloons in data set.
Given "data set 12345"
When "800" to "1_000" balloons are identified
Then the model should find at least "80 red balloons"
You don't have to use Cucumber. You don't even have to use a formal test framework of any kind. However, setting a goal for the iteration and then determining how you will measure whether or not you've met that goal, is absolutely essential.
Time-Boxed Development vs. Continuous Operations
When you have "rinse and repeat" tasks, you're usually dealing with operations rather than product development. There are always exceptions, but treating your ML work as evergreen tasks is:
- Unsuited to time-boxed iterative development using frameworks like Scrum.
- While potentially suitable to flow-based or operations-oriented frameworks—no kibitzing about Lean and Kanban also being suitable for creative development, please; yes, they can be, but that's not their ideal use case IMHO—having separate frameworks for a pair of teams that are inherently working towards the same product goals is another anti-pattern.
While you can make arguments for the teams being separate, you can also make arguments for the teams to work together so that they're delivering vertical slices of value rather than running on decoupled tracks.
Even if you decide that you'd prefer to have your web development team use Scrum and your ML team to work a separate track using Kanban or the like, you still need to ensure that the work for the ML team meets INVEST criteria. Broadly speaking, that means the machine learning work should be goal-oriented and measurable too. Even pure research projects, which I've previously addressed, can be decomposed and redefined with testing and measurable goals firmly in mind.
Invite the Team to Collaborate on Cadence and Decomposition
Ultimately, the ML team is part of your product delivery cycle. As such, it makes sense to have a conversation with those team members to see how you can work with them to set goals, measure progress, and visualize the work.
You may also want to have a collaborative discussion with your web development team. See if there are synergies that can be leveraged with CI/CD, automated testing, or other aspects of that part of the project that could help the ML specialists think like agile team members.
You should also invite the ML folks to your Backlog Refinement meetings, and either roll them into your overall Sprint Planning events or have similar meetings with them where work can be decomposed, planned, and some Definition of Done defined so that their work is both directed and performed on some sort of predictable cadence.
Whatever your framework, cadence is the heartbeat of agility. It's the inspect-and-adapt loop at work, and failure to define a cadence or provide routine opportunities to inspect, adapt, or redefine the goals and deliverables within a project are among the most common reason such projects fail. At the very least, that's part of the reason the machine learning portion of your project is struggling and why you should address the lack of cohesion between the work streams head-on.