Skip to main content
Minor grammar and semantic changes for clarity.
Source Link
Todd A. Jacobs
  • 50.7k
  • 7
  • 60
  • 181

While you could treat these as a rinse-and-repeat process, doing so is most likely an anti-pattern. Each iteration, the ML deliverable ought to have some sort of well-defined and achievable goal, even if the goal is just to learn something that helps follow-on planning (e.g. what many agilists call a story spike"story spike"). Identify the ML goals and success criteria for each iteration, and then build your work plans accordingly.

You don't have to use Cucumber. You don't even have to use a formal test framework of any kind. However, by setting a goal for the iteration and then determining how you will measure whether or not you've met that goal, is absolutely essential.

  1. Unsuited to time-boxed iterative development using frameworks like Scrum.
  2. While potentially suitable to flow-based or operations-oriented frameworks—no kibitzing about Lean and Kanban also being suitable for productioncreative development, please; yes, they can be, but that's not their ideal use case IMHO—having separate frameworks for a pair of teams that are inherently working towards the same product goals is another anti-pattern.

While you can make arguments for the teams being separate, you can also make arguments for the teams to work together, so that you'rethey're delivering vertical slices of value rather than two different work streamsrunning on decoupled tracks.

Even if you decide that you'd prefer to have your web development team use Scrum and your ML team to work a separate track using Kanban or the like, you still need to ensure that the work for the ML team meets INVEST criteria is at least. Broadly speaking, that means the machine learning work should be goal-oriented and measurable too. Even pure research projects, which I've previously addressed, can be decomposed and re-definedredefined with testing and measurable goals firmly in mind.

You may also want to have a collaborative discussion with your web development team, and see. See if there are synergies that can be leveraged with CI/CD, automated testing, or other aspects of that part of the project that could help the ML specialists think like agile team members.

Whatever your framework, cadence is the heartbeat of agility. It's the inspect-and-adapt loop at work, and failure to define a cadence or setprovide routine opportunities to inspect, adapt, or redefine the goals and deliverables within a project are among the most common reason such projects fail. At the very least, that's part of the reason the machine learning portion of your project is struggling, so and why you should address itthe lack of cohesion between the work streams head-on.

While you could treat these as a rinse-and-repeat process, doing so is most likely an anti-pattern. Each iteration, the ML deliverable ought to have some sort of well-defined and achievable goal, even if the goal is just to learn something that helps follow-on planning (e.g. what many agilists call a story spike). Identify the ML goals and success criteria for each iteration, and then build your work plans accordingly.

You don't have to use Cucumber. You don't even have to use a formal test framework of any kind. However, by setting a goal for the iteration and determining how you will measure whether or not you've met that goal is essential.

  1. Unsuited to time-boxed iterative development using frameworks like Scrum.
  2. While potentially suitable to flow-based or operations-oriented frameworks—no kibitzing about Lean and Kanban also being suitable for production, please; yes, they can be, but that's not their ideal use case IMHO—having separate frameworks for a pair of teams that are inherently working towards the same product goals is another anti-pattern.

While you can make arguments for the teams being separate, you can also make arguments for the teams to work together, so that you're delivering vertical slices of value rather than two different work streams.

Even if you decide that you'd prefer to have your web development team use Scrum and your ML team to work a separate track using Kanban or the like, you still need to ensure that the work for the ML team meets INVEST criteria is at least goal-oriented and measurable. Even pure research projects, which I've previously addressed can be decomposed and re-defined with testing and measurable goals firmly in mind.

You may also want to have a collaborative discussion with your web development team, and see if there are synergies that can be leveraged with CI/CD, automated testing, or other aspects of that part of the project that could help the ML specialists think like agile team members.

Whatever your framework, cadence is the heartbeat of agility. It's the inspect-and-adapt loop at work, and failure to define a cadence or set routine opportunities to inspect, adapt, or redefine the goals and deliverables within a project are among the most common reason such projects fail. At the very least, that's part of the reason the machine learning portion of your project is struggling, so address it head-on.

While you could treat these as a rinse-and-repeat process, doing so is most likely an anti-pattern. Each iteration, the ML deliverable ought to have some sort of well-defined and achievable goal, even if the goal is just to learn something that helps follow-on planning (e.g. what many agilists call a "story spike"). Identify the ML goals and success criteria for each iteration, and then build your work plans accordingly.

You don't have to use Cucumber. You don't even have to use a formal test framework of any kind. However, setting a goal for the iteration and then determining how you will measure whether or not you've met that goal, is absolutely essential.

  1. Unsuited to time-boxed iterative development using frameworks like Scrum.
  2. While potentially suitable to flow-based or operations-oriented frameworks—no kibitzing about Lean and Kanban also being suitable for creative development, please; yes, they can be, but that's not their ideal use case IMHO—having separate frameworks for a pair of teams that are inherently working towards the same product goals is another anti-pattern.

While you can make arguments for the teams being separate, you can also make arguments for the teams to work together so that they're delivering vertical slices of value rather than running on decoupled tracks.

Even if you decide that you'd prefer to have your web development team use Scrum and your ML team to work a separate track using Kanban or the like, you still need to ensure that the work for the ML team meets INVEST criteria. Broadly speaking, that means the machine learning work should be goal-oriented and measurable too. Even pure research projects, which I've previously addressed, can be decomposed and redefined with testing and measurable goals firmly in mind.

You may also want to have a collaborative discussion with your web development team. See if there are synergies that can be leveraged with CI/CD, automated testing, or other aspects of that part of the project that could help the ML specialists think like agile team members.

Whatever your framework, cadence is the heartbeat of agility. It's the inspect-and-adapt loop at work, and failure to define a cadence or provide routine opportunities to inspect, adapt, or redefine the goals and deliverables within a project are among the most common reason such projects fail. At the very least, that's part of the reason the machine learning portion of your project is struggling and why you should address the lack of cohesion between the work streams head-on.

Source Link
Todd A. Jacobs
  • 50.7k
  • 7
  • 60
  • 181

TL;DR

You lack coherent goals for the machine learning track. This is ultimately the problem you need to solve. Your underlying issue isn't really the type of work, or the framework you're using; it's that part of your project is currently missing clearly-defined goals and the ability to routinely measure progress (or the lack of it) towards those goals.

The one-sentence solution to your problem is: Treat machine learning as goal-directed, test-driven work. That's certainly an oversimplification, but I provide more details below that should help you and your teams design a workflow to do that more effectively.

Analysis and Recommendations

There are some recurring steps that need to be done over and over again:

  1. Upload test images to the app to run analyses
  2. Check if the results are correct (or at least acceptable)

While you could treat these as a rinse-and-repeat process, doing so is most likely an anti-pattern. Each iteration, the ML deliverable ought to have some sort of well-defined and achievable goal, even if the goal is just to learn something that helps follow-on planning (e.g. what many agilists call a story spike). Identify the ML goals and success criteria for each iteration, and then build your work plans accordingly.

As an example, it's unlikely you're uploading random images and that you have no Definition of Done or measurable metrics for the expected results. So, flip the script and define what "done" means for that iteration first.

Define Objectives and Key Results

For example, using a BDD framework like Cucumber:


NB: The following is simply a basic example for project management purposes, and not intended to be a detailed technical example. You might want to use Cucumber Scenario Outlines if you need more granular information about what the AI is getting right or wrong.



Feature: Identify Red Balloons in Images
  With a sample set of 1,000 balloons and 4,000
  other non-balloon images, the system will
  correctly identify only the 100 red balloons
  with >= 80% accuracy. 

  Rule: Correctly Identify Balloon Shapes

    Example: Find at least 800 balloons in data set.

      Given "data set 12345"
      When the model ingests "images" from the data set
      Then it should find at least "800 balloons"

      # Make sure to add some failing examples, and
      # test for both false positives and false
      # negatives. A Cucumber Scenario Outline might
      # be useful for defining multiple success/failure
      # criteria including tests for boundary conditions.

  Rule: Correctly Identify Only Red Balloons

    Example: Find at least 80 red balloons in data set.
      Given "data set 12345"
      When "800" to "1_000" balloons are identified
      Then the model should find at least "80 red balloons"

You don't have to use Cucumber. You don't even have to use a formal test framework of any kind. However, by setting a goal for the iteration and determining how you will measure whether or not you've met that goal is essential.

Time-Boxed Development vs. Continuous Operations

When you have "rinse and repeat" tasks, you're usually dealing with operations rather than product development. There are always exceptions, but treating your ML work as evergreen tasks is:

  1. Unsuited to time-boxed iterative development using frameworks like Scrum.
  2. While potentially suitable to flow-based or operations-oriented frameworks—no kibitzing about Lean and Kanban also being suitable for production, please; yes, they can be, but that's not their ideal use case IMHO—having separate frameworks for a pair of teams that are inherently working towards the same product goals is another anti-pattern.

While you can make arguments for the teams being separate, you can also make arguments for the teams to work together, so that you're delivering vertical slices of value rather than two different work streams.

Even if you decide that you'd prefer to have your web development team use Scrum and your ML team to work a separate track using Kanban or the like, you still need to ensure that the work for the ML team meets INVEST criteria is at least goal-oriented and measurable. Even pure research projects, which I've previously addressed can be decomposed and re-defined with testing and measurable goals firmly in mind.

Invite the Team to Collaborate on Cadence and Decomposition

Ultimately, the ML team is part of your product delivery cycle. As such, it makes sense to have a conversation with those team members to see how you can work with them to set goals, measure progress, and visualize the work.

You may also want to have a collaborative discussion with your web development team, and see if there are synergies that can be leveraged with CI/CD, automated testing, or other aspects of that part of the project that could help the ML specialists think like agile team members.

You should also invite the ML folks to your Backlog Refinement meetings, and either roll them into your overall Sprint Planning events or have similar meetings with them where work can be decomposed, planned, and some Definition of Done defined so that their work is both directed and performed on some sort of predictable cadence.

Whatever your framework, cadence is the heartbeat of agility. It's the inspect-and-adapt loop at work, and failure to define a cadence or set routine opportunities to inspect, adapt, or redefine the goals and deliverables within a project are among the most common reason such projects fail. At the very least, that's part of the reason the machine learning portion of your project is struggling, so address it head-on.