0

My apology if this questions is more opinion based but as an answer, can you back it up by provide a link to resources that backup your argument.

My team is doing Agile for a Saas software where we are currently pushing out a release every 3 months. Ideally I want to move to pushing a release to production after each sprint which is 3 weeks long.

Our organization has a release process in place calls ECO - Engineering Change Request. It calls for beta period with customer before production release.

Is this against Agile methodology for Saas application? We showcase new feature from time to time to customer and ask for their input and we deploy it ahead of time to Beta site that anyone can access, but we don't have a "dedicated" beta customer to test this feature on.

In my mind, Agile helps to release the feature as quick as you can with good quality control so that you can get it into your user's hands and trigger that feedback loop process to improve upon that feature. The idea of beta period before production release is really like a water fall model to me.

2 Answers 2

3

What you describe doesn't sound like what I typically consider to be a "Beta". It sounds more like User Acceptance Testing, or UAT.

In my experience with regulated customers, they expect the ability to perform additional testing prior to adopting new features or functionality. This helps them to maintain compliance by executing and documenting tests against their specific requirements, confirm functionality against any integrations, and make sure the system fits their needs. Some non-regulated customers may also perform similar tasks, especially for systems that are related to critical business functions.

Beta releases, in my experience, happen when functionality is feature complete but may have bugs or need additional user experience polish. This lets users use the software and provide feedback to the development organization.

Neither UAT or Beta releases are anti-Agile. I'd argue that Beta releases go very well with iterative and incremental release models and help the team get rapid feedback from large numbers of users. UAT is a bit harder to manage, since it requires users to participate in the process, but there are techniques (both technical and stakeholder management) that can help make it easier to fit into agile approaches.

When it comes to UAT, my recommendation is to typically consider UAT feedback as being equal to production feedback. It is not an opportunity to find defects in the software, but for your customer or users to carry out internal quality assurance activities. It's normal to get feedback, but if the feedback is that the product is unacceptable, that should be treated the same as production defects. I encourage teams to consider root cause analysis on production defects to find opportunities to improve both the product and the processes used to build it to prevent production issues.

1

Thomas's answer is on the money, however I wanted to add another couple of points and ideas.

First, Agile doesn't necessarily suggest you must release features as quick as you can, it suggests that you should always be in a position to release as quick as you can. Such as this is, often we complete our work for our sprint, get sign-off from the PO in the demo and then forget about it. At that point, we've done our bit, and it's up to the PO to choose when to ship. Normally, that would be ASAP, but often, for a number of reasons, it isn't. There's nothing wrong with that approach, but as you mention, it limits the impact of the feedback loop, which is a negative point.

Another reason we go through this "beta" testing phase (or "eating our own dog food" because internally, we'd pick up the product and use it for a set period before any of our clients did) was because our teams were quite immature and quality was lacking. It was critical for us to pick up any issues ourselves before shipping to a client. In a mature technical environment, this shouldn't be the case, but needs must. It's not ideal, but I wouldn't say it's "anti-agile" either.

Agile isn't black and white. Some things work for teams and others don't. Take what you can from it, but don't fit a square peg into a round hole just to say you're being "agile" - that's more likely to result anti-patterns than not.

3
  • 1
    Beta testing and "eating your own dog food" are very different. In order to "eat your own dog food", the organization building the software has to be able to use it. Not all software can be used by the teams creating it, or at least use it in the way that it was intended. Even if the organization can't use their software, they could still release beta functionality in some way to end users who can use the software and provide feedback.
    – Thomas Owens
    Apr 15 at 11:43
  • a follow up question, so for case of Hot Fixes or Patches, are those fair game then? since the turn around for some of these on my team can be just days from the bug created to the time it is being pushed to Production.
    – Fylix
    Apr 18 at 13:59
  • 1
    @Fylix The reason the manifesto is so short is because it doesn't prescribe exactly what to do in all instances. Instead, it suggests you try different things and keep those things that add value. So, do you get much value from running a beta test for a small change/fix? Only your team will know after trying it. Sorry it's a bit of a non-answer, but it really depends on a lot of things and I wouldn't want to guess how much value your team gets from trying something out :)
    – dKen
    Apr 18 at 17:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.