Is Beta period required for Saas Agile?

Question

My apology if this questions is more opinion based but as an answer, can you back it up by provide a link to resources that backup your argument.

My team is doing Agile for a Saas software where we are currently pushing out a release every 3 months. Ideally I want to move to pushing a release to production after each sprint which is 3 weeks long.

Our organization has a release process in place calls ECO - Engineering Change Request. It calls for beta period with customer before production release.

Is this against Agile methodology for Saas application? We showcase new feature from time to time to customer and ask for their input and we deploy it ahead of time to Beta site that anyone can access, but we don't have a "dedicated" beta customer to test this feature on.

In my mind, Agile helps to release the feature as quick as you can with good quality control so that you can get it into your user's hands and trigger that feedback loop process to improve upon that feature. The idea of beta period before production release is really like a water fall model to me.

Answer 1

What you describe doesn't sound like what I typically consider to be a "Beta". It sounds more like User Acceptance Testing, or UAT.

In my experience with regulated customers, they expect the ability to perform additional testing prior to adopting new features or functionality. This helps them to maintain compliance by executing and documenting tests against their specific requirements, confirm functionality against any integrations, and make sure the system fits their needs. Some non-regulated customers may also perform similar tasks, especially for systems that are related to critical business functions.

Beta releases, in my experience, happen when functionality is feature complete but may have bugs or need additional user experience polish. This lets users use the software and provide feedback to the development organization.

Neither UAT or Beta releases are anti-Agile. I'd argue that Beta releases go very well with iterative and incremental release models and help the team get rapid feedback from large numbers of users. UAT is a bit harder to manage, since it requires users to participate in the process, but there are techniques (both technical and stakeholder management) that can help make it easier to fit into agile approaches.

When it comes to UAT, my recommendation is to typically consider UAT feedback as being equal to production feedback. It is not an opportunity to find defects in the software, but for your customer or users to carry out internal quality assurance activities. It's normal to get feedback, but if the feedback is that the product is unacceptable, that should be treated the same as production defects. I encourage teams to consider root cause analysis on production defects to find opportunities to improve both the product and the processes used to build it to prevent production issues.

Answer 2

Thomas's answer is on the money, however I wanted to add another couple of points and ideas.

First, Agile doesn't necessarily suggest you must release features as quick as you can, it suggests that you should always be in a position to release as quick as you can. Such as this is, often we complete our work for our sprint, get sign-off from the PO in the demo and then forget about it. At that point, we've done our bit, and it's up to the PO to choose when to ship. Normally, that would be ASAP, but often, for a number of reasons, it isn't. There's nothing wrong with that approach, but as you mention, it limits the impact of the feedback loop, which is a negative point.

Another reason we go through this "beta" testing phase (or "eating our own dog food" because internally, we'd pick up the product and use it for a set period before any of our clients did) was because our teams were quite immature and quality was lacking. It was critical for us to pick up any issues ourselves before shipping to a client. In a mature technical environment, this shouldn't be the case, but needs must. It's not ideal, but I wouldn't say it's "anti-agile" either.

Agile isn't black and white. Some things work for teams and others don't. Take what you can from it, but don't fit a square peg into a round hole just to say you're being "agile" - that's more likely to result anti-patterns than not.