13

I'm a software engineer working in a regulated industry. Releasing our product involves significant work which is required to evaluate risks and potentially submit a filling to the regulator.

As this work can't be automated to reduce the workload, we have settled into a yearly release cadence. Our product is also an integration system, meaning it is hard to effectively evaluate features in a short demo. As a result we are struggling to get effective feedback to sprints. What techniques can we use to more effectively improve the feedback we receive?

  • "Our product is also an integration system, meaning it is hard to effectively evaluate features in a short demo." - could you expand on this please? What do you mean by 'an integration system'? – Barnaby Golden Apr 27 at 17:19
  • The product interacts with various other third party systems, and they are the primary interface. In its expected use, there is the no expectation for anyone to interact directly with the system. – user1937198 Apr 27 at 17:45
  • I have seen some teams go to the extent of creating a fake front-end on their product just so they can demo it to stakeholders. This only makes sense if the value of the feedback from your stakeholders exceeds the time/effort put into building the fake front-end though. – Barnaby Golden Apr 27 at 21:21
  • 1
    Does the product include hardware (like an embedded system)? Does functional safety play a role? – Peter Mortensen Apr 27 at 23:23
  • @PeterMortensen It doesn't include hardware, but functional safety requirements are a big part of the problem. – user1937198 Apr 28 at 8:48
17

After having worked in a few different regulated contexts, I find it difficult to believe that you can't automate at least portions of the work needed to satisfy your compliance requirements. In my experiences, the claim that you can't is based on reading too much into the regulations and standards and relying on common misinterpretations rather than actual definitions or intents.

That said, depending on what happens as part of a release process, perhaps an annual cadence is appropriate with respect to the cost of the release process and how often your customers and users are able to accept a new version of the system.

My suggestion is to create a demonstration or sandbox environment that is updated much more frequently than yearly. It should be updated to track against proposed changes such that the version of all components on it when the release process starts is the same set of versions that are going through the release process. Then, use this environment in two ways. You can invite customers to use it for non-production data and to ensure that integrations and processes won't be affected by upcoming changes. Your sales, marketing, and support teams can also use it to make sure that their material is up-to-date and they understand what they are selling and supporting well before it goes live. Since these people also know the users, they can offer feedback on how customers and users may respond to changes.

The frequency of updates may still differ from your Sprint cadence but should be related. Your Sprint cadence should be designed to align your development teams with the business and reduce risk. The same rules of Scrum would apply - each Sprint would have a fully working and integrated system and the Product Owner would make the decision to release or not. In this case, release would be to the sandbox environment.

| improve this answer | |
  • 5
    I would say it's less that we can't automate any of it, and more that we've automated what the development team understands as automatable, and what is left is still enough of a burden. – user1937198 Apr 27 at 10:51
  • 1
    @user1937198, in a regulated environment, some things just have to become part of the daily routine (e.g. traceability) and some times you need to go into a discussion with the notified body how you can ease the perceived burden while remaining compliant with the intent of the regulation. – Bart van Ingen Schenau Apr 29 at 13:14
6

I would propose setting up in-house a mini-site that mirrors all the (important) functionality of the live site.

This way you can Release & Test to your test site as often as you want. You can get continuous feedback from an internal testing team. You could even propose to your customer to review select interim releases and provide feedback.

Another advantage would be that you can create and test a comprehensive "roll-out" document so that your yearly release will have a well documented and tested document.

Each Sprint will consist of an entire internal cycle; coding, testing the release document, testing the new features, and feedback.

About 15 years ago I helped launch a nationwide Telcom Carrier and we did just that: Create a miniature telcom site with all the same functionally and we would tweak & test the miniature site using predefined documented procedures before upgrading the live site using the same document.

| improve this answer | |
  • We would be able to replicate one site, though as we deploy to multiple prod sites with different third party systems, this would miss many of the important issues. Still it would be an improvement. I will look into it. – user1937198 Apr 27 at 10:28
1

Even though you have an annual release cadence, you would still have sprint deliverables for your scrum team. Your Product Manager would have defined these sprint deliverables at the start of your PI.

Use these sprint deliverables as a checklist (a fully set up functionality is not required for a demo). To improve your feedback loop, set up a sprint deliverable demo at the end of each sprint. Invite the Product team to the demo. Since at the end of each sprint, you would be promoting your code to the staging environment, use that for your demo. Even if your feature is purely technical with no UI - still showcase it and talk about it. If your Product Manager is non-technical, you may need to explain the business aspects of the deployed code. The PM will have questions - and this will act as your feedback conduit. But make sure to stick to the sprint deliverables checklist. Hope this helps.

| improve this answer | |
  • 1
    Hi Aayush - as you mentioned PI (as program increment) and Product Manager, I believe you're implying the OP is working under a scaled agile framework, would that be the case? – Tiago Cardoso Apr 27 at 7:30
  • Yes, you are correct. The question mentioned that they had to "settle into a yearly cadence" - looks like they had to do it even though they wanted a faster release cycle. Since the OP does use sprints, having regular checkins after each sprint can be considered as a possibility. This is where a defined sprint deliverable checklist comes into the picture. – Aayush Jain Apr 27 at 8:11
  • Due to the nature of our product, we don't deploy to a prod environment under our control, but release it to third parties. As such, most staging environment style deployment happens after release, and we can only do limited testing without full integration testing with external dependencies before release. – user1937198 Apr 27 at 10:03
  • That feedback the product team is currently able to provide is limited, even when presented with the feature, as it's most echoing back what they passed to the Dev team for implementing. This is our problem. (And yes our product has nothing you directly interact with, all interaction happens through third party products) – user1937198 Apr 27 at 10:08
0

As this work can't be automated

Factually wrong.

I have implemented automation and continuous deployment in United States Federal Government. Which is possibly the most regulated environment ever. What is your excuse? - Jez Humble

When DevOps Meets Regulation: Integrating 'Continuous' with 'Government'

| improve this answer | |
  • 3
    Hi, I believe your answer would fit better as a comment. To be fair, I agree with you - automation may be challenging but should be possible - but the point is that your answer, as it stands, doesn't seem to be addressing OP problem. – Tiago Cardoso Apr 27 at 7:32
  • 2
    Unfortunately the platform concept doesn't apply in our case, as we are dealing with a single regulation, and our product is a monolith, where the platform code is what changes most often. As for the autogeneration, I'm not clear how the tool handles integration risks coming from the interaction of multiple components. – user1937198 Apr 27 at 9:54
  • 3
    I agree with Tiago ... you make a very interesting comment about a very useful concept, then tell us nothing (here) about what and how you did it! You only say(!) ... "what's your excuse?" (Huh?) Please revise your post to add some details! – Mike Robinson Apr 27 at 13:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.