How do large organizations, such as Amazon, Netflix or Spotify organize 24/7 support. These companies are known to have agile development models, which include "You build it, you run it". That means, there is no operations department which takes care of the application / feature once it is available to the end user. The team itself does maintenance and support. I know about all the advantages of this method, but I see some problems in terms of execution:
- Agile teams are designed to be quite small, so there is just a limited number of people who can deal with an incident. These people are usually not available 24/7 as they are on vacation, sick or simply not reachable.
- Theoretically, every developer needs access to production, meaning deployment, databases, filesystems etc. Are all these permissions just granted and it is only about trust?
- Who must be involved to do a fix, rollback etc.?
How is incident management done in a large scale Scrum environment?