There can be no general answer to this. Multinational or public companies do vary greatly. Software developed by such companies varies even more in size, type, purpose, usage, life expectancy etc. In general, the bigger, longer used and maintained, more complex the software is, and the more people are involved (at the same time and/or over the long term), and the more regulated the domain is, the more documentation it requires.
The new Scrum Master balked at any documentation
... which sounds a bit overzealous to me. All these people can most likely quote the relevant parts of the Agile Manifesto by heart:
[...] we have come to value: [...] Working software over comprehensive documentation [...]
They often forget to quote this though:
That is, while there is value in the items on the right, we value the items on
the left more.
You say
we seem to have settled upon documenting any aspect which has the risk
of impacting another departments dependency.
(In short - if they break something that is not theirs, they have some
documentation to back up the process, intent and reasoning).
which to me sounds a pragmatic approach, the same as I would follow. There is indeed no value in producing documentation just for the sake of it, but whenever stakeholders request documentation, and/or there is a problem identified stemming from inadequate or out of date documentation, the Scrum team must address it. Of course, face to face conversation is more effective on many levels than written documentation; however, it lasts only as long as participants' memories, and it is not feasible in some cases (e.g. distributed teams). Whenever we need more lasting and/or more transferable traces of things, we need to document them.
The primary rebuttal against in-depth documentation was that
- It needs factored in as a User Story and given a points value (which I accept)
- It ruins the flow of the developers work (also accepted)
Well, one may also say that testing ruins the flow of the developers work. As well as lunch breaks ;-P
Note that whatever one is not accustomed to, tends to break the flow indeed. Once we get used to it and gain experience, it will become part of the flow.
- Documentation is out of date within a week because of code changes/imrpovements etc (which I do not wholly accept)
It is indeed a risk, but it depends entirely on what is documented. Document not the low level details of the code which may change by next week, but the higher level aspects - architecture, design decisions, patterns used, interfaces etc. - which don't tend to change that often and/or are not directly present within the code, but are crucial for e.g. an outsider or a new team member to quickly get an overall picture of the system.
- Documentation is contained within the code (Which I have rejected outright as worst practice)
Depends on what this means. If this means autogenerating documentation from code comments like Javadoc, it is fine although in my experience mostly of limited use (except for public interfaces). If it means embedding explanatory comments within the code, IMHO it is almost always a Bad Idea (except for rare and limited cases like implementing a specific algorithm, or performance tuning changes). If it means writing readable, self-explanatory, fluent code, I fully agree. IMHO well written, clean code rarely needs any comments.
So, to sum up with a few rules of thumb:
- Talk to your PO, stakeholders and the Scrum team, to find out who needs what kind of documentation and why (and whether their reason is good enough).
- (during e.g. reviews and retrospectives) Identify any problems caused by missing or inadequate documentation.
- If there is a justified need, do the minimum necessary work to fulfill that need. (Taking into account the expected lifetime of the need / artifacts - you may want to optimize for the long term, not the short term. E.g. writing a script to autogenerate a document from config files is obviously costlier than copying it manually once, but if this document needs to be updated every week for the next 5+ years, the script will quickly pay for itself.)
- Always strive to avoid duplicating info as much as possible. Ideally every piece of information has a single source, so if it needs an update, you can do it once at the source. It may need to be replicated / copied to other places, but strive to solve this via automation instead of manual work.
- If (and as much as) you produce documentation, strive to make it in the most accessible, easiest to maintain format (unless of course it is confidential). In my experience, this means a wiki like Confluence you mentioned. It can import Word docs, it keeps a history of all changes, and it can even be autoupdated using scripts.
- Also - as processes and needs change - look for out of date, out of use documentation and if you find such stuff, eliminate it.
