2

When an auditor performs an inspection on a process area, the auditor will use interviews and documented evidence to make a judgment about how well the process is being performed. Inspecting a risk management capability should not be any different, but I think it is.

Auditing a risk management process on a project I would expect to see:

  • Risk management plan
  • Risk management process and procedures
  • Evidence of risk meetings on some cadence
  • Evidence of escalated risks
  • Evidence of risk analyses and reanalyses
  • Evidence of various mitigation plans
  • Risk log

But the absence of all of this evidence does not necessarily indicate with a high degree of validity that risks are not being managed well and vice versa. In our daily lives, we manage all kinds of risks without any documentation at all and, if you're alive reading this, you're doing a reasonable job.

So my question is: what truly indicates a high-performing risk management capability?

3 Answers 3

3

There are two standards to which risk management should be compared. First, the compliance standard.

  1. Do you have the documentation? Can you defend that you have expended effort? By this standard a "risk management softball team" is equal in vale to any other risk management efforts. I have little respect for this, but it is what is normally meant by a risk management program. Management wants to know that someone has expended effort (and generally wants those who expend that effort to remain silent, or failing silence to utter reassuring words). This effort will pass the muster with auditors and other compliance organs. After all, the Titanic was compliant with all safety standards.

  2. Return on Risk Investment. If you are really serious about risk, then you should be able to measure the reduction in risk (increase in opportunity). Naturally, this is a probabilistic measurement, and it demands some sophistication to articulate, but mature organizations are content when the risk management function makes concrete recommendations that

    a. Reduce the probability of risk/increase the probability of successfully exploiting opportunities

    b. Reduce the impact of realized risk/ increase the impact of exploited opportunities

    c. Transfer risk to competent external parties and demonstrate a cost savings by doing so

    d. Plan and document alternative strategies that give management the option of avoiding risk.

If you do that, you can measure the risk avoided, subtract the cost of the program and calculate the return on risk investment.

This is not for the faint of heart; it demands a serious investment, rather than a compliance eyewash. It demands that the organization commit to understanding risk, and take action based on empirical, albeit probabilistic, evidence. Sometimes this means making a decision on evidence rather than on bias or "gut" (which, in my experience, nearly all managers are reluctant to do). This is rare, but there are significant institutions that do this.

There is a third standard that I've discussed theoretically, but I don't think I have ever seen in practice. What % of capital is spent on investments, rather than on reactions? Every good manager has a list of investments/projects that will increase their value to the organization. Every manager and every CFO is forced to redirect some of that capital to less productive purposes to cover issues ("issue" is a risk that was not managed properly, and results in unexpected costs/damages). Risk cannot be eliminated, but if you study your institution and your competitors, you can reduce your risk, and keep unplanned spending within control. That gives you more capital available to invest in capital deepening, and reduces the frequency and impact of "one time charges" that appear on your balance sheet.

Simply sit down periodically and classify all your budget requests (not expenditures, requests) into three buckets

  • O&M that is inflexible; can't be reduced without further capital spending (this should be static)
  • Capital investment that will increase the value (reduce costs/add new services) These are long term planned costs with specified budgets
  • Unplanned remediation - money that had to be repurposed/special requests, etc. Money that you had to spend because something went wrong.

Unplanned remediation can be further divided into

  • Risk Management Failures - technical debt that you knew was accruing and failed to manage. People who quit, but for whom you didn't have replacements in the pipeline (succession plans). Every employee who leaves because the enterprise didn't provide enough training/appreciation/opportunity/career path/fairness/flexibility. Equipment that you failed to maintain, or failed to depreciate and acquire bench stock. Bugs and defects of which you were aware, but for which you failed to explore alternatives. Other issues that you were aware of, informed your management/responsible parties, but failed to be noisy enough or persistent enough to transfer the risk. The effect of attacks or disruptive actions by competitors or other adverse entities. Ransomware events. Lawsuits, EEO complaints. discrimination actions. Inability to take advantage of opportunities because your workforce isn't nimble, or T-Shaped, or is missing some critical skill, or lacks some other attribute that prevents you from acting in a timely fashion.) Failure to be aware of shifts in demand and supply. Other supply chain issues. Building your nuclear reaction in an area that is prone to Tsunami's in a predictable period, and failing to cover that. Accepting the cost cutting recommendations that result in halving the double hull on the Titanic. Ignoring the briefing that warned you about O rings.

  • True Force Majeure events. COVID. Sure, in theory we could have estimated the probability of a pandemic. I'm not going to fault you for failing to do so; the probability was so low.

If you do that exercise with honesty and integrity, the risk management failures bucket is the rough order magnitude of the budget for your risk management function. If you invested some of the unplanned spend money, could you have avoided one or more of those unplanned spends? You don't have to be completely successful - you just have to save the enterprise more money than you spend. Building a risk management program won't be magical; it will take time to build and integrated processes that avoid losses and exploit opportunities

2

The list that you have posted in the question would be a good starting point as necessary processes, but perhaps as you suggest it would be insufficient to demonstrate high performance.

What your list misses out is evidence of action to mitigate risks. Mitigation plans and re-analysis of risks are great, but they don't guarantee that the risks are being treated. Many projects pay lip service to risk by having all of the processes in place, yet they still encounter situations when the risks crystallise into issues. Active mitigation of the risks should minimise the number of risks that actually become issues, so I suggest that a relatively high number of risks compared to issues is a valid measure, and if that is reflected in a high number of mitigation actions - especially if targeted against the high level risks (high impact and / or high probability) - then you are managing your risks effectively. Conversely, too high a level of activity to mitigate low level risks (low impact and / or low probability) may indicate too much focus on the wrong risks, but I don't have any metrics to offer on this. In addition, a regular flow of new risks into the register, and old risks being closed within the register, should indicate that you are actively managing risks. You don't want the same old risks just sitting there for ever, never moving, and never being managed.

If your organisation keeps track of risks across multiple projects, you may be able to see patterns emerging to allow you to compare and identify well-managed projects (from a risk perspective) versus those that are less well managed. Every organisation is different, so the metrics for this will almost certainly be dependent on the organisation and its risk appetite.

1
  • Superb answer. (which is all that needs to be said, but the comment needs to padded to exceed the brevity penalty)
    – MCW
    Oct 23, 2021 at 18:14
1

It's not about having a high-performance capacity, but a capacity to give potential customers or clients the confidence that the entity can meet requirements and deliver the expected products and services.

As someone who has conducted supplier assessments, one of the first things that I do is open-source research. How long has the company been in business (including through various mergers and acquisitions)? How long have they been offering the product or service that is potentially being acquired? What are the major competitors or alternatives? Has the business been involved in any legal or regulatory actions, especially regarding this product or service? Who are the current clients of the business, especially of the product being looked at? Are there any case studies available? Are there third-party audits or assessments of the vendor's products and/or processes?

If the open-source information is positive and it makes sense, then the acquirer's risk management practices may dictate further audits or assessments that would require entering into agreements with the potential supplier. The end goal would be to require the potential supplier to give enough confidence that they can do what is being asked.

In the specific case of risk management, even if the potential supplier wasn't claiming conformance to a standard, I'd want to understand how they identify, analyze, treat, and monitor risks. It would be up for them to tell me their story and for me, as an auditor, to determine if their story and any evidence that they can use to back up their story, gives me confidence that they can do what is expected.

In the end, the acquirer of a product or service needs to carry out their risk management activities, perhaps even deciding that there is not enough confidence that a prospective supplier is able to provide the products or services desired.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.