Measuring the Effectiveness of Risk Begins with Assessing Risk

In a previous blog, I discussed the importance of moving risk assessment beyond a heat map to ensure that leadership has enough detailed information to inform decision-making.  In today's blog, I will discuss the importance of this underlying risk assessment methodology for determining an accurate picture of residual risk.

Enterprise risk management (ERM) establishes processes to identify, assess, mitigate, and monitor risks across an organization. While each component of an ERM program is essential, the ultimate goal of an ERM program is to reduce an organization's exposure to risk. If the organization is aware of risks and their impacts but unaware of how successful the organization is at reducing risk exposure, the ERM program is not serving its intended purpose.

Many ERM programs, however, are challenged in progressing beyond the risk identification and assessment phase of an ERM program cycle, often being in the position of continually assessing risk and updating risk registries, and not focusing adequate attention on risk mitigation. Additionally, organizations may conflate the assessment of risk and the assessment of risk mitigation in the risk identification and assessment phase of its ERM program cycle. This might involve risk partners who may not have the appropriate degree of knowledge about risk mitigation strategies to inconsistently or inaccurately weigh in on the effectiveness of a mitigation strategy and unknowingly over- or under-assess the impact the strategy has in reducing the assessed risk exposure. In addition, the incorporation of risk mitigation assessment into the risk assessment process often involves a superficial assessment of the impact of mitigation strategies – assigning an overall value of mitigation by which the risk exposure calculation is reduced rather than comprehensively evaluating how impactful a mitigation strategy is across all categories of assessed risk (likelihood, consequence, etc.).  This approach can skew an organization's true understanding of its risk exposure.

A comingled risk assessment and mitigation assessment may also lead organizations to make decisions about risk acceptance, avoidance, transfer, or reduction by assuming the risk assessment provides a comprehensive understanding of risk exposure and the impact of risk mitigation strategies without having methodically assessed the effectiveness of mitigation on reducing the organization's risk exposure. A dedicated accounting of mitigation strategies layered with knowledge of the effectiveness of those mitigation strategies is integral to an organization's determination of risk tolerance.

To create a baseline against which the effectiveness of mitigation strategies can be evaluated, the University of Massachusetts (UMass) began assessing the inherent exposure of risks during the risk identification and assessment process. This means that during our risk assessment process, we intentionally do not account for mitigation strategies being implemented to reduce our risk exposure. This approach to risk assessment allows us to gauge our foundational risk exposure using established risk assessment tools and prevents us from inconsistently or inaccurately accounting for the effectiveness of mitigation strategies while assessing risk exposure.

By assessing inherent risk, UMass can conduct a separate but correlated assessment of mitigation strategies. The dedicated mitigation assessment process increases transparency by documenting all ongoing risk mitigation strategies and sets the stage for informed risk tolerance discussions by providing a detailed evaluation of the effectiveness of these strategies in reducing risk exposure.

Leveraging the concepts of our risk assessment process, the UMass ERM program created a mitigation assessment methodology that enables the university to document all ongoing risk mitigation strategies and evaluate their effectiveness through three different lenses: individual effectiveness, comparative effectiveness, and aggregate effectiveness.  The three assessments are completed through one evaluation process using a newly developed mitigation assessment tool called MATRX. Assessments are correlated to, but not a replacement of, the risk assessment process and the Inherent Risk Score.

The mitigation assessment results do not define whether the university is satisfied with the residual risk but inform leadership as they make determinations about accepting risk, further reducing risk, transferring residual risk, or altering activities to avoid risk. In addition, the mitigation assessment assists in identifying opportunities that risk presents by providing a comprehensive perspective of the university's risk-related activities.