Using Risk Assessment to Support Decision Making

"Construction Signs" by jphilipg is licensed under CC BY 2.0

An effective and mature risk governance program drives better decision making in all directions of an organization: up to leadership and the board, down to individual contributors and laterally to all lines of business. Risk-aware decision making, regardless of the domain (e.g., finance, technology, enterprise, cyber), is the cornerstone of effective resource management at any organization.

COBIT® 5 for Risk defines a risk assessment as “[T]he process used to identify and qualify or quantify risk and its potential effects,” describing the identification, scoping, analysis and control evaluation. Successful organizations integrate the entire risk management life cycle process with business decision making, but how do they do so? First, the organization must know what a decision is and how decisions drive risk assessment activities—not the other way around. After this is understood, the rest of the pieces fall into place.

What Is a Decision?

Without a decision, a risk assessment is, at best, busywork. At worst, it produces an unfocused, time-intensive effort that does not help leaders achieve their objectives. Information risk professionals operate in a fast, ever-changing and often chaotic environment, and there is not enough time to assess every risk, every vulnerability and every asset. Identifying the underlying decision driving the risk assessment ensures that the activity is meaningful, ties to business objectives and is not just busywork.

The idea that risk analysis helps decision making by reducing uncertainty is as old as probabilistic thinking itself. The concept was formalized by Ron A. Howard, a decision science professor at Stanford University (California, USA), in his influential 1963 paper, Decision Analysis: Applied Decision Theory.1 He formalized and defined the components of a decision, all of which can be used to focus risk assessment activities.

Components of a Decision

Howard identifies 3 components of a decision: choice, information and preference (figure 1).2 Together they are the foundation of decision-making; without all 3, a decision cannot be made. The decision maker uses logic to identify and evaluate the components individually and together, leading to a conclusion.

Figure 1—The Components of a Decision

Once the risk analyst understands the components and how they work together, it is easy to see how they support a risk decision:

• Choice—This describes what the decision maker can do. There must be multiple courses of action possible for a decision to be made. With only one course of action, there is no decision.

• Preference—The decision maker must have a preference or inclination for a desired outcome. For example, in information risk, the decision maker often prefers to optimize user functionality, effective security, efficient resource allocation (i.e., money, time, people) or some combination of these options. Understanding the requestor's preferences is a valuable exercise to help scope a risk assessment. The decision maker should be able to articulate what the requestor wants to achieve as an outcome of the decision.

• Information—Information is an equal part of the trio when making a decision, and information is also an integral part of any risk assessment. When making a decision—and by extension, assessing risk—information is available from a variety of sources.

Framing a Risk Assessment as Decision Support

If any of these components are missing, there is no decision to be made and, by extension, a risk assessment will be an exercise in frustration that will not yield valuable results. If the risk analyst starts a risk assessment by identifying the choice, preference and information, the assessment will be easier to focus and scope. Alternately, one may conclude that a risk assessment is not necessary or a different methodology may be more appropriate.

ISACA’s Risk IT Framework, 2nd Edition describes 3 high-level steps in the risk assessment process:

1. Risk identification

2. Risk analysis

3. Evaluating the business impact(s) of the identified risk

Integrating the decision-making process into risk assessment steps requires the analyst to ask questions to understand the full scope of the decision before and during the risk identification phase. This provides the opportunity to align assessment activities with the organization’s strategic objectives.

Figure 2 provides a simple matrix that illustrates this.

Figure 2—Understanding the Decision Before and During Risk Identification

Real-World Examples

Here are 3 common examples of poorly scoped risk assessment requests and tips for the risk analyst to clarify the decision and determine if risk analysis is the right tool.

Risk Assessment Request 1
“An employee on the development team keeps unjoining his computer from the Active Directory Domain Service (AD DS) to avoid system updates and required device management. Can you perform a risk assessment on this so we can force the employee to stop doing it?"

What Is Missing?
Choice. There is not a clearly articulated choice or alternatives. The requestor is presenting only one choice: forcing an employee to do something specific. In other words, the requestor does not need help in deciding what to do.

What Is an Alternative Approach?
Management or human resources (HR) action and escalation are most appropriate here, assuming there is a written policy for security circumvention and IT management software uninstalls. A risk assessment would be appropriate here if there were a choice to be made such as, “Should the enterprise let users circumvent endpoint management, and, if so, what is the risk?” A risk assessment would help management weigh the risk and benefits and make a decision.

Risk Assessment Request 2
“We are evaluating 2 different antivirus vendors to replace our existing solution, and we need a risk assessment to help us decide.”

What Is Missing?
Preference. The decision maker has not expressed the desired outcome from the decision. Are there security concerns, cost savings or usability issues with the current solution? Without a clearly defined preference, the assessment will be unfocused and could analyze the wrong risk.

What Is an Alternative Approach?
Interviewing leadership and asking why they are considering switching vendors and what information needs to be included in the risk assessment will aid the decision. A requirements comparison matrix would be a good first step, comparing product features and potential security issues. After developing a list of gaps each product has, a risk assessment may be the best path forward, but it needs to be scoped. For example, a potential gap might be, "Product Y is cheaper than Product Z, but it is missing these 3 security features. What additional risk exposure would Product Y introduce to the organization?" 

Risk Assessment Request 3
“I would like you to assess black swan cyberevents.”

What Is Missing?
Information. According to Nassim Taleb, who coined and popularized the term in the modern business context, a black swan event is an “outlier, as it lies outside the realm of regular expectations.”3 Only a true clairvoyant can look into the future and predict events that are unknowable today.

What Is an Alternative Approach?
The decision maker may be misunderstanding the term “black swan.” It would be useful to ask, “Do you mean high-impact, low-probability events?” If that is the case, a series of risk assessments can be performed to identify control weaknesses that affect business resilience.

Conclusion

Risk assessments are an excellent tool to reduce uncertainty when making decisions, but they are often misapplied when not directly connected to an overall decision-making process. The failure to frame a risk assessment as decision support, supported by the 3 decision components, decouples the analysis effort from business objectives. Time is wasted by performing assessments when there is not a decision to be made, when there is a lack of complete information or when there is no understanding of the preference of the individuals responsible for the decisions. Having clear, complete information and understanding the motivations and options behind a decision help frame the assessment in a meaningful manner.

This understanding will help develop a response the next time someone drops off a 170-page vulnerability scan report and asks for a risk assessment on it.

Endnotes

1 Howard, R. A.; “Decision Analysis: Applied Decision Theory,“ Proceedings of the Fourth International Conference on Operational Research,” 1966
2 Edwards, W.; R. F. Miles, Jr.; D. Von Winterfeldt (eds.).; Advances in Decision Analysis: From Foundations to Applications, Cambridge University Press, USA, 2007
3 Taleb, N. N.; “The Black Swan: The Impact of the Highly Improbable,” The New York Times, 22 April 2007

---

This article was previously published by ISACA on April 12, 2021. ©2021 ISACA. All rights reserved. Reposted with permission.

Bring Uncertainty Back

We need to bring uncertainty back to risk measurements.

Suppose I ask you to measure the wingspan of a Boeing 747. Right now, wherever you may be, with the knowledge and tools you have on hand. You may say this isn’t possible, but Doug Hubbard has taught us that anything can be measured, once you understand what measurement is. With that mental hurdle out of the way, you can now measure the wingspan of a Boeing 747. 

There are two different approaches to this in modern business.

Option 1:Think about the size of a passenger jet and say, “Big.” 

Technically, this answers my question. There’s a problem with this answer, however - it’s neither precise nor accurate. In everyday language, the words precise and accurate are used interchangeably. In areas of science where measurements are frequently used, they mean different things. Accurate means the measure is correct while precise means the measure is consistent with other measurements.

The word “big” is an adjective to describe an attribute of something, but without context or a frame of reference to make a comparison, it’s virtually meaningless. Furthermore, using an adjective in place of a measurement is a little dishonest. It’s true that we don’t know the exactwingspan of a 747. Besides, wingspans vary by model. However, we chose a word, “big,” that conveys precision, accuracy, and exactness, but is not any of those. If that wasn’t bad enough, we’ve completely obfuscated our level of uncertainty about our ability to estimate the wingspan of a 747.

Option 2:What Would Fermi Do?

Thinkers like Enrico Fermi and Doug Hubbard approach the problem differently. They – just like us – probably don’t know the wingspan of a 747 off the top of their heads. Just like Fermi estimated the number of piano tuners in Chicago simply by thinking through and decomposing the problem, we can do the same.

• I’ve seen a 747 and even flown on one several times, so I have some frame of reference.

• I'm 6'2," and I know a 747 is larger than me

• A football playing field is 100 yards (300 feet), and I'm sure a 747's wingspan is smaller than a football field

• My first estimate is between 6’2” and 300 feet – let’s improve this

• I know what a Chevy Suburban looks like – they are 18 feet long. How many Suburbans, front to back, would equal a 747? Maybe…. 7 is a safe number. That’s 126 feet.

• I’m going to say that the wingspan of a 747 is between 126’ and 300’.

• Am I 90% sure that the actual number falls into this range (aka confidence interval)? Let me think through my estimations again. Yes, I am sure.

Let’s check our estimation against Google.

It’s a good measurement.

Two remarkable things happened here. Using the same of data as “big” – but a different mental model - we made a measurement that is accurate. Second, we expressed our uncertaintyabout the measurement - mainly, we introduced error bars.

One missing data point is whether or not the level of precisionis adequate. To answer this, we need to know why I asked for the measurement. Is it to win a pub trivia game or to build an airplane hangar to store a 747? Our minds are instruments of measurement. We may not be as accurate as a tape measure, which is not as accurate as a laser distance measurer, which is not as accurate as an interferometer. All instruments of measurement of have error bars. When determining the level of precision needed in a measurement, we always need to consider the cost of obtaining new information, if it’s relevant and if we need additional uncertainty reduction to make a decision.

If this seems like a nice story to you, but one that’s not too relevant - think again. 

Using adjectives like “red” or “high” in the place of real measurements of risk components (e.g., probability, impact, control strength) are neither precise noraccurate. Even worse, uncertainty is obscured behind the curtain of an adjective feelsexact, but is not. The reader has no idea if this was a precise measurement – using a mixture of historical data, internal data and many calibrated subject matter experts – or if it was made by a guy named Bob sitting in an office, pondering the question for a few seconds and then saying, “That feels High.”

Managing risk is one of the most important things a business can do to stay in business. It’s time to bring uncertainty back to risk measurements. It’s the honest thing to do.

How Many Lottery Tickets Should I Buy?

When lottery jackpots are at record highs, as they are this week at $1.6 billion, I’m usually asked by friends, family, and colleagues for the same advice – should I buy a lottery ticket, and if yes, how many should I buy?

Being trained in economics and a risk manager by trade, one would expect me to say that lottery tickets are a waste of time, money – or, maybe a rant on how the lottery is a regressive tax on the poor. Not this economist/risk manager. I’ve spent a good deal of time studying odds at craps, horse races, and roulette tables in Vegas and the answer lies in understanding a little bit of probability theory.

First, look at this problem in terms of the expected value of buying a lottery ticket, which is based on the probability of winning and how much you could win. The expected value of the Mega Millions drawing on Tuesday, October 23rd, is $5.53, for a $2 ticket. It’s quite rare for the expected value of a game of chance to exceed the price of entry. Economically speaking, you should play this lottery on Tuesday.

The question remains, – how many tickets?

To answer this question, think of the problem this way: how much money do I need to spend to increase my odds? If you don’t play the lottery, the chance of winning is near-zero*. Buying one $2 ticket increases your odds from near-zero to 1 in 302 million. What a deal! You can increase your odds of winning by such a colossal amount for only $2, and the expected value exceeds the price of a ticket! Here’s the trick – the second, third, tenth, hundredth ticket barely increases your odds over 1 in 302 million. You could buy enough tickets to demonstrably increase your odds, but at that point, you would have to buy so many tickets, the expected value would be below $2.

The answer: one ticket. Just buy one. One is a good balance between risk and reward.

Not coincidentally, knowing how to calculate expected value is a superpower for risk managers when trying to optimize investments and expenditures. 

(*Near zero, not zero because it’s possible you can find a winning lottery ticket on the ground, in a jacket at Goodwill, etc. It’s happened.

GDPR, Ragnarok Online and Decision Analysis

Did you ever think you would read about GDPR, risk analysis and a 16-year old MMORPG in the same place? You can stop dreaming, because here it is!

First: GDPR, tl;dr

General Data Protection Regulation(GDPR) is a massive overhaul of privacy on the Internet that applies to all European Union (EU) persons. Any company outside of the EU needs to comply with GDPR if they store personal data of any EU person. On May 25, 2018 GDPR becomes enforceable, and many companies — including US-based companies with data on EU persons — have been making changes to become compliant. (This explains why you have been receiving so many privacy notice updates lately.)

The cost of GDPR compliance is not cheap or easy, and the price of non-compliance can involve hefty fines and litigation. Every company that stores personal data has most likely spent the last two years performing analysis on whether GDPR applies to them, and if so, what the cost of compliance is.

What Happened with Ragnarok Online?

This leads to a story that took the gaming world by surprise: On April 25, 2018, the online gaming company Gravity Interactive announced they are shutting down all games and services in the EU, effective May 25th– the day GDPR takes effect. All EU-based IP-addresses will be blocked. Understandably, there’s an uproar, especially from EU-based players of Ragnarok Online, one of Gravity Interactive’s most popular games. Gravity Interactive has operated EU-based servers for 14 years and to many, the sudden decision to pull out of the market entirely seems unfair and unexpected. It’s understandable that people would be upset. The company has been the subject of much derision over the decision. But clearly there’s more to the story disappointed gamers.

This is an interesting case study because it illustrates several points in the decision-making process:

• How a quantitative risk analysis can be used to help with strategic business decisions;

• Every sound risk analysis starts with a clearly defined question; and

• Avoidance can be an appropriate way to eliminate risk exposure.

Let’s analyse this problem with, first, forming a question that articulates the decision being made, then identifying possible choices, and last, estimating costs for each choice.

The Question

Every company faces strategic decisions. Sound, informed, decision making requires information about benefits and risk exposure. Risk analysis always needs to answer a question, in other words, a decision that someone needs to make. In our case, the decision for Gravity Interactive is whether to invest the time, money and resources to achieve GDPR compliance. GDPR introduces data privacy, security, compliance and legal requirements that are new for most US-based companies, therefore the cost of compliance can be significant. Most companies, US-based or otherwise, spent the last two years performing analyses of GDPR compliance: the cost of complying with the regulations from many perspectives, including technological. Companies can comply with GDPR, ignore GDPR or pull out of the EU market and analysis will help find the best course of action to take.

Decision: should we invest in GDPR compliance?

The Decisions

A company faces three options when deciding whether to invest in GDPR compliance. First, they need to price out the cost of compliance. This can be an upfront cost, as well as ongoing. Compliance involves funding and starting projects to align people, processes and technologies with applicable regulations. The analysis in this area would include a survey of all changes the company needs to make, estimating the cost, and performing a cost-benefit analysis.

The next option is to ignore compliance. This is where risk analyses are most useful to help a company. Ignoring compliance is always an option — and as risky as it may sound, many companies choose to ignore laws and regulations; some unintentionally, some wilfully. This happens more often than most of us should be comfortable with. We typically find out about this when companies are caught, regulators levy penalties and the story is splashed all over the news. At the same time, many companies successfully fly under the regulatory radar for years without being caught. A risk analysis on compliance risk would involve the length of time it would take for regulatory action to take place (if it takes place), what the regulators would force the company to do and, penance projects to achieve compliance.

Lastly, they can choose to withdraw from the market altogether. In the risk management world, we call this risk avoidance.This is the elimination of risk by choosing not to pursue a potentially risk generating activity. In this case, a company can avoid non-compliance risk by exiting the EU market.

The box below contains sample output of these different analyses. I obviously don’t know any of the costs or risk associated with Gravity Interactive’s decision, so I created a sample Company A with example values.

Company A: Projected Costs of GDPR Compliance Options

It’s clear that the company should not ignore compliance. This activity creates significant risk exposure. It’s likely they would have to pay fines, face litigation and be forced to make changes to comply with GDPR anyway.

Based on the two remaining options — comply with GDPR or exit the market, we can perform a cost/benefit analysis of current EU market share, projected EU growth and balance it against the cost of GDPR compliance. Based on my analysis of Company A, it should exit the EU market.

If I were responsible for risk management at either Company A or Gravity Interactive, I would want to perform additional risk analyses on the current state of data privacy and security. If compliance with GDPR is too costly, does the company currently comply with US privacy and security regulations?

In the case of Gravity Interactive, the company clearly decided that forgoing a portion of its customer base, losing the loyalty of its EU fans and risking the ire of gamers worldwide was worth the potential costs of compliance or non-compliance with GDPR. Or in short, to avoid being stuck between Ragnarok and a hard place.