Evaluating risk and the limits of qualitative scoring¶
Turning scenarios into a level¶
With scales defined, evaluation is the act of placing each scenario on the matrix: combine its severity (from the strategic scenario) and its likelihood (from the operational scenario) and read off a risk level. Scaldimet's line-halt path, serious-to-critical in severity and high in likelihood, evaluates as an intolerable risk. This provides the risk level used for the treatment decision: a defensible statement that this specific risk, to this business value, by this attacker, is at this level and must be treated. This Lesson does the evaluation, settles a distinction the course has deferred twice, and then examines what that qualitative result can and cannot tell you.
Inherent versus residual, in its operational reading¶
Lesson 1.3 introduced, and Lesson 6.2 revisited, the difference between inherent and residual risk.
Inherent risk is the risk before the treatment you are considering. Residual risk is what remains after that treatment.
There are two ways to interpret ‘before treatment’. The formal approach starts from a theoretical position with no controls at all. That can be useful for comparison, but it does not describe Scaldimet's real situation: its socle is already in place.
For this course, we therefore use the operational reading. We treat the existing socle as the starting point, then assess what additional controls are needed for the specific risk.
The limits of qualitative scoring¶
The matrix you just built is qualitative, and qualitative scoring has real, known limits that a CISO states out loud rather than hiding behind colours.
- The labels are ordinal, not arithmetic. "High" is not a number. Multiplying a likelihood of 3 by a severity of 4 to get "12" is mathematically meaningless: the levels are an order, not quantities, and arithmetic on them is not valid even though tools invite it.
- Range compression. Forcing wildly different risks into four buckets hides large differences. Two "serious" risks might differ by a factor of fifty in real expected loss, and the matrix shows them as the same colour.
- False agreement. A shared word can hide unshared meaning: two people both say "medium" and believe they agree, while picturing very different things.
- No aggregation and no budget link. A list of "high" risks does not tell you the total exposure in money, nor how much to spend, nor whether a control's cost is justified by the risk it removes.
None of this means the matrix is useless. For triage, communication and prioritisation it is genuinely valuable, and for many decisions it is enough. But when a decision turns on how much (is a one-million-euro control justified? what is our total cyber exposure? which of two serious risks is actually larger?), qualitative scoring cannot answer, and treating it as if it could answer those questions would go beyond what qualitative scoring can support. That is precisely the gap risk quantification methods fill: expressing risk in ranges of money and probability so it can be aggregated, compared and weighed against cost. This Lesson is the hand-off: use the matrix for what it is good at, and know exactly where it runs out.
Going further: a second impact axis (optional)
This course builds the matrix on the two dimensions that boards, auditors and frameworks expect today: severity (the harm an incident does to Scaldimet) against likelihood. That is deliberate as it is the common language of risk, and fluency in it comes first.
It is worth knowing what that single impact axis leaves out. By aggregating every consequence into harm to the organisation, it can hide the harm the organisation causes to others: customers and suppliers, interdependent critical infrastructure, public health, the environment, even national sovereignty. An incident can be minor for the company (absorbable, perhaps insured) yet severe for society, and insurance shrinks the perceived internal impact while leaving those external costs untouched. NIS2 already pushes in this direction: CyFun's measure GV.OC-04.1 asks you to identify and prioritise the services that external stakeholders depend on, and to feed that into the risk assessment.
One response, developed in this article, keeps a two-axis matrix but redraws the axes as internal impact against external impact, and encodes probability and level of control as the size and colour of each plotted risk. Two scenarios that look identical on a conventional grid — say, an IT-only outage versus an OT compromise that delays medical-oxygen deliveries — then separate into different zones that call for different governance decisions.
Scaldimet's matrix in this course stays with the conventional model, because that is what is common and expected. Treat the external dimension as a direction to explore as your programme matures, not as a requirement of this lesson.
Key takeaway
Evaluation places each scenario on the matrix by combining its severity and likelihood into a risk level; Scaldimet's line-halt path evaluates as intolerable. The course uses the operational reading of inherent versus residual risk: inherent is the risk with the socle already in place (not the purist "no controls at all"), and residual is what remains after the bespoke treatment of Chapter 4.6. Qualitative scoring has real limits: its labels are ordinal not arithmetic (multiplying them is meaningless), it compresses ranges, hides unshared meaning behind shared words, and cannot aggregate risk or link it to budget. It is excellent for triage and communication, but when a decision turns on how much, you need the quantification method.