Problems with Probability
© March 2007, Dr David Hillson
PMP FAPM
david@risk-doctor.com
- Risk
is defined in two dimensions: uncertainty and effect on
objectives. It is common to use the terms “probability”
and “impact” to describe these two dimensions, and assessing
the significance of any given risk means considering both.
It is relatively simple to assess effect on objectives,
since this merely requires defining the situation after
the risk has occurred, and then imagining what happens:
“If this risk occurred, what would the effect be?” Probability
is not so easy however. Risk practitioners and project
teams alike experience repeated difficulty in assessing
the probability that a given risk might occur. There are
a number of reasons for this.
- Language.
In English, different words are often used interchangeably
to describe the uncertainty dimension of a risk, such
as “probability”, “frequency”, “likelihood” or “chance”.
In fact these do not mean the same thing, and confusion
can arise if the terms are misused. For example “frequency”
describes how often an event or set of circumstances is
expected to occur based on previous experience, either
in a period of time (e.g. once per year) or in a number
of trials (e.g. seven times out of ten). So frequency
really applies to repeatable events. This is not the same
as “probability” which is a statistical term describing
how likely a single uncertain event or set of circumstances
is to occur. One solution is to use a more general term
such as “likelihood”, and recognise two variants called
“probability” (for single events) and “frequency” (for
repeatable events).
- Format.
The uncertainty dimension of a risk can be expressed in
several ways, including both numerical and textual formats,
such as: 35%, “once per month”, 2:7, “unlikely”, “one
in six times”, 10-4, “low probability”, 0.2, and so on.
Most people have problems interpreting different numerical
formats, and even the textual phrases can mean different
things. This problem can best be overcome by education,
as well as using a set of agreed definitions which everyone
understands.
-
Subjectivity. Assessment of probability requires forming
an opinion about a future event or set of circumstances
which have not yet happened. Different people will take
different views about the future, and there is no “single
right answer” since the future has not yet happened. Risk
probability cannot be measured, only estimated. Assessments
of the uncertain future are influenced by many factors,
including perceptual filters, motivational bias, cognitive
bias, or subconscious heuristics. The solution here is
to take a team-based approach, exploring different perspectives,
examining underlying assumptions, and reaching consensus
wherever possible. Sources of bias should also be understood
and corrected where possible.
Lack of data. Some risks have never been experienced
before, especially those relating to the unique aspects
of projects. In other cases, even though a risk might have
been encountered previously, there may be no record of its
existence due to absence of a learning mechanism (such as
a knowledge base or checklist). As a result there is no
body of evidence to assist in estimating the probability
of occurrence of these novel risks. Addressing these shortfalls
requires acknowledging that some areas lack relevant previous
experience, as well as implementing an effective lessons-to-be-learned
process (e.g. a post-project review).
All this matters for two reasons:
- Faulty probability assessment means risks
will be wrongly prioritised, leading to a failure to focus
on the most significant risks, selection of inappropriate
responses, inability to manage risks effectively, and
loss of confidence in the risk process.
- Sound assessment of risk probability improves
the understanding of each risk, allowing appropriate prioritisation,
better response selection, enhanced risk management effectiveness,
and more reliable achievement of project and business
objectives.
We need to understand the problems associated with assessing
probability, and take action to address the concerns,
by using appropriate language and formats, identifying
and managing sources of bias, learning lessons to improve
the effectiveness of the probability assessment process,
and monitoring risk management performance to determine
the accuracy of assessed risk probability.
|