Professor Norman Fenton’s invited video appearance at the
Annual Conference of the Society for Expert Witnesses (Studley Castle,
Warwickshire, 6-7 October 2006).
The Society for Expert Witnesses
made a video interview with Norman Fenton for their Annual Conference
at Studley Castle. The Society was keen to get a proper explanation of
the so-called “Prosecutor’s fallacy” whereby lawyers make incorrect
deductions about probabilities that are known to have a profound impact
on juries. The Society’s Press Officer Tom Magnum had sought out a
number of statisticians and probability experts but was unable to get a
simple explanation that he felt could be understood by lawyers and
judges. He turned to Norman Fenton after reading an article Norman had
written with Martin Neil about the subject of probability fallacies in
legal reasoning.
Norman explained the prosecutor’s fallacy as follows:
“Suppose a crime has been committed and that the criminal has left some
physical evidence, such as some of their blood at the scene. Suppose
the blood type is such that only 1 in every 1000 people has the
matching type. A suspect X who matches the blood type is put on trial.
The prosecutor claims that the probability that an innocent person has
the matching blood type is 0.1% (1 in a 1000). Person X has the
matching blood type and therefore the probability that X is innocent is
just 0.1%.
But the prosecutor’s assertion, which sounds convincing and could
easily sway a jury, is wrong. And it is easy to see why. Let’s suppose
that the crime could only have been committed by an adult male and that
in the population there are 10 million adult males. Then from this
population we would actually expect a very large number of people who
have the matching blood type (about 10,000). If there is no evidence
other than the blood to link X to the crime then X is no more likely
than any of the other 9999 matching blood type men to have committed
the crime. This means that the probability X is innocent is actually
99.99% which is rather different to the 0.1% claimed by the prosecution.
So what is the source of the fallacy and why do lawyers so commonly
make it? It all boils down to a basic misunderstanding about
probability (a misunderstanding which many intelligent people have
because this kind of basic probability is never taught at schools). The
misunderstanding is to assume that the probability of A given that we
know B is true (written P(A|B)) is the same as the probability of B
given that we know A is true (written P(B|A)) . In this case let A be
the assertion “Person X is innocent” and let B be the assertion “person
X has the matching blood type”. What we are really want to know is
P(A|B) (the probability of innocence given the evidence of matching
blood type) and this is what the lawyer claims is equal to 0.1%. But
in fact, what we actually know is that P(B|A) (the probability of the
matching blood type given innocence) is equal to 0.1%. The lawyer has
simply stated the probability P(B|A) and claimed this is actually the
probability P(A|B).
The fallacy becomes especially challenging when DNA evidence is used.
In such cases P(B|A) can be extremely low, such as 1 in 10 million.
When the lawyer wrongly asserts that the probability of innocence is
therefore 1 in 10 million it seems especially convincing. But even in
this case the probability of innocence is actually very high. Assuming
again a population of 10 million people who could have committed the
crime. Then it turns out that the probability of more than one person
having the matching DNA is still actually quite high - about 0.46. So
instead of the claimed 1 in 10 million probability of innocence the
real probability is not much less than 1 in 2. In such circumstances
the ‘beyond reasonable doubt’ criteria can hardly be claimed to be met.”
Here
is a film containing an edited version of Norman Fenton's interview
(this is a Quick Time file of about 6Mb and the Fenton interview is in
the middle).
Link to Fenton and Neil paper:
Fenton NE and Neil M, ''The Jury Observation Fallacy and the use of
Bayesian Networks to present Probabilistic Legal Arguments'',
Mathematics Today ( Bulletin of the IMA, 36(6)), 180-187, 2000.
https://www.dcs.qmul.ac.uk/~norman/papers/jury_fallacy.pdf
