Overwhelming evidence? It's probably a bad thing

Tuesday, 12 January 2016

The old adage that says “If it sounds too good to be true, it probably is” has finally been put to the test – mathematically – in research led by the University of Adelaide.

In a new paper to be published in Proceedings of the Royal Society A, a team of researchers has found that overwhelming evidence without a dissenting opinion can in fact weaken the credibility of a case, or point to a failure of the system.

One of the examples cited by the research team is an ancient Jewish law that said a suspect could not be convicted of a capital crime if all judges unanimously handed down a guilty verdict.

“It might sound counterintuitive to say that a unanimous verdict could be wrong, but this ancient law indicated that the system may be in error if there was complete agreement among the judiciary,” says corresponding author Professor Derek Abbott, a probability expert from the University of Adelaide’s School of Electrical & Electronic Engineering.

The team put three different scenarios to the test based on mathematical probability: the use of witnesses to confirm the identity of a criminal suspect; the accurate identification of an archaeological find; and the reliability of a cryptographic system.

They found in each case that there was a point at which “too much of a good thing” actually weakened confidence in the result.

“In our first example, we imagine there are 13 witnesses who all confidently identify a criminal suspect after seeing the suspect briefly. But getting a large group of unanimous witnesses in these circumstances is unlikely, according to the laws of probability. It’s more likely the system itself is unreliable,” says Professor Abbott.

“In our scenario, the probability that a suspect is guilty is strong after three positive identifications by witnesses. But our tests showed that the more positive confirmations you have beyond those three, the more it erodes our confidence that this is the right person being identified.

"The situation would be quite different if the witnesses had all been taken hostage for a month by the suspect. Then you would expect them all to agree very well who the kidnapper was.

“The ancient Jewish legal practice referred to in our work indicates a surprising level of intuitive sophistication for the time, when such statistical tools would not have been at their disposal. They knew that it was rare for everyone to agree,” Professor Abbott says.

The full paper can be found online.

Contact Details