Examiners had long testified in court that they could determine what fingertip left a print, what gun fired a bullet, which scalp grew a hair “to the exclusion of all others.” Research and exonerations by DNA analysis have repeatedly disproved these claims, and the U.S. Department of Justice no longer allows technicians and scientists from the FBI and other agencies to make such unequivocal statements, according to new testimony guidelines released last year.
Though image examiners rely on similarly flawed methods, they have continued to testify to and defend their exactitude, according to a review of court records and examiners’ written reports and published articles.
ProPublica asked leading statisticians and forensic science experts to review methods image examiners have detailed in court transcripts, published articles and presentations. The experts identified numerous instances of examiners overstating the techniques’ scientific precision and said some of their assertions defy logic.
The FBI declined repeated requests for interviews with members of the image group, which is formally known as the Forensic Audio, Video and Image Analysis Unit.
ProPublica provided the bureau written questions in September and followed up in November with a summary of our reporting on the bureau’s photo comparison practices. The FBI provided a brief prepared response last month that said the image unit’s techniques differ from those discredited in recent studies. It said image examiners have never relied on those methods “because they have been demonstrated to be unreliable.”
But the unit’s articles and presentations on photo comparison show its practices mirror those used in the studies.
The bureau did not address examiners’ inaccurate testimony and other questionable practices.
Judge Jed Rakoff of the United States District Court in Manhattan, a former member of the National Commission on Forensic Science, said the weakest pattern analysis fields rely more on examiner intuition than science. Their conclusions are, basically, “my hunch is that X is a match for Y,” he said. “Only they don’t say hunch.”
Rakoff said that image analysis hadn’t come before him in court and wasn’t taken up by the commission but said that investigators, prosecutors and judges should make sure evidence is reliable before using it.
Scandals involving other areas of forensic science have shown the danger of waiting for injustices to become public to compel reform, Rakoff said.
“How many cases of innocent people being wrongly convicted have to occur before people realize that there’s a very broad spectrum of forensic science?” Rakoff asked. “Some of it is very good, like DNA. Some of it is pretty good, like fingerprinting. And some of it is not good at all.”
Details on FBI caseloads and testimony are not readily available to the public. As such, there is no way to determine exactly how often image examiners testify and when their photo comparisons serve as central evidence in prosecutions. In court, examiners have said they analyze photos in hundreds of cases a year, according to trial transcripts.
To try to identify some of those cases, ProPublica searched court databases and found more than two dozen criminal cases since 2000 in which documents mentioned the FBI’s image examiners, nearly all cases that were appealed and thus had a substantial written record. Few criminal convictions, though, make it to an appeal.
None of the appealed cases led to judges reversing convictions, nor has evidence emerged to show that the defendants were innocent. Still, flaws in forensic science techniques often emerge decades after they’ve been allowed by judges and been used to secure convictions.
The problems with the FBI’s photo comparison work plague other subjective types of forensic science, such as fingerprint analysis, microscopic hair fiber examination and handwriting analysis, said Itiel Dror, a neuroscientist who trains U.S. law enforcement on cognitive bias in crime laboratories. Dror is a researcher at University College London, frequently teaching at agencies like the FBI and New York Police Department on ways to minimize personal beliefs from influencing casework.
Even DNA analysis can be swayed by bias, Dror said. But pattern-matching fields like image analysis are especially vulnerable. Image examiners’ lab work is, generally, only seeing if evidence from a suspect “matches” that from a crime scene.
“Many of them are more concerned by what the court accepts as science rather than being motivated by science itself,” Dror said.
To read more CLICK HERE