The problem is, this technique has never been subjected to
thorough scrutiny, and evidence acquired through it may not be as strong as it
has been claimed to be. A paper published in PNAS this week puts denim-pattern
analysis through its paces, finding that it isn’t particularly good at matching
up identical pairs of jeans—and may create a number of “false alarm” errors to
boot.
Shoddy evidence
For some time, there have been rumblings about the
reliability and quality of commonly used forensic techniques. In 2009, the
National Academy of Sciences published
a weighty report observing that, apart from nuclear DNA analysis, “no
forensic method has been rigorously shown to have the capacity to consistently,
and with a high degree of certainty, demonstrate a connection between evidence
and a specific individual or source.”
The problems with forensic evidence—including fingerprint,
bloodstain, and ballistics analysis—have terrible real-world consequences. According
to the National Registry of Exonerations, nearly a quarter of wrongful
convictions in the United States for the last 30 years can be attributed to
flawed or misleading forensic evidence.
Computer scientists Sophie Nightingale and Hany Farid wanted
to look at one technique in particular: photographic pattern analysis, which
matches up the patterns of details on faces, hands, or clothing between
suspects and crime-scene photographs. Jeans, for example, have a “barcode”
pattern of dark and light splotches along their seams.
Denim barcodes
These patterns have been used as central evidence to convict
people, but is this kind of analysis reliable? That hasn’t been established. To
test it out, Nightingale and Farid went out to buy 100 pairs of jeans from
second-hand stores. They laid the jeans out flat on a hard surface,
photographed the seams along the legs, and digitally traced the pattern of
light and dark points along the seams. To bump up their sample, they had Amazon
Turk workers supply images from another 111 pairs, photographed using careful
instructions.
Then, the researchers set about quantifying how different
the patterns were across different pairs of jeans. Obviously, there’s a lot of
randomness at play here—two pairs could be quite similar, just by chance, while
another two pairs could be entirely different, also by chance. And most pairs
would fall somewhere in the middle, with some degree of similarity. Based on
these measurements, Nightingale and Farid worked out the range of similarity
between the "barcode" patterns on different pairs of jeans.
The important question, of course, is whether these patterns
can be used to determine whether two images show the same pair of jeans. So the
researchers selected 10 pairs of jeans and took 10 photos of each using
different cameras, in different lighting, and with different draping. What they
found was that any given pair of photos could come back with a lot of
similarities but could also come back with very different readings on the
pattern. The range was broad—as Nightingale and Farid point out, soft fabric
photographed in a bunch of different ways is going to have distortions that
vary from one image to the next.
False alarms
So if one pair of jeans can look noticeably different in
different photos, is denim-pattern analysis actually a useful forensic
technique? The researchers used their measurements to estimate how often a true
match would come up and how often their jeans would throw up a “false alarm“—a
score that looked like a match even though the images actually came from two
different pairs.
They found that the false alarm rate could be as high as one
in a thousand. Given that the FBI has reported using photographic pattern
analysis in hundreds of cases each year, that’s a meaningful possibility. The
true match rate was also not great, at around 40 to 50 percent, depending on
factors like the length of the seam being analyzed.
This means the technique of matching up jeans is likely to
be pretty hit and miss—not catching actual similarities a lot of the time and
possibly throwing up a high rate of false alarms. And that’s under controlled
experimental conditions using high-quality images and jeans laid out nice and
flat, not grainy security footage showing jeans being worn. On the other hand,
different features like damage, branding, and size could corroborate an
analysis to improve the evidence one way or another.
There’s more work needed on whether jeans could be analyzed
in a more reliable way using additional features—and also whether other pattern
analysis—like freckles on a face or patterns on other types of clothing—are
similarly unreliable. But for now, write Nightingale and Farid, “identification
based on denim jeans should be used with extreme caution, if at all.”
To read more CLICK HERE
Posts
No comments:
Post a Comment