The Fools Gold Standard

Some years ago, I took a court appointment to represent a guy on appeal who'd been convicted of raping and killing a young child.

It was a horrible case with no dispute about what had been done to the kid.  There was just one question:

Was the defendant the guy who did it?

This was no standard mistaken identification case.  There was no eyewitness to cross-examine.  No fingerprints to argue about.  And it wasn't a circumstantial evidence case.  It was DNA.  You know, the gold standard.  And the DNA results were, well, compelling.  The usual 1 in 741 septillion or something.  More than the stars.  It simply had to be him.

Or not.

Because there was also this other DNA test.  The one that excluded him.  Whoever it might have been, couldn't have been the defendant.

In theory, both of those tests could have been right.  Those astronomic numbers don't exactly prove.  They just make really likely.

The other possibility is that one set of those test results is simply wrong.

I've talked about astronomical DNA numbers before (see here) and explained that they don't exactly prove what they're thought to.  But those numbers and what they do or don't mean are only part of the DNA story.  What made me think about this stuff today, and especially reminded me of my former client, is this blog post by Walter Reeves. Here's the plot.

Itiel Dror and Greg Hampikian recently published a paper titled Subjectivity and Bias in forensic DNA mixture interpretation. They wanted to address the potential problems in interpreting evidence when there may be multiple suspects. The situation most commonly occurs in sexual assault cases where there are multiple perpetrators. There may not be enough to conclusively identify one person, but there may be enough to say an individual cannot be excluded - which in the minds of most jurors means you must have done it.
They obtained an actual case out of Georgia that involved a gang rape. One of the alleged suspects identified and testified against the others. The results of the DNA analysis - by examiners who knew who the suspects were - could not exclude the others, which corroborated the co-defendant testimony.
The actual data was submitted to 17 qualified analysts who routinely do forensic work. No other facts were sent, so the examiners did not know who the suspects were. The results obtained without that contextual information were startling. Only one of the 17 agreed with the original examiner. Even more startling is that 12 examiners would have EXCLUDED the suspect they looked at. The remaining 4 would have called the results inconclusive.

Now, Reeves goes on to talk about how this is serious and how it shows the need for independent testing and not trusting the government's expert to have got it right because, after all, it's DNA.  All that is surely so.  But it's important to be clear about just what is so.

The science of DNA is in the technology.  Done properly, with properly preserved and protected samples, the results are exactly what they are.  But along with the science is the art. (Or the bullshit, depending on how you want to discuss it - Truman Capote famously said of Jack Kerouac's books, "That's not writing, it's typing.")  That's the part where someone tries to figure out what the science revealed.

The science of DNA is good and reliable and actually science if it's done properly and if the samples tested were properly taken and preserved.  The bullshit of DNA is in making sense of the science.

And while Reeves is certainly right that one lesson is that you can't trust the government's bullshit artists, there's a broader lesson, which is that we can't trust at all.  Not even the things we know to be true.

We know, if we've been paying attention, that fingerprints and ballistics and hair comparisons and bite mark matching and all that stuff done by trained lookers with special microscopes is just so much bullshit.  It might be right in any given case, but it might as likely be wrong and since there are no normative studies, it's impossible to know the odds even when matching and testing is double or triple blind and done by the superlatively trained.

We know that eyewitness identification is shoddy at best, and rarely done best.  We know the flaws of memory.  We know about junk science.

But we believe all that stuff anyway.  More to the point, cops and then prosecutors and then judges and then jurors believe it.  And Nancy Grace sure as hell does when it points to guilt (just as she doesn't when it suggests innocence).  And so do newspaper editors and the general public.

Just as they believe that cops and snitches and rolling co-defendants tell the truth on the stand.

And sure, maybe sometimes.  But maybe not.

My favorite cross-examination question, perhaps a bit too snarky but it was from a suppression hearing with no jurors present, asked by the late Jim Van Deilen of a crime lab chemist who said that he'd tested the stuff, and it was indeed cocaine or whatever it was.  He was experienced, you know.  He'd done 10,000 (or whatever mega-number) similar tests in the last five years.

Did you do any of them right?

Truth is a chimera.

DNA is exactly as meaningful as it is convincing.

Proof is what the jury believes.

Reasonable doubt is when they don't trust.

Really, they never should.

My former client, by the way, is still serving his life sentence.  I lost the appeal  There's some reason to believe the DNA test excluding him wasn't particularly reliable, and there were real problems with the state of the record.  But there were problems with the other test too.

Sometimes we just get hosed.