95% Error Rate

They call themselves criminalists or forensic scientists or something with a fancy name.  They come into court explaining that they've done these tests hundreds of times.  There are, they say, a whole bunch of points of comparison.  And when they find a match, they can point to those points.  

You probably can't see those things certainly not without the forensic scientist pointing them out.  That's because you haven't been properly trained.  Had you been, just like those forensic scientists, you could see them.  And like those forensic scientists, you'd know.  Because, as they like to say on the witness stand, they're never wrong.

Except, of course, they are.  

Nearly 20 years ago, Fred Whitehurst blew the whistle on FBI's flawed, and sometimes dishonest, forensic work.  The Inspector General issued a scathing report.  The Department of Justice promised to review everything, opening case after case.  And then to right the wrongs.

In 2004, no wrongs having been righted, they stopped.  No flies on them.  Until they investigated again.  And concluded that just maybe.

I've written before about the FBI's "stunning" admission that hair comparison, the very hair comparison its forensic guys testified to thousands of times, is bullshit.  And while they claim they're getting the word out to the boatloads of folks who were convicted based on that testimony, well, we'll see how that goes.

While we wait, turn to Al Jazeera America Monday night for Fault Lines.*  Monday's show is "Under the Microscope: The FBI Hair Cases," and it tells that story.  

More, it tells of Joseph Sledge who spent 37 years in prison for a pair of rape murders he didn't commit.  It was an horrific crime scene.  Blood everywhere.  Everywhere except on Joe Sledge.  There were palmprints in the blood.  They weren't Joe's.  But there were also a handful of hairs.  Those, the FBI's crack hair analysts said, those were just like Joe's.  There are 16 points of comparison.  They matched up at all 16.

It tells the story of Kirk Odom, arrested at 18 for a rape he didn't commit.  But the hair, the hair.  And Santae Tribble.  He was 17 when the cops got him for murder.  There were 13 hairs.  It was a murder case.  The FBI had two of their forensic scientists look at them.  Match his hair, they both said.  

You know what happened next.  DNA testing on the hairs.  On the hairs that convicted Joe and Kirk and Stantae.  On the hairs that weren't theirs.  In Santae's case, one of the hairs that sent him down for murder, one of the 13 hairs hairs that the crack guys from the FBI told the jury was just like his, so, you know, . . . . That hair came from a dog.

A fucking dog.  Who he says probably committed the crime.

The bureau's released some preliminary reports on its latest round of reviews.  26 out of 28 of the forensic scientists made false claims at trials.  David Colapinto, General Counsel of the National Whistleblowers Association explains, 

We can now say, based on a statistically sizable sample of cases they have reviewed, [the FBI] were wrong 95% of the time.

They concede that their experts said things that weren't so in helping convict Odom and Tribble. They didn't find a problem in what their folks testified to in sending Joe Sledge to prison for murder.  It just happens that when DNA from the hairs in his case was analyzed, well, they were wrong.  Wasn't him. Just like he'd been saying since he was arrested.  

Because hair comparison, even when done right and testified to accurately, is bullshit.  The comparison microscope is a lovely thing.  But the FBI forensic folks who use it, they aren't scientists. And what they're doing isn't science.  

If you read this blog, you know all that.  You should watch the show anyhow.  It brings home the human dimension and slams home the real science.  And how else will you learn Joe Sledge's answer to the question:

What can they do make this right for you?

--------------------
*Lest there be any confusion, Al Jazeera America's Fault Lines is not affiliated (so far as I can tell) with Lee Pacchia and Scott Greenfield's group blog Fault Lines at which I've been writing lately. 


NB: Thanks to Al Jazeera America for making the show available to me for review.

Where Mistakes Are Almost Never Made

Paul Kennedy refers to it as "the (in)justice system."  Which pretty well makes the point.

But he's like me.  Just a simple little blawgger.  Read, I suspect, mostly by folks who already agree with us about these things.  So for most of the other blawggers I encourage people to read.  They write well and honestly about things that people should hear.  They think carefully.  They have things to teach even to those who generally get it.  You can learn from them.  I do.   

Still, when they we talk about false confessions or the evisceration of the 4th Amendment or some innocent person being killed by the police for not understanding English or how the court just rode roughshod over the law to ensure that some poor sod can continue to get fucked or . . . .

For the most part, we're preaching to the choir.  Of course, some have a bigger choir than others, but the point remains.  What's really necessary is to reach beyond.

So, on Sunday night, turn on your TV and tune it to Al Jazeera America for the first episode of The System with Joe Berlinger.*  

Berlinger's the guy who made Paradise Lost: The Child Murders at Robin Hood Hills.  This time he's taken on not just one case but the whole shebang.  He told Capital New York

Having made a number of films about the justice system, I have become aware of problems, so we started with that. What are the issues? Paradise Lost involved a false confession, and in my opinion some alleged prosecutorial misconduct. So I wanted to use the series to look at those issues at a systemic level. How do we get wrongful convictions? Or misapplication of justice? It is not just a wrongful conviction series, it is about the misapplication of justice. We spoke to experts, and I have my own set of knowledge. Instead of just focusing on a case and what the problems were, i wanted to to focus on an issue and find cases around the issue.

The System isn't a single documentary, it's a series of 8 to be shown once a week each on a different topic, each focusing on, it seems, two specific cases.  Berlinger talks to the folks who were convicted, to their devastated families, to their lawyers, to reporters who covered the cases, to investigators. He talks to law enforcement and (if they'll let him) prosecutors.  And he talks to the families of victims who typically remain convinced, regardless of evidence to the contrary, that the cops nailed the right person.  
It is as restrained and calm and quiet as can be.  Narrated by and with the regular presence on camera of Berlinger himself.  Scraggly beard.  Soft spoken.  Restrained.  And all the more damning for that.

The first episode, "False Confessions," looks at the case of Kirstin “Blaise” Lobato from Las Vegas. She was convicted of the rape and murder of a homeless man based largely on her supposed confession. Except it seems she wasn't.  It appears, instead, that what they took to be a confession was in fact a description of her own rape a month earlier on the other side of the city.  They might be able to find out what really happened if the prosecutor would just test the fucking DNA.  Though if they did, it might turn out that . . . . 

The other story in that episode is Jeff Deskovic's.  I've mentioned him here a couple of times before, and a few years ago I had the pleasure of having a drink with him at a celebration when Illinois repealed its death penalty law.  (He probably doesn't remember me.)  Jeff was in high school when he confessed to raping and murdering a classmate.  That confession led to 16 years in prison.  In his case, they did finally test the DNA.  Which wasn't his.  Cause he didn't do it.

It's powerful stuff.  

The second show, about Mandatory Minimums, is less successful.  Again it looks at individual cases.  Orville Lee Wollard is spending 20 years in a Florida prison because he fired a warning shot - inside his own house - into the wall. Didn't hurt anyone.  Scared the guy who was preparing to attack his daughter.  Let's run that down again.  Inside his own home.  Warning shot into the wall.  Hurt nobody. 20 years. On the other hand, if the guy who shot and killed 16-year-old Hadiya Pendleton in Chicago had gotten a 20 year sentence for his earlier gun offense he'd have been in prison and Hadiya would likely still be alive.  The contrast isn't really fair.  

The problem in Chicago isn't that Hadiya's killer didn't get a mandatory minimum.  Oh, a longer sentence in that case (or the other case in Chicago Berlinger presents) might have prevented that particular shooting.  But criminal laws and penalties built on individual tragedies pretty much always have severe and deeply troubling unintended consequences.  In any event, the Florida and Chicago cases aren't really comparable.  And frankly, the most serious problems with mandatory minimums aren't in gun cases but in drug cases. 

And then there's Flawed Forensics.  In this instance, it's hair comparisons that sent John Huffington to prison in Baltimore for 30 some years and Willie Manning to death row in Mississippi.  The bullshit comparisons were done by the FBI.  As were thousands of others.  And countless more by state criminalists trained incompetently by the FBI.  DNA in Huffington's case showed that hair wasn't his. He's out now, awaiting retrial (there is that messy fingerprint still to deal with).  Mississippi doesn't want to do DNA testing. Manning's execution was called off with just a few hours to go when the FBI sent out word that it's false testimony led to his conviction.  But he's still on the row.  And Mississippi won't allow the DNA testing because . . . you know.

And there are the victims' families.  Who insist, bullshit hair comparisons and DNA exclusions be damned.  Willie Manning should be executed.  Today.  John Huffington should be back in prison and never again see the light of day.

Of course, we know that hair comparison is the least of it.  Fingerprints, bite marks, breathalyzers, even when the science is good (it mostly isn't) its subject to human error and dishonesty.  There are the pathologists who don't cut open the bodies.  The drug analysts who don't bother actually analyzing the drugs.  And then there's the FBI.  It's lab's problems are by now well known.  They have been since Fred Whitehurst blew the whistle, got forced out, sued and won 1.16 million dollars.  And, again, it's their lab that did the hair comparisons that provided the key evidence that got Manning and Huffington convicted.

If there's a story to The System it's that the system ain't all it's cracked up to be.  Those of us who deal with it every day know that.  So do it's victims.  Other people need to know it, too.

Fred Whitehurst, on camera near then end of the Flawed Forensics episode makes the point.

We need systems that we trust.  But we don't need to trust them blindly.

There's a pretty good case to be made that we can't fairly trust them at all.  Joe Berlinger's making a pretty good start at getting the word out.



-----------------------
*My thanks to the publicists at Al Jazeera America who sent me screeners of the first three episodes. 

You Thought NSA and DEA and the Rest of the Alphabet Agencies Were Invasive? You Ain't Seen Nothing Yet!

How different is that from Big Brother?

So wonders Alan Alda as yet another academic neuroscientist runs yet another subject through the functional MRI machine and gets a hint of what they may someday be able to tell.

Used to be if they wanted to figure out if someone was a witch, they'd tie the person up and toss her in the water.  If she sank, she was innocent.  Ah, but if she floated up to the surface and escaped . . . . Only a witch could do that.  How much better we are now, what with lie detectors and the collective guesses of 12 good men (and women) and true.  And we have all that nifty forensic bullshit of fingerprints and bite marks and ballistics that trained lookers just know provides proof positive because dammit those things look alike and they can't ever be mistaken about that sort of thing.  Just ask Brandon Mayfield, who was positively and repeatedly identified by the best fingerprint examiners at the FBI as the guy who bombed the train in Madrid because his

Ooops.  Son of a bitch.  They fucked up.  Not him.  Innocent guy.  Shit.  That can't happen.

But just suppose. Suppose that the next wave of forensic scientists can look inside your brain.  Imagine that they can roll you into the magnetic resonance imaging machine, do what's called a functional MRI (fMRI to the cognoscenti), and tell from the parts of your brain that light up when you answer questions whether you're lying.  Or whether that person you identified is someone you've actually never seen before.  Or whether the guy who says I was never in that 7-11 actually has a picture of it etched into his brain in a way that the fMRI can develop.

That's the sort of thing that explored in a frankly fascinating - and if you believe in autonomy and civil liberties and due process and our adversary system of justice fairly terrifying - two-part series on PBS to be shown September 11 and 18.  (And thanks to PBS and WNET for providing me with an opportunity to screen the series in advance.)  It's called Brains on Trial, and it's well-worth watching.

Start with a fake crime.  Three guys burst into a carry out to rob the joint.  One's pointing a gun at the guy behind the register.  It's loud and messy and naturally the guy's wife comes out from a room at the back and the one with the gun is startled and frightened and turns and pulls the trigger.  She lives, but with massive brain damage.

From fake crime to fake trial.  But in a real courtroom with real lawyers, an AUSA and a criminal defense guy.  And a real judge, the Honorable Jed Rakoff.   And there's a jury (we don't learn who they are).  Of course, the trial is truncated and scripted and shown in fragments and lasts all of maybe 6 or 7 minutes if that (I didn't time it).

But the faux trial, like the faux crime, is just window dressing.  The real show is what the various neuroscientists and researchers show Alda about where their research seems to be heading.  And Alda's chats with Duke Law Prof Nita Farahany about what it all means and how it can be used in a courtroom.  (Gee, if they can read the defendant's brain from a distance does the Fifth Amendment's protection against self-incrimination apply?  Tough call, Nita says while ignoring completely the 4th Amendment implications.)

Some of this stuff isn't quite so cutting-edge as they'd like you to believe.  We've known for a while now that the brain does not fully develop, it's wiring doesn't get properly wrapped, until a person is around 25. The loosely wired brain suffers from weak judgment and poor impulse control.  That's why kids are, well, kids.  And it's why the legal system treats them differently.  And, frankly, it's why it should treat young adults differently, too.  A point Alda and Farahany discuss and agree on.

And some of it - well, the idea of running scans of psycopaths so that (at least this is the suggestion) similar scans can be done of the convicted to determine whether they're psychopaths and should therefore be locked up forever.  Except, of course, the whole idea of psychopathy is something of a parlor trick and psychological charlatanism.  And the test - we'll look at the brains of psychopaths to tell whether they're different so we can figure out who's a psychopath - is self-referential and circular and built on a foundation of bullshit.

Some of what we learn about, on the other hand, is quite stunning.  It turns out, at least according to one study, that when adolescents are with their friends, just the fact of the friends' presence makes a serious difference in their behavior.  Alone, they're just like adults.  But while adults are the same with friends as they are alone, adolescents become more reckless, more impulsive, inclined to show off. It's not that the friends distract or egg them on.  It's just their presence.

It's when you put adolescents with their friends that their behavior and their brain pattern changes.

Maybe.  As I said, it's one scientist's study.  And of course it's preliminary.  And maybe there are a dozen other neuroscientists who get contradicting results.  I don't know.  That's the problem across the board.  Where the studies are truly at the forefront, breaking new ground, figuring out how they can use fMRI to measure and predict and reveal details about behavior, they're not trustworthy - at least, not yet.

This not-science-but-we're-working-on-it isn't ready for prime time (except on TV where it's in fact being shown in prime time).  When it is, if it is someday, if the scientists can simply look inside the brain and see just what happened and why, if they'll be able to determine actual remorse, if they can show that to the jury.

Which brings us back, of course, to the adversary system and those 12 good people and true.  And again to Nita Farahany who observes, rightly, that

every time they bring new science into the courtroom, there's an overreliance by the jurors.

They believe.  They want to believe.  Because the men in lab coats who speak with such certainty.

Of course, this is television even if it's PBS.  That means sexy wins out over careful.  It means more focus on the hocus pocus and the sci-fi and less on the hang-on-this-may-just-be-bullshit.  See, here's a clip that didn't make it into the finished product.  But it's an essential caveat.