In this New Thinking podcast, Reuben J. Miller, assistant professor of social work at the University of Michigan, and his research collaborator Hazelette Crosby-Robinson discuss some of the criticisms that have been leveled against risk assessment tools. Those criticisms include placing too much emphasis on geography and criminal history, which can distort the actual risk for clients from neighborhoods that experience an above-average presence of policing and social services. "Geography is often a proxy for race," Miller says. Miller and Crosby-Robinson spoke with the Center for Court Innovation's Director of Communications Robert V. Wolf after they participated in a panel on the "The Risk-Needs-Responsivity Framework" at Justice Innovation in Times of Change, a regional summit on Sept. 30, 2016 in North Haven, Conn.
The following is a transcript of the podcast:
Robert V. WOLF:
Hi, I'm Rob Wolf, director of communications at the Center for Court Innovation. Today with me at the Justice Innovation in Times of Change conference here at the Quinnipiac School of Law in North Haven, Connecticut, are two of the panelists who participated in a discussion about risk needs assessment tools. They are Professor Reuben Miller, who is an assistant professor of social work at the School of Social Work at the University of Michigan and his research assistant at the School of Social Work, Hazelette Crosby-Robinson. Thank you so much for taking the time after your panel to sit down and talk with me.
Reuben J. MILLER:
Thank you. Thank you for having us.
Robert V. WOLF:
I wanted to just start off talking about the risk assessment tools and some of the criticisms that have been leveled against them, because as we heard on the panel from Sarah Frichy, a colleague of mine at the Center for Court Innovation, their use has exploded and they've been embraced as a decision-making tool in the criminal justice setting. But you raised some potential concerns about them and some of their limitations. I wondered if you could share what some of those limitations are as you see them.
Reuben J. MILLER:
Sure. I'm happy to. Hazelette is kind of my research associate and collaborator, so anyway, even though she's super modest.
I'd like to first preface this by saying some scholars have suggested that we really entered an actuarial age. It's not just risk assessment in criminal justice, but a whole cost-benefits calculus, the whole risk calculus that's based on actuarial models that try to predict future harm. They try to predict much like an insurance company would try to predict the future risk of a car accident in a criminal justice setting. These risk needs assessments are trying to, one, gauge the needs of incarcerated individuals or people who have been convicted of a crime to try to figure out where they could show up deficits in their skillsets or in their general stability. For example, they might examine things like housing stability or whether or not one was employed or what kinds of service needs they might have.
For example, if one has a history of substance use and abuse, that would indicate that they need treatment or some sort of intervention based around these things. At the same time, they're trying to gauge the risk of re-offense, so the risk that they will commit a crime.
There are a number of criticisms. The literature that engages this is fairly long. I tend to think about some of the movers and shakers in this field, Kelly Hannah-Moffitt, Bernard Harcourt, Sonja Star, Faye Taxman. Faye Taxman's work is actually helping us to think about important ways that we can implement risk assessment that reduce some of the biases that are sort of baked into it. But just to talk about some of the critiques that come from this literature, and of course from my own.
On the one hand, there are static factors like where one lives, so geography, their prior criminal history. These are things that they can't avoid. And the privileging of recidivism as an indicator of success, these are all problematic for the following reasons. Geography is often a proxy for race. We know that we live in a country that has a pattern of residential racial segregation. We know that policing and criminal justice resources of all kinds of overwhelmingly distributed in areas where poor people of color tend to live.
The problem is people are now being arrested from, return to, and even given programs designed to rehabilitate them all within low-income communities, very bounded geographic districts. What you get is you get the overwhelming concentration of criminal justice resources, and you get a signaling of what that all means. If the substance abuse treatment house is located in a neighborhood, that tells me that there's substance abusers there. That signals narcotics forces to the community. It says something about the community. Halfway houses also overwhelmingly there.
One must think about what the concentration of these things do. Now, as it relates to risk, being in a neighborhood like this triggers a higher risk score. It is indeed one of the measures of risk. In that way, it's a proxy for race. Sony, I know I'm talking quite a bit.
WOLF:
No. Just to kind of summarize though, or to recap what you've said so far, the way risk assessment tools work, they place a high value on location someone's from. They place a high value on their history with the rest.
MILLER:
That's absolutely right.
WOLF:
If there's a preponderance of enforcement there, so people are more likely to have an arrest record.
MILLER:
That's absolutely right. The studies don't stop. Let's make this abundantly clear, that even when people aren't doing anything wrong, they're being overwhelmingly stopped if they're black or Latino. We know that criminal justice contact increases the likelihood that one will be arrested.
Anyway, this is a big problem of using prior arrest records, for example, and even prior conviction records. Now you've got a bunch of arrests. By the time you get to the prosecuting attorney, they're going to say, "Look, you've been arrested 14 times." "Well, I've been arrested 14 times, but never charged." "No, but you have a history of arrest and so I'm going to now charge you because I see a pattern." This is how statistical discrimination might work, or does in fact work in practice.
Now the prosecuting attorney sees a pattern, sends it before the judge who looks at this pattern and interprets it to make a decision about the length of the sentence, when the conviction is read, as is a jury, if it ever goes to trial. 97% of cases never go to trial. But when it goes to trial, jury is presented with the same evidence of patterns which have more to do with where the police are concentrated, then what people are actually doing.
WOLF:
What do you say to the notion that these instruments are validated, that they predict? I mean, this information, whether there's a potential bias incorporated into them, they still can predict six months to a year out whether someone is going to recommit a crime.
MILLER:
Yes. With great reliability, but that it's a population is being normed against itself. I overwhelmingly concentrate criminal justice resources and in particular neighborhood, which leads to more arrests, which leads to more convictions, which leads to more imprisonment. Then I look at those who were imprisoned and I use that to validate my measures.
The problem is this sort of self-fulfilling prophecy, this feedback loop, this is one problem. Another problem is that, and Kelly had a Moffitt points this out brilliantly, correlation and causation are very different things. It's like the standard social science response that any bench chair social scientist gives when they look at two relationships and people use that as some sort of cause, but likelihood that a particular groups of people are more likely to commit a crime doesn't mean that having committed a crime in the past means you will actually commit a crime.
What we're doing is we're treating a relationship as if it's a cause, as if it's a fact. I will sentence you now, based on my assumption of your future danger to commit a crime based on a set of assumptions that I used to justify the overwhelming consecration police to begin with. The police aren't the culprits here. It's a rationality. It's a way to approach problems that I think must be critically investigated.
WOLF:
You also pointed out in your presentation that perhaps the cultural context, the environment, and the changing policy culture, where for instance, marijuana arrests, which were so vigorously pursued several years ago, are now considered a low priority or they're not even being done anymore, and yet people have a record of those arrests. If history of arrest is a factor, I mean, someone in the audience also questioned this, should we drop that as those particular kinds of arrests as a factor because we don't care about them anymore? Do they indicate further likelihood of going against the law, or were they just something someone did because they liked marijuana and that's it?
MILLER:
That's right. That's right. This is a part of the rigidity of risk assessment. This is the rigidity of risk categories. To place one in a category, you are an offender.
In Michigan where I've done a lot of research and where I've worked habitual offenses, and this is, it's not just like this in Michigan, it's like this in many, many states, most states I would argue, being a habitual offender means more time, greater risk, more punishment.
Hazelette CROSBY-ROBINSON:
Up to life?
MILLER:
Absolutely. Absolutely. What does it mean to habituate? What am I looking at? Well, if I'm not being careful about the criminal codes, if I'm not carefully examining what I considered a crime at a given moment in time and adjusting my instrument for that, which must happen probably annually, if I'm not adjusting my instrument for that, if not quarterly, if I'm not adjusting that for different understandings of what is right and wrong, then what I'll end up doing is habituating someone, giving them longer sentences, giving them harsher treatment, deeper levels of punishment, or indicating that they need deeper levels of intervention.
WOLF:
Well, so tell me what recommendations you'd make, because you also made a point in the panel that there are some good things about risk assessment. They do take away discretion from judges or people whose own bias might lead them to make the wrong decisions.
MILLER:
Absolutely. The benefit of risk assessment is to use it to avoid the criminal record to begin with. This is one benefit. If you have low risk, low leverage, as my colleague pointed out earlier today, then you are not indicated for intervention of any kind. It's better to just release these folks without intervention of any kind.
WOLF:
That's what the research supports.
MILLER:
The research supports it. Absolutely. What risk assessment allows the careful prosecutor, judge, defender, the public defender, et cetera, to do is to remove some of the discretion, because much of it, you mentioned this, much of the decisions that are being made are being made based on a gut feeling. I am reading something in the defendant. They don't have remorse, or they haven't shown accountability for their actions, or they have, as one of the panelists raised, belligerent interactions, let's say, with their parents or the prosecuting attorney or the defendant. My assessment is happening, divorced from what it means to actually be in court in that moment in time. How might a child, 17 years old, respond to facing 20 years in prison? How should they respond? Should they be depressed, sad, angry, avoidant? What are our expectations in this moment?
What risk assessment allows us to do is say, "Okay, wait a minute, let me take a step back. Let me look at what actually happened. Let me get away from my intuition. Let me think about a more objective way to assess how this defendant should be treated."
An interesting note, so here it is. We can use smart risk assessments to think carefully and critically about how we treat offenders, what level of intervention that we lay out, whether that intervention be prison or jail time, or a diversion program, or a treatment group. There's no perfect way to do this, which is why constant re-evaluation is necessary. You can't settle. This is the instrument for me. You can't settle. It's not the instrument for you.
CROSBY-ROBINSON:
Continuous improvement.
MILLER:
Continuous improvement.
WOLF:
And maybe testing. If I understood what Sarah Frichy said, my colleague, a researcher at the Center for Court Innovation, at you also can test these instruments within certain populations and see are they producing more negative outcomes for an African-American population when ask these questions that you're asking to weed out the bias that might be built into them?
MILLER:
Absolutely. The questions that we're raising in some ways a set of philosophical questions, but they're questions about the application, the use of, the embrace of instruments to determine whether or not someone is a future danger. Perhaps this is just the wrong approach altogether. Not the risk assessment. Not that one doesn't need to think about ways that they can help predict behaviors of individuals. I think that's useful in some ways, but it certainly needs to be challenged. It needs to be questioned. What am I predicting? Who am I predicting this for? What are the possibilities for this person once these predictions are made? These are questions that need to be addressed.
WOLF:
Ms. Crosby-Robinson, let me ask you, as we talk about these kinds of assessments, you bring to bear your own set of experiences with the correctional system as a researcher, and your own past history, which you referred to on the panel, as someone who had been formally incarcerated. I wonder what insights that has allowed you to bring to bear to this notion. Presumably a long time ago, they didn't have these risks assessments. I don't know when you were initially had your first contact with the correction system, the justice system, and now they do, and you've had a lot of contact and opportunity to interview and spend time with people who aren't incarcerated. I wonder where you come down on this issue.
CROSBY-ROBINSON:
Well, first of all, I think it's a good idea to have a risk assessment, as Reuben said earlier, because it removes some of discretion from judges and prosecutors to make these decisions based on their own personal bias or how they're feeling at the time.
But what it does not account for are all of the little various innuendos that a person is going through when they come a family reunification can create a stressor, or if somebody coming out and they have to be paroled to a family address, a suitable relative placement. They're coming to this family address, but the family addressed that the parole officers decided that the person can parole to is not really the best environment.
Sometimes the issues that they had that led to their incarceration stemmed from the family issues that they were having at the time, or it's not in the right environment, or they don't have really enough support from their family. Things happen because lives are fluid and things change. For instance, we interviewed a person who was 17 years old, and she was pregnant. She had a mental illness. She had been in the mental health system since she was eight years old. She lived with her grandmother.
We interviewed people three times as soon as they were discharged, and then 30 days after they'd been out, then 60 days and 90 days. Following her for, by the time we got to her third interview, her grandmother dies. She's living in her grandmother's house. This is the only stable person she's known in her life. Her grandmother has raised her since nine years old. She just had a baby. The baby isn't a year old yet. Now she's 18 years old. She has mental illness and she's relying on that system to become her support now where her grandmother was everything.
Well, these are things that a risk assessment just would not pick up because you never know what's going to happen. Now what happens to this individual? We're at the end of the time that we follow this person for our study. But you know the question is in our mind. What happens?
Another thing that I find frustrating is no matter what your risk assessment is, and if you get it right or not, then when a person gets out into the community, whatever the risk assessment decided that they need as a support or an intervention, there's no community resource for that.
WOLF:
The theme I'm hearing from both of you is that these risk needs assessment tools cannot be judged or effectively used apart from the environment, whether it's the environment that it created the measurements of risk or the needs, because if you can identify needs and say, "Great," but if you don't have the resources in the community, it's meaningless information.
MILLER:
Absolutely. Faye Taxman has a great paper. She finds that of on the needs side of things, substance abuse treatment is indicated in about 90% of the folks who are just as involved, but the capacity to provide the treatment, either in prison or out, something like 25% of folks in prison were able to actively engage in regular substance abuse treatment that needed it. What this does is it creates another deficiency that one might judge or regard as a part of the risk of this individual recidivate. Did you complete programming? What was programming available, either in prison or out?
WOLF:
Well, this has been a very vigorous and interesting conversation. I really appreciate you both taking the time to speak with me about your work.
MILLER:
Thanks for having us.
WOLF:
I've been speaking with Professor Reuben Miller, assistant professor of social work at the School of Social Work at the University of Michigan and his research assistant and collaborator, Hazelette Crosby-Robinson. We're all here today in New Haven at Quinnipiac University School of Law for the conference Justice Innovation in Times of Change, which is sponsored by the Center for Court Innovation and the Department of Justices Bureau of Justice Assistance and hosted by Quinnipiac University.
You can find out more about risk needs assessment and criminal justice reform in general at our website www.courtinnovation.org. I'm Rob Wolf. Thanks for listening.