In this New Thinking podcast, Harvard Law School Professor James Greiner talks with Aubrey Fox about why he launched the Access to Justice Lab, which has researched topics as varied as how to provide self-help materials to defendants involved in civil debt collection cases to the effectiveness of commonly used pretrial assessment interview tools in criminal court. Greiner also discusses what he sees as a strategy for improving the legal profession's openness to evidence-based thinking.
The following is a transcript of the podcast:
Aubrey FOX:
So this is Aubrey Fox, I'm today's guest host for the New Thinking podcast, which is the Center for Court Innovation's podcast, where we interview legal experts and researchers. And I'm very fortunate to be joined today by Professor James Greiner, who is the William Henry Bloomberg Professor of Law at Harvard University. Does this mean that you're connected to our former Mayor Bloomberg?
James GREINER:
Actually, I think that, if I have it right, the chair was named after his father and he established it on his father's behalf and it's a rotating chair. So I have it for, I guess, another few months before it rotates to someone else. But that is the connection.
FOX:
So you're not dining with Mayor Bloomberg on a regular basis?
GREINER:
I haven't had the pleasure yet. I'm sure it'll be terrific. I'm guessing he probably sets a good table, but I haven't had the pleasure yet.
FOX:
Among many things, Jim, is the Faculty Director of something called The Access to Justice Lab, which is a really exciting new initiative that he started where he's trying to bring evidence-based research to the study of criminal justice and civil justice. And so we want to talk to Jim today about his work with The Access to Justice Lab, and also some of his thinking about why it's important to introduce this evidence-based approach. And want to talk to him a little bit about some of the particular issues he's investigated.
But, Jim, maybe where we'll start is just to let you talk about how you got interested in this area of criminal justice and civil justice. And tell us a little bit about your background.
GREINER:
Sure. So when I graduated from law school, I hadn't taken any statistics courses or any quantitative courses in college or anything like that. I just wanted to be a regular practicing lawyer. And so I did that for three years for the Justice Department and then three years for a private law firm, a law firm called Jenner & Block in DC. And at the time I was litigating cases that involved some numbers. So employment discrimination, class actions, litigation about the 2000 decennial census, whether you could use statistical techniques to try to correct for under-counts and over-counts according to racial groups.
I'd always been interested in social justice issues and in criminal law issues. When I moved over to the private law firm, I did a couple of pro bono cases that we picked up and, basically, through the practice, redistricting was another area I did, got interested in numbers and what numbers and evidence-based thinking could bring to the table in terms of social justice issues.
And so after six years as a practicing attorney, I left the practice of law for five years to get a PhD in statistics. And then when I got my PhD in statistics, I started researching and was researching in a bunch of areas, but the one I was having the most fun with, really enjoying, and the one that I thought I could potentially do some good with, was access to justice issues, more on the civil side than on the criminal side, but some on the criminal side also.
So issues like how do you make law work and be accessible for folks who can't afford to hire lawyers to interface with the legal system? And on the criminal side, how can we make existing institutions perform what we want them to perform, better? How can we release more people who are not dangerous? How can we reduce the criminal footprint on communities and still maintain a system of law and order, et cetera? So those are the big picture issues that I found most interesting.
FOX:
So having a statistician as a professor of law, is that an unusual development or is that something that's been going on for decades and decades?
GREINER:
No, I think there are only two of us still in the legal academy that have stat PhDs. I know the other person surprisingly. But having someone as a law professor who knows as, we say, how to count, that's not all that unusual. It's just folks, typically, have economics degrees or political science degrees or something like that. But the hardcore statistics is what I really wanted to get invested in. And the reason for that was I, pretty much, knew when I was going into the statistics program that I was a litigator through and through. I was always going to be a litigator. I was always going to be somebody who's going to care about courts and administrative agencies and the practice of law at a person by person level.
And so I didn't feel I wanted to learn a lot about the big picture political science theory and big picture economic theory that those folks do. And so I just said, "Let's go straight into the heavy numbers."
FOX:
And so tell us a little bit about The Access to Justice Lab. What does it do? When did you get it started?
GREINER:
We're talking in December of 2016. It's only been in existence since July of 2016, and it is funded by a generous grant from the Laura and John Arnold Foundation. The foundation itself cares about evidence-based thinking across a lot of different spheres and it funded the lab. And, basically, the lab has two overall missions: One is to produce useful rigorous evidence that would assist policymakers and judges and lawyers in what they do, especially with respect, again, to access to justice related to folks who can't afford to hire lawyers to interface with the legal system for them.
And then the second overall purpose is to try to tear down the resistance to rigorous empirical evidence, especially via the randomized control trials or randomized field experiment, that resistance that exists within the legal profession and within the judiciary. I think that, in law, especially with the practice of law, we are roughly where the medical profession was in about 1938 or 1940 where, basically, we are beginning to engage in a debate about whether our profession should be evidence-based in the way that medicine was engaging in a debate about whether drugs and medical devices should have evidence behind them before they are allowed to be sold to the public.
And there are many folks in law, on both the bench and the bar, who think that randomization in law is unethical or is unnecessary. Unethical because randomization, it takes away the professional judgment about who should get what; and, unnecessary, because we know that, as lawyers and judges, all of our professional judgments are perfect or very, very good and, therefore, we don't really need to investigate them all that much.
And so The Access to Justice Lab's second purpose is to try to suggest to folks that scientific-based thinking and evidence really can bring a lot to the table and it may end up overturning accepted truths within the bench and the bar.
FOX:
I'm glad in your comparison to the medical world, you didn't bring up leeches. So at least we're a hundred years ahead of where we could be.
GREINER:
Right.
FOX:
And just to follow up for a moment, I want to get your sense of the state of play on this, because I understand that there may be resistance to RCTs, which are the most rigorous form of evidence-based analysis and require dividing a control group from an experimental group and giving some people something that you don't give the control group.
But there is a history of doing research into criminological issues so it's not like you're starting from square one. So, I guess, in this mix of some history and some resistance, where would you say we are at the moment?
GREINER:
I'd say more resistance than history with respect to courts and judges and lawyers. So you're absolutely right that there is a reasonably well-developed literature in the criminology field, and a good portion of it, not as much as many criminologists would like, but still a good portion of it backed up by randomized studies to try to figure out whether, say, afterschool programs prevent people from getting arrested. Or whether certain types of treatment as a condition for probation are effective in preventing recidivism.
But if you notice in the two examples that I gave you, one of them is prior to the involvement, the intervention is prior to the involvement, of the criminal justice system. And one of them is post-sentencing, after the involvement of what we call the lawyer-based or court-based criminal justice system.
And so there has been substantially less in that portion of the criminal justice system where the lawyers and the judges get involved to the point where, when a co-author and I tried to catalog all of the randomized studies that had been done in the United States law, criminal or civil, that involved randomization of a decision that would otherwise have been made by a judge or a lawyer. So that's how we defined randomized studies in law. I mean, we're taking a decision that otherwise would have been made by a judge and a lawyer and randomizing it to find out whether the decision is worthwhile or the [inaudible 00:09:52] treatment or something like that.
We could only find about 50 in the history of the United States, based upon all areas of law, all sorts of settings. So if you think about medicine starting from 1938 or 1940, the number would probably be uncountable. It'd be tens of thousands or hundreds of thousands of randomized studies that have involved replacing a judgment that would have otherwise have been made by a medical professional with randomization or to figure out whether the treatment or the intervention works, whereas we could find fewer than one per year in law.
So, again, I absolutely agree with you. Criminology has much more of a history, it's a much more developed field, but they are, basically, working in their rigorous evidence prior to the former legal system or after it in the lifetime of a particular defendant or person or crime, and where we think the evidence really needs to be focused is ... At least, in terms of a new effort, we need to focus on where there isn't as much evidence right now, which is inside courts and inside law offices.
FOX:
One of the reasons that I wanted to speak to you, Jim, was the Center for Court Innovation, while we're known more for our work on criminal court programs, we also have been developing a civil justice portfolio. And so I'm curious to hear you talk about some of the work you've done on issues, and you're continuing to do, like debt collection and the assignment of counsel in housing court cases.
And, I guess, the framework I have for my question is making the case for this research may rest on finding practical implications that flow from the research. So I'm wondering if you can give us some examples of research you're doing and what you see as the things that a practitioner would immediately identify as, "Well, that's really interesting and useful."
GREINER:
Absolutely. So I, guess, two examples ... And I don't want to go on for too long, so stop me if this is starting into a monologue, but two examples with respect to the financial distress work which you mentioned, the debt collection work that you mentioned, one of the premises of that work is that there is just never going to be enough money. There's just never going to be enough of a social commitment to provide a lawyer at state expense for every individual who can't afford to hire a lawyer who has a legal problem.
And so even if we were to restrict that set to people who are facing court adjudications that affect basic human needs by some definition, which is the ADA's proposed definition for what's called a Civil Gideon project, this idea that we can save expense, I just don't think the resources are ever going to be there. I mean, some smart economists have tried to calculate that and said, "Billions upon billions of dollars would be needed."
And so with that in mind, it's probably going to take a battery of lots of different things to address the access to justice problems that we're having in the United States society. And one of those things, one of those many things that we need, is going to be self-help. How are you going to make complex subjects accessible, complex procedures and complex principles, accessible to folks who can't afford to hire lawyers for them so they can try to address legal problems they have on their own?
And so the financial distress work that I'm doing is an attempt to try to say, "Well, if what we need are self-help materials and we need, basically, to educate people about how to solve their own problems, what principles can we draw from non-law fields that would help us make self-help materials more effective or what we hypothesize that would make self-help materials more effective?"
So, for instance, if we're trying to get people to a particular location at a particular time of day and on a particular date to do something they would rather not do because it's good for them, we think, or they decide it's good for them, how do we persuade people to do that? Well, if you pose the question at that level of generality, you recognize that voting ... Political scientists study how you get people to vote on a particular day at a particular time, public health officials study how to get people to take flu shots or to get colonoscopies on a particular day. These are all studies that have been run. But, in law, our self-help materials don't actually use any of the lessons from those other fields about how to get people to a court date, and we think that's bad. We think that if good lessons are learned from those fields, we should try them in law and see if they work.
Similarly, there's an entire field called adult education, or just education period about how you communicate complex concepts to people who need to be able to use them. But in law or self-help materials, don't currently use the lessons from adult education and you can see that because if you read them, the self-help materials, they're word heavy. They're just lots and lots of texts. Whereas adult education folks, or education folks, generally, are saying, "No, no, no. You've got to use images." So in our case, we're using cartoons.
And so the idea there is to say, "Okay, well, let's try to create the best possible self-help materials that under the state of the art we currently know how to create using the lessons from all these different fields, but then let's don't declare victory there because maybe there's something about law, or maybe there's something about the way that we did the self-help materials that would make them not effective or maybe there's something they're not understanding or maybe the best possible self-help materials are just not enough in a system that is so complex that self-help is just not an option."
And we go test them. We go put them into the field and we do a randomized experiment where, in one randomized experiment, we might randomize people to either get self-help materials or an offer of representation from an attorney. Or in another experiment we got randomized, whether people get self-help materials versus no further assistance, our self-help materials versus existing self-help materials which, in many cases, just don't exist. And so that's one example.
I've got another one, but again, I don't want to go on and on.
FOX:
I'll just cut it for a moment to say the research you're doing on self-help or pro se counsel is relevant to us because we run a program called Legal Hands which, essentially, employs volunteers who are not lawyers. And so it's a bit of a blended model. This is people who are physically talking to a community member. So it's not something that you go online to get self-help materials. But they're not lawyers. And it speaks to your general point, which is, it may not be possible to have a lawyer for every person on every case of relevance to them.
GREINER:
Absolutely. And I think what your program is doing is mixing a partial service-based solution with a partial self-help. So it might be a guided self-help. And, absolutely, I mean, again, I think part of the problem ... One of the reasons why I find this field so much fun to operate in, but also so frustrating, is that we just don't have any evidence to suggest, to guide us, as to when we need a partial service-based solution, partial self-help and in what settings a total self-help solution is sufficient. Or maybe it's not the legal setting, it's the type of person.
FOX:
Right.
GREINER:
Which people can take care of it on their own if guided appropriately, either by self-help materials or which ones are going to need some service along with it? So, absolutely.
FOX:
Let's hear your other example.
GREINER:
Another example that comes to mind is the problem of triage, which is, basically, whenever there is more of a particular type of case, or another way to put it, more clients than you have the capacity to provide the highest level of assistance to. So in our particular study that we've been trying to get underway and it's still in the planning stages for this setting, is in victims of domestic violence who come to a legal services provider and seeking assistance with obtaining civil protection orders from courts. And legal services provider has the capacity to assist in terms of providing full representation to around a third of the folks who come to it, who are eligible for the help, income and asset eligible, and who are actively seeking civil protection orders.
Now, that, by the way, that number one-third would go down substantially if the legal services provider did more outreach in the community. So we've looked in the court systems where this LSP is operating and, in fact, a substantial number of people are already in the court system seeking self-protection orders without counsel. And so you only see the tip of the iceberg and you can't even handle the tip of the iceberg in terms of providing full representation, the traditional attorney-client relationship to these folks, much less the whole iceberg.
So what do you do? And it turns out that in, Madison, again, to use that analogy, there's been lots of study about how to make these triaged decisions. It's both an ethical question and an operational question: Ethically, what do you want to do when you're in that scare situation? And that comes up both in medicine in terms of something like organ ... When you don't have enough organs for transplant. It comes up when you have Hurricane Katrina knocking out power in a hospital and you don't have enough evacuation capacity to get all of your patients out of your hospital. And that was the subject of a recent Radiolab episode. So there's an ethical dimension to that question.
There's also an operational dimension. Suppose what you wanted to do was to say, "We should use these resources to save the most lives." Or in the civil protection order context, "We should use these resources to try to get as many people as possible civil protection orders because that's what they're for and that's best for the community." How would you do that? How would you go about doing that?
And one way you would go about thinking about it, the way that the military thinks about it when we're talking about battlefield casualties, is you say, "Well, try to distinguish people into three groups." One, is a group of people who are going to be able to succeed on their own, so in the battlefield casualties these are going be people who are going to survive the battlefield, even if not assisted immediately. You don't provide immediate assistance to them ... And the legal services provider, people who are going to be able to get civil protection orders on their own, you don't offer full representation to them.
For people who are never going to be able to survive on the battlefield or are never going to be able to get civil protection orders on their own, even if given the full representation, you don't give them full representation either because it's not affecting the outcome.
The people you do give battlefield assistance to or full representation to are the folks that you can change their outcome. If you give assistance, they'll survive or they'll get the civil protection order. If you don't give assistance, they won't survive or they won't get the civil protection order.
So that's one way to think about it. But if you want to do that, you face the problem of how do you go about distinguishing those people? Because a lot of study in a lot of different settings, again, emergency room setting is one example, turns out that's a very hard thing to do, to figure out who you can actually change the outcomes for.
And so we're pursuing a randomized experiment that's designed to try to add to our knowledge of how do you distinguish those folks? And, most importantly, whether lawyers exercising professional judgment can, in fact, predict which people really would benefit from the help?
FOX:
And, I guess, what you're talking about raises a big issue for me, which is, let's give a concrete example: in New York City, they're talking about investing potentially $200 million more a year in hiring lawyers to represent people facing eviction in housing court. So there is actually more money now in areas of the law where there hasn't been a lot of money in the past.
However, you're no doubt correct that there isn't enough money to cover every case at every moment. And so, I guess, having a RCT or a research study is great progress towards this habit of thinking in a ... I would use the term problem-solving way, but how do we get to a more fundamental change? What's the way to inculcate this way of thinking in a legal services organization in a more fundamental fashion?
GREINER:
It's a really hard question and one that we've been scratching our heads about in The Access to Justice Lab about, I mean, because that's our overall mission. I mean, one way of thinking about this is if The Access to Justice Lab is going to be one of the few institutions around the country that's going to be doing randomized studies in the law and we're going to depend on those few institutions to produce evidence, we should stop. We're never going to be able to produce enough evidence even as a lab and even as an institution. One or two institutions are never going to be able to do it just the same way you got one center, Johns Hopkins, can never produce enough randomized experiments in medicine to make progress in medicine possible.
And so we have to start a movement. We have to get people to think this way and to demand this evidence. In terms of what the strategies we're pursuing, The Access to Justice Lab we are, first of all, going to try to produce studies and try to educate folks about, "Hey, this evidence from this study can be useful in your practice. It can get you to think about a way other than the way you were thinking before or ask new questions about what you should be doing to be more effective." We're in the process of producing a short course. So that folks who either want to do these studies as researchers or to form partnerships with researchers to get the study done, so these are field operators, we'll know more about what's required and how to do them.
We will be doing persuasion in terms of just appearing at conferences and giving speeches and talking to anyone who will listen to us about why this is so critical. And, quite frankly, we think that there's an ethical duty to do these studies because if you think about it in the civil protection order context, our view is that if we triaged more effectively, more people would have civil protection orders than currently do with no additional injection of resources. When you think about it that way, there may be an ethical duty to do these studies as opposed to "Oh, well, they're just curiosities."
We think, ultimately, that we're going to need to persuade funders to demand evidence, maybe not demand the evidence, "Show it to me right now or I'll cut your funding." Rather, say, "If you don't have the evidence, I want you to ... " This is a funder speaking to a legal services provider ... "I want you to engage in an evidence-gathering study. And, sure, I'll fund your services as long as you are gathering evidence about whether those actually work."
FOX:
Well, this is fascinating work. And thank you so much for agreeing to speak with us.
Again, I'm Aubrey Fox. I'm the guest host of the New Thinking podcast for the Center for Court Innovation and joined today by Professor James Greiner, who is a Professor of Law at Harvard University and the Faculty Director of The Access to Justice Lab. They have a great website that you can go online and take a look at some of the studies that they're participating in.
Thank you so much, Jim.
GREINER:
My pleasure. Thank you for the opportunity.