I’m in Dallas, Texas for the the National Conference for Higher Education in Prison. Today I’ll be presenting a paper from a larger project on loyalty and social science research methods which draws on an argument I first encountered in Peter Levine’s work. Here’s a link to the PowerPoint of my talk.

It is fairly typical for those who work in college prison education to have arrived at this work through exposure to extraordinary classes of incarcerated students. That’s certainly the case for me: the students in my first class six years ago were so inspiring that I’ve kept doing it ever since. In that sense, we are motivated by loyalty. This may make us good teachers, organizers, and activists. But there’s some question whether it makes us good researchers.

When we publish we are sometimes asked to fill out a conflict of interest form. A typical conflict of interest disclosure for social science journals will ask for potentially biasing obligations we may have incurred through financial incentives, but also about potential bias from non-financial relationship, including personal relationships with students and programs we have helped to build:

The authors whose names are listed immediately below certify that they have NO affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational grants; participation in speakers’ bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements), or non-financial interest (such as personal or professional relationships, affiliations, knowledge or beliefs) in the subject matter or materials discussed in this manuscript.

This Elsevier form is a typical one. It’s quite short compared to similar disclosure forms that people in​ government are asked to fill out. In addition to the obvious sorts of things–such as climate change skepticism funded by a coal mining company–the form asks us to share our “personal or professional relationships, affiliations, knowledge, or beliefs.” There are, I think, two kinds of implications for such disclosures:

1. Mere knowledge of my “conflicts” is sufficient to warn the reader that my research may be biased.
2. The sources of conflict go well beyond my paycheck, to encompass my friendships, my preexisting beliefs, and ultimately my loyalties.

The evidence seems to clearly suggest that (1) is false: we are not very good at discounting the assertions of experts even when we know they may be biased. (For instance, we do not sufficiently doubt a mechanic when we know that he may prefer an expensive fix partly because he stands to profit from it.)

But we think the implications of (2) are important for social scientists to consider.  Our affiliations, friendships, and loyalties ultimately dictate the choices we make as researchers, our commitments and priors as we approach evidence, and provide something like a “stopping rule,” whereby we keep researching until we find evidence to fit these preferences. (See literature on motivated reasoning & skepticism, and perhaps also cultural cognition.) Thus we are potentially “conflicted” or “biased” by the very relationships that motivate our scholarship

Consider what this can mean:

• Education researchers usually enter the field with various preferred policy outcomes (perhaps related to race, gender, poverty, and unionization, but sometimes also preferred teaching environment). They know their work will be used by activists on both sides of various policy questions, and insofar as they continue to work in these areas they tend to want that to happen, whether it’s to promote universal pre-K programs, charter schools and vouchers, or protect collective bargaining and teacher tenure.
• Medical researchers typically have culturally-specific loyalties to fitness and health–which can cause them to overstate the risks of overweight and obese bodies, as well as overstating the prospects for weight-loss.
• Political scientists are usually well aware of the prospects for partisan bias–while ignoring their deeper affiliations to their own nation-state’s constitutional norms, such as judicial independence or bicameral legislature. Meanwhile political theorists often have a fundamental political orientation that guides them, like liberalism, conservatism, or even participatory–rather than merely representative–democracy.

I don’t think this kind of loyalty is biasing, though I’m happy to report it when required. In fact, I think it’s a kind of methodological superpower. Peter Levine expanded on this theme in a recent blog post, “Loyalty in Intellectual Work:”

“I’ve noticed that sometimes people expect me to endorse the underlying “theory of change” of a given field very strongly and are disappointed when I won’t. I usually cannot say that a given strategy or premise is the best one available, because I don’t really believe that. Instead, I think that a field or movement turns into what people make of it. So I see myself as a member who wants to make the movement as good as it can be, not as an independent scholar who has judged the movement and found it superior to others.”

Levine is a member in good standing of at least nine different scholarly communities and social movements, and as far as I can tell he has produced and disseminated useful scholarship in each. Yet he claims to have little attachment to the specific accounts of the world to which those communities seem to cling. This is only possibly because he rejects the cognitivist view of the joint endeavor: his loyalty is to the people, not the ideas.

On this approach, any active scholarly endeavor that is attached to, or feeds into, a political project on behalf of a group of people needs to hold its middle range theories relatively lightly. Our loyalties are to communities of practice and inquiry rather than to the reigning theory of etiology and efficacy that the community holds.

To me, this feels familiar. Prison education has a reigning theory of efficacy: the decreased recidivism rates of our incarcerated students. But there’s good reason to hold this view a bit lightly: our loyalty to incarcerated students themselves. If a particular theory of the efficacy of prison education is disproven, I don’t suddenly lose interest in working with prisoners. Indeed, I hope this happens frequently and rapidly in any field I participate in, because theories imply methods, and using the wrong method means we’re being ineffective.

Therefore, even though the body of evidence for prison education looks quite strong, it is not unassailable. We should anticipate that some of the most famous causal claims will come under fire, and we shouldn’t worry: we should seek new theories and work to clarify the old ones. Our loyalty is to the national community of incarcerated student scholars, not to the particular vision of education spelled out in the Three-State Recidivism Study or the RAND Study. I’d even willingly rethink the claims that Daniel Levine and I made in our own contribution to this literature given the right kind of evidence. Nor is our loyalty to particular students: the project of improving prison education nationally and ending mass incarceration has to take precedence over our loyalty to the men and women who are currently benefiting from the Second Chance Pell experiment, since this small group can make or break the program for the country.

Yet this is still a bias of sorts: we’ll always going to be looking for research that humanizes prisoners (thankfully, they are in fact our fellow human beings, so we’re unlikely to be in error there) and reduces our reliance on incarceration. If a strategy doesn’t work, we’re more likely to ask whether it can be tweaked or fixed than to abandon it.

And there’s good evidence that this is an important feature of social science research. One can evince a generic loyalty to a community of people affected by an overarching problem, but who are presented with conflicting narratives for resolving that problem. The researcher can then help adjudicate these explanations and theories of change. For instance, Kristie Dotson’s paper, “How is this paper philosophy?” proposes that disciplinary norms in philosophy departments and journals be altered to make room for work like hers by and in service of diverse practitioners. And in explaining her work, she celebrates the communities to which she belongs for inspiring it:

“I use philosophy to help support, generate and defend research, advocacy and activism that might change the current plight of Black people in the US, particularly promoting better conditions for Black cis- and trans* women, girls and gender non-conforming people. In other words, I am a Black feminist professional philosopher working in the service of Black feminist agendas.” (Philosop-her interview)

Loyalty to a community is not loyalty to a specific theory of change or efficacy, and in fact loyalty can motivate dissent from reigning theories in favor of alternatives. In prison education in particular, I think we need to worry about the sheepskin effect and the signaling theory of education: most of the college wage premium comes from completing school, rather than bit-by-bit along the way. That’s surprising: most skills are learned incrementally, and are beneficial in that incremental way. College, it seems, is all or nothing. Half or even 90% of a college degree does very little to increase your income, while finishing that last course makes a big difference.

Our anxiety should be that college seems to serve more as a signal of ability and conscientiousness than as training in necessary skills. The difference between someone who has a bachelor’s degree and someone who has 117 credits is quite small, in terms of knowledge. But the person who is unable or unwilling to finish his degree must have some incapacity or impediment. The degree is a signal that those elements are missing. Employers are paying for smart and hardworking staff, and a college degree is a reliable signal of those qualities. And indeed in college campuses throughout the country we see evidence that this is true: no one thinks that a cheater or a plagiarist is “only cheating himself,” they worry that he has an unfair advantage. The grade matters more than the work, it seems, which is also why students seek out “easy As” and rejoice when class is canceled. And many students readily engage in “cramming” for exams knowing that they will not retain the material in the long-term. (I owe these examples to Bryan Caplan, though they now seem almost too obvious to attribute.)

In this sense, then, we have to worry that prison education will, like education outside of prison, create a meritocratic hierarchy among prisoners. Rather than humanizing those behind the fence, it may sometimes have the effect of selecting a chosen few, “the exceptions.” So we should be willing to hold the various theories of education lightly.

• Perhaps prison education is valuable because it enhances signals of employability, reducing the stigma of incarceration.
• Perhaps prison education works to connect high status faculty with low status prisoners, and that association passes along cultural capital.
• Perhaps prison education works by identifying and raising the profile of certain organic intellectuals among people who are incarcerated. Or perhaps it simply allows the Du Bois’s Talented Tenth to rise.
• Perhaps education is a human right and we should ignore efficacy evidence for prison education.
• Perhaps prison education in the humanities and liberal arts teaches students important self-regulation and conflict resolution skills.
• Perhaps prison education provides important opportunities for deliberation over fundamental values which can lead to effective reprobation and rehabilitation of the moral injuries of a crime.
• Perhaps prison education is a part of a larger process of reframing people who are incarcerated through the lens of deficits to seeing them as assets to their community.
• Perhaps the soul knows no bars, and prison education is an important corrective to our overly punitive system of mass incarceration.

Loyalty requires to keep exploring these alternatives rather than rest easy with the RAND study. We’re better researchers because of it.