I’m back from a meeting of people who practice and advocate mixed-methods research (research that integrates quantitative and qualitative data). They have identified barriers or biases against such work. Editors and reviewers tend to be either quantitative or qualitative experts, journals impose tight word limits that are frustrating if you want to describe two complementary methods, and so on.

A more general question is how any type of scholarship gains legitimacy. I have observed several efforts to legitimize new methods, such as Community Based Participatory Research (CBPR) and Participant Action Research (PAR), as well as defenses of older methods that are being squeezed out, such as philosophical argumentation within the discipline of political science.

It is worth considering what gives scholarly methods legitimacy in the first place. I would offer a roughly Weberian theory. For Weber, “modernity” means secularization and specialization. Under those two conditions:

1. It pays to demonstrate a specialized skill or capacity, because desirable social roles are now doled out to specialists—not (or at least not officially) to people who have social rank and pedigree.
2. Specialists not only receive, but they also need, tools and methods that require scarce resources. A particle physicist needs a supercollider, to name an extreme example. If you can’t get access to the necessary instruments, you can’t practice the trade.
3. The society as a whole lacks confident, consensus beliefs about ethics, metaphysics, epistemology, and aesthetics (the classic four pillars of philosophy). But you can’t talk or think very well without having beliefs about those matters, and it’s difficult to justify them satisfactorily to people who disagree. Therefore, we make routine progress within smaller communities that share beliefs or that may even be defined by their shared beliefs.

You can see the result of these conditions in the development of academic fields. For instance, classicists used to be numerous and influential in universities. They possessed specialized skills: fluency in Greek and Latin and the experience of having already read all the major ancient classics at least once. These texts were widely believed to be better (ethically and aesthetically) than most or all of what had been written since. However, whole categories of people could not read them; for instance, very few women were taught Greek or Latin. They certainly couldn’t see the rare ancient manuscripts needed for the philological work of establishing authentic texts. Thus, being a classicist was rewarded with status and with scare resources, such as access to teaching jobs and libraries.

As the Greco-Roman classics receded in importance and lost their privileged place in the culture, people began to want to study modern literature. But virtually everyone in a country like England could read works in English. How then could English literature professors justify their special social role? One step was to develop a canon of difficult works that could claim to be as valuable as the ancient classics were. Having read Shakespeare and Milton set you apart a bit. Another step was to introduce philology and epigraphy to the study of modern texts. (That also required direct access to manuscripts and rare printed volumes). And a third step was to develop specialized and non-intuitive ways of reading, such as by applying theoretical frameworks.

Already in the eighteenth century, the editors of The Literary Magazine could claim legitimacy on the basis of specialization: “a selection has been made of men qualified for the different parts of the work, and each has the employment assigned him, which he is supposed most able to discharge” (quoted in Kramnick 2002).

At that time, there was still considerable consensus about values. In modernity, however, ethical, metaphysical, epistemological, and aesthetic values are seen as controversial and perhaps culturally relative. Fortunately, you needn’t justify a given philosophical premise in order to write an ordinary work of literary criticism today; you can just cite a major theorist who has been deemed legitimate within the scholarly community. Names of theorists become tokens that justify premises, much as scripture might justify spiritual assumptions within a traditional religion.

This is a purely external, sociological explanation of the development of modern literary criticism. I believe that the discipline yields valuable insights, so I welcome its development. Indeed, if literary criticism produced little public value, it might collapse. Specialized occupations need public support in the long run. Still, a Weberian perspective allows us to identify specialization and a reliance on canonical theorists as two responses to modernity, irrespective of whether the resulting scholarship is any good.

Most disciplines have used these means to capture scarce positions that bring status and resources.

• Many natural and some social sciences use advanced mathematical techniques, which are difficult to learn. Physics and economics enjoy relative prestige in part because they use harder math than kindred disciplines do.
• Many natural scientists need expensive instruments.
• Ethnographers seem at first to be doing what anyone can do—observing human beings in their settings. But if you have done fieldwork in an isolated village in the global South, you have bona fides to be an ethnographer instead of a layperson.
• Quantitative social science requires not only math skills but also large-n data, which is expensive to collect.
• Qualitative researchers who achieve inter-rater reliability among numerous observers have the budgets and institutional support to hire and train those observers.
• Some humanistic research requires access to rare objects.
• Some practitioners of CPBR and PAR have social capital and cultural fluency in both academia and in highly disadvantaged communities. Their ability to code-switch sets them apart.

Within these communities, certain philosophical premises are typically shared. For instance, in most of the social sciences (both qualitative and quantitative), a moral value is something that a person or group holds and that has causes and consequences. It is not something that can be shown to be right or wrong, better or worse. However, a belief about the divine is incompatible with science and thus (implicitly) false. Among theologians, obviously, both of those assumptions are widely rejected. You have to be a kind of moral relativist to speak the language of social science, but that is a minority position in philosophy and theology.

Under such conditions, an approach like mixed-methods research struggles for legitimacy. Perhaps integrating quantitative and qualitative data would yield the most reliable findings under a range of common circumstances. However, the Weberian logic of modernity encourages some researchers to maximize their specialization in math, others to maximize their specialization in ethnography; and mixed methods fall uncomfortably in between.

One solution is a Weberian judo move: as experts criticize your lack of expertise, use their momentum against them by defining what you do as a difficult new specialization. That was one tactic recommended in the conversation about mixed methods. I find it more interesting to think about ways to combat the harmful consequences of modernity in intellectual life so that we begin to assign legitimacy differently. Obviously, the ideal way would be to reward solutions to public (including cultural) problems, rather than academic methods for their own sake. But that is a hard shift to accomplish.

[Citing Jonathan Brody Kramnick, “Literary Criticism Among the Disciplines,” Eighteenth-Century Studies, Volume 35, Number 3, Spring 2002, pp. 343-360. See also the future of classics and why social scientists should pay attention to metaphysics.]

(Los Angeles) Academics and scholars most commonly relate to institutions, movements, or fields of practice by assessing them. They identify the underlying theory or rationale of a given practical effort and assess its plausibility and its consistency with principles of justice. They also observe the actual performance of the practice to date and render judgments about success or failure.

Since my undergraduate days, I’ve instinctively adopted a different stance toward fields of practice. I’ve seen them basically as groups of people. I’ve never taken their theories completely seriously, because I expect them to evolve. And I’ve never seen the empirical data about success or failure so far as dispositive, because I assume that efforts will fail until they are refined and improved. You can start from many premises and get good results if you are open to reflection and change. The theory is less important than it seems.

Fields of practice are working communities of people who are either worth joining or not. What inclines me to want to join a group is a sense of its members’ motivations (in a very general sense) and their capacity or potential. Once I feel that I’m part of the group, I adopt a stance of loyalty. That doesn’t prevent me from making critical comments, either privately or publicly, if that seems helpful to the cause, but it does pose a question about any possible communication: is it helpful?

In this general mode, I’ve found myself part of the following fields or movements since my undergraduate days in the late 1980s:

• service-learning
• public deliberation and dialogue
• university/community partnerships
• campaign finance reform
• public or civic journalism
• k-12 civic education
• relational community organizing
• certain political campaigns
• Action Civics

Clearly, these efforts share some principles or norms. Of the enormous variety of projects and groups that are active around us, most wouldn’t appeal to me as much as these. In We Are the Ones We Have Been Waiting For, I tried to analyze and defend the norms underlying the fields that I most admire in generic terms. Still, I don’t go around looking for movements that match all these principles. Instead, I tend to join movements that seem appealing and then try to reflect on their emergent principles.

Relating to fields of practice in this way sometimes causes misunderstandings. I’ve noticed that sometimes people expect me to endorse the underlying “theory of change” of a given field very strongly and are disappointed when I won’t. I usually cannot say that a given strategy or premise is the best one available, because I don’t really believe that. Instead, I think that a field or movement turns into what people make of it. So I see myself as a member who wants to make the movement as good as it can be, not as an independent scholar who has judged the movement and found it superior to others.

See also:  loyalty to place in the age of jet-set academia; bringing loyalty back; Albert O. Hirschman on exit, voice, and loyalty; and “Seeing Like a Citizen: The Contributions of Elinor Ostrom to ‘Civic Studies‘” (because I see Ostrom as having a similar stance).