how to use empirical evidence

I’ve written a chapter for a forthcoming book edited by Harry Boyte (Democracy’s Education: A Symposium on Power, Public Work, and the Meaning of Citizenship, Vanderbilt University Press) in which I summarize evidence that colleges and universities can improve the economy by teaching their students civic skills and by being good institutional citizens, participating in local networks for community development.

I think the evidence is reasonably strong. But, like all empirical claims, it has exceptions and caveats. And even if civic education and engagement really do pay economic dividends, something else could work even better: for example, distance-learning, educational video games, or installing surveillance cameras in schools.

Thus we must be careful about how we generate, interpret and use empirical findings. This is where the emerging idea of Civic Studies provides guidance.

Often, social scientists presume that their job is to study a real-world practice that is already fully developed to learn whether—and why—it “works.” Usually they define success in terms of the objectives of the practitioners or their funders. In this case, we would ask whether college-level civic education and engagement generate what politicians demand: jobs.

But nothing simply “works.” Success always requires experimentation, assessment, adjustment, reflection, and new experimentation, in an iterative cycle. By the same token, many things can work if they are developed properly. One could start with civic engagement or with surveillance cameras in schools and improve either one until it enhanced students’ employment prospects.

In the best cases, the researchers who study a given practice are part of the reason that it works. They contribute to its development by offering their data and insights. They choose to work on this practice rather than something else because of a more fundamental commitment. I, for example, have studied deliberation. I want deliberation to work and I hope that the research that I produce will contribute to its success. I have no such commitment to surveillance cameras. I would not study them or strive to improve their impact.

The reason for my hope in deliberation is fundamentally moral. I think a world in which people reason and work together is better than one in which they achieve the same levels of security, income, or welfare without freely collaborating. Deeper down, I believe in a theory that the good life is a life of freedom, reflection, and mutual commitment.

Thus I hope that civic education and civic engagement boost employment because I am fundamentally committed to civic values. My colleagues and I seek evidence of economic benefit to persuade policymakers to support what they should support anyway. If the economic evidence is favorable, we will use it strategically to expand support. If not, our values and commitments should encourage us to improve civic education until it enhances democracy and also produces jobs. Regardless of the empirical results we find, we owe a public explanation of our core values.

A public defense of our values also yields criteria by which to assess the practices that we have been studying empirically. For instance, my chapter for Harry Boyte’s book is about college-level civic engagement (an input) and jobs (an output). I discuss the empirical link between the input and the output, as they exist today. But both are subject to criticism and change.

Today, many civic programs basically take the form of volunteering. But civic education can be reconceived so that it is less about volunteer service than about working on public concerns, where “working” implies serious commitment and accountability for results. “Public work,” in the phrase championed by Boyte and colleagues, means work that is done in public, by diverse citizens, on common issues. Reconfiguring civic education at the college level to look more like public work would satisfy core values that Boyte and colleagues have defended well. It might also strengthen the impact of civic education on jobs and careers. Students would be more likely to learn skills useful for employment if their civic experiences in college were more like paid work.

Meanwhile, jobs could become more public. A given job might serve only the interests of the employer and deny the worker any scope to address community problems in public with diverse other citizens. But even if the employer is a for-profit firm, the job can promote and encourage public work. For example, I presume that the corporate executives, government officials, and labor leaders who attended meetings of the Lehigh University board contributed insights from their daily work to the conversations about Lehigh and Allentown. They then brought ideas from those discussions back to their jobs. If that is true, they were doing “public work” in the Lehigh boardroom and in their own offices. Public work is obviously harder for low-paid service workers and low-ranking bureaucrats, but within many industries and professions, a struggle is underway to recover their public and democratic traditions.

If we made civic education into public work and also created jobs of greater public value, then the alignment between civic education and employment would be stronger and we would find more impressive evidence of economic impact. The data would then satisfy governors and presidents who want to see colleges produce jobs. More importantly, we would be building a better society and the educational system to support it.

The post how to use empirical evidence appeared first on Peter Levine.

assessment and accountabillity for civics

These are some notes for a presentation I will make later today at the New England Association of Schools & Colleges conference. NEASC is one of the six regional accrediting associations in the US. It works by “developing and applying standards, assessing the educational effectiveness of pre-school, elementary, middle, secondary, and postsecondary educational institutions.”

As measurement and accountability have become more important at all levels of education (from pre-K to graduate school), the measurement of civic outcomes has generally been forgotten. It is not clear that civic education has been dropped as a result. All states still have some kind of civic education requirement at the k-12 level. Most colleges still have programs that emphasize service or activism. However, levels of attention, innovation, and investment have clearly suffered because we do not measure civics very seriously.

Measuring anything valuable and complicated is a challenge, and trying to improve any form of education by imposing measures from the outside is always somewhat problematic. But measuring civic education raises special challenges:

  1. Civic engagement is intrinsically interpersonal. Being a citizen means relating to other citizens and to institutions. Measures of individual civic performance (such as multiple-choice tests, essays, or surveys of individual behavior) may miss the point altogether.
  2. Citizens engage on current issues that are often local. That means that the topics of their engagement vary and change rapidly. Standardized tests of civics–simply because they are standardized–must emphasize abstract and perennial questions (such as the US Constitution) and omit equally important current and local matters.
  3. Civic engagement can be either good or very bad, depending on the means, methods and objectives of the participants. Margaret Mead said, “Never doubt that a small group of thoughtful, committed citizens can change the world; indeed, it’s the only thing that ever has.” But Mussolini and his fellow fascists started as a small group of thoughtful and committed citizens. They changed the world for the worse. Measures of activity or impact that are value-free fail to distinguish between fascists and Freedom Riders.
  4. In many fields, we can decide what students should learn by assessing whether they are prepared to succeed in their chosen profession or in the labor market more generally. For instance, good engineering education makes good engineers, and good engineers are those who succeed in engineering jobs. Likewise, good citizens succeed in democracy and civil society. But what “success” as a citizen means is controversial. That is what radicals, liberals, conservatives, libertarians, patriots, cosmopolitans, Greens, and others argue about: what we owe to each other (and to nature and future generations) and how we should relate to the community and the state.
  5. When assessing education overall, it makes sense to ask whether it enhances the long-term well-being of the students, which can be measured in terms of earnings, health, or psychological flourishing. Some evidence suggests that being an engaged citizen boosts such outcomes. For instance, being able to define and address problems with peers is a civic skill that can also pay off in the labor market. Contributing to your community can make you happier. But the relationship between being an excellent citizen and flourishing as an individual is complex. In his great book Freedom Summer, Doug McAdam shows that the volunteers paid a severe personal price for their efforts to register Black voters in Mississippi in 1963. They were worse off than a comparison group in terms of happiness, career success, and health ten years later. That is no argument against the Freedom Summer program, which wasn’t meant for their benefit. It was one part of a glorious struggle against Jim Crow. To measure it in terms of the developmental benefits for the participants would have been a travesty.

I think it’s essential to measure civic education in an era of assessment and accountability–if only so that educators and students can track their own progress. Assessments must be interactive, not private and individual. Evaluation must consider ethics and values; it is not enough to act or to affect the world–you have to make it better. The question of what to measure is somewhat controversial because it relates to questions about what kind of society we should have. But there is a lot of common ground and room for compromise. In any event, we should decide what makes a good citizen not by asking what skills pay off in the marketplace or what civic activities boost students’ welfare. We must start with a theory of the good democratic society and then ask what skills, values, knowledge, and commitments we need from the next generation of citizens.

In my recent book, We Are the Ones We Have Been Waiting For, I argue that citizenship  fundamentally means: (1) deliberating with other citizens about what should be done, (2) actually working with other people to address problems and reflecting on the results, and (3) forming relationships of loyalty and trust. That theory derives from my study of politics, not primarily from a theory of education or youth development. I argue that the US political system depends on these three aspects of citizenship, all of which are in decline for deep, structural reasons. If I am right, these are the attainments that we should try to teach, and our measures should capture whether people can (1) deliberate, (2) collaborate, and (3) form civic relationships. If I am wrong, the counterargument should be a different theory of what our society needs from its people.

The post assessment and accountabillity for civics appeared first on Peter Levine.

MOOCs are old and shopworn

This is from a Connected Planet article in 1997:

Ah, spring – the time of year when students decide to skip classes en masse and sit outside enjoying the sun and fresh air. For the students of the University of Phoenix Online Campus, however, that ritual loses something in the translation: To duck their professors, all they have to do is turn off their PCs and unhook their modems.

But it’s a tradeoff that they’re willing to make in order to earn their undergraduate and graduate degrees on a part-time basis from the comfort of their own homes. The University of Phoenix opened its doors to its first 12 on-line students in 1989, and it now boasts 2500 students, 250 faculty members and eight degree program. …

However, one education industry analyst wonders how much credibility an on-line degree really has in the marketplace. “I would imagine there would be a bias against on-line degrees of any kind,” said Rick Hesel, principal at Art & Science Group. “Face-to-face contact with the faculty is considered to be a mark of quality, and because this program doesn’t have that, I think both employers and prospective students would be wary.”

But that could change soon, as the big names in education get into the on-line arena, Hesel said.

“Once you see Harvard or other prestigious MBA programs getting into it, all bets are off,” he said.

And Hesel believes that will be sooner rather than later.

Contrast that with the talk of a “MOOC Revolution” in (for instance) this 2103 Tom Friedman article. Friedman, like many others, presumes that MOOCs (massive open online courses) are very new, rapidly spreading, highly promising, originating in institutions like Stanford and Harvard with distinguished educators like Michael Sandel, and motivated by the goals of better and more accessible education. But, as Aaron Bady argues in Liberal Education, even the word “MOOC” is now almost six years old, and the basic practice dates to 1989. Even then, students were assigned to online discussion groups and showed videos of lectures. MOOCs did not originate at luminous, global intellectual powerhouses but at the University of Phoenix, which is now rapidly shrinking and faces widespread criticism for achieving a loan default rate higher than its graduation rate. Dispersion of the MOOC model has been slow and halting due to poor reputation and questionable impact. The prediction that “Harvard and other prestigious MBA programs” would soon adopt MOOCs turned out to be 16 years premature.

As Bady argues, there is no reason to rush to adopt MOOCs. We are not going through a “MOOC revolution.” Rather, we have extensive experience and it is not encouraging. To be sure, online courses have educational potential; a CIRCLE paper outlines some advantages. But we must avoid the hype. If college administrators were asked whether they wanted to implement the University of Phoenix’s 1989 model instead of Stanford’s latest MOOC, I doubt they would feel as excited.

(I take this overall argument from Bady, but I found the 1997 article quoted above.)

The post MOOCs are old and shopworn appeared first on Peter Levine.

the state humanities councils, connecting the public to scholarship

(Elon, NC) Elizabeth Lynn has published an important paper entitled State Councils, The Humanities, and the American Public. It tells the story of the formation of the National Endowment for the Humanities as a means to fund high scholarship, the almost accidental creation of state humanities councils (composed of laypeople as well as scholars), and how those councils helped save and strengthen the NEH from the grassroots up.

I contribute a relatively long preface that tells a story of its own. In brief summary, these are the stages in my story:

The humanities were invented by the ancient Sophists and then reinvented in the Renaissance to teach rhetoric, practical reasoning, and other skills for public life.

Professional humanists uncovered truths about the texts they studied that tended to reduce their immediate relevance to current public life. For example, they first mined classical history for models of virtue and wisdom, but the more they understood the past, the more complex, distant, and even irrelevant it seemed.

Humanism as professional expertise reached is apogee in Germany, and many of the greatest German scholars migrated to the United States because of Hitler. In the immediate post-war period, those exiles coexisted pretty comfortably in elite American universities with Anglophone public intellectuals who wrote appreciative essays on high culture for relatively broad audiences. Together, they produced scholarship that was widely respected and reasonably noncontroversial.

At around the same time, the federal government attained peak levels of public trust and frequently allocated public funds and decision-making power to specialized groups–military officers, business and union leaders, and scientists–who also had the public’s trust. Thus it was natural for Congress to appropriate funds for the humanities and turn the cash over to distinguished professional humanists in elite universities.

But all that collapsed as the public lost trust in government and specialized experts of all types, and as the calm consensus within the humanities gave way to intense and abstruse controversies, often with a political edge.

Today, even if you want to use public funds to support high scholarship in the humanities, you’ve got to think about strategies that tie scholarship to laypeople’s concerns. Elizabeth Lynn depicts the state humanities councils as means to that end. By the way, we are working with her and the Indiana state council (now known as “Indiana Humanities”) on an empirical study of the public humanities in that state. I hope it will demonstrate the breadth and robustness of the network.

The post the state humanities councils, connecting the public to scholarship appeared first on Peter Levine.

Publishers Bully a Digital Research Library

Book publishers love that libraries can act as free marketing venues, introducing readers to new authors and keeping them focused on books.  But publishers don’t like it when libraries act as commons – that is, when they promote easy access and sharing of knowledge.  A successful commons may modestly limit a publisher’s absolute copyright control – and even minor incursions on this authority must be stoutly resisted, publishers believe.     

One of the more egregious such battles now underway is a lawsuit filed by Harvard Business School Publishing, John Wiley and the University of Chicago Press against the Institute for the Study of Coherence and Emergence.  ISCE  is a small, nonprofit membership group that “facilitates the conversation between academics and business people regarding social complexity theory, particularly the implications for the management of organizations.” 

The focus of the publishers’ lawsuit is ISCE’s virtual library of 1,200 books.  May ISCE self-digitize and lend its virtual books to its members on a one-usage-at-a-time basis, for private, educational, non-commercial purposes? 

The publishers say no, and are seeking to establish their legal authority to shut down such unauthorized “reproduction, display and distribution” of the books.  But ISCE counter-claims that the fair use and first-sale doctrines of copyright law give it the legal right to lend its virtual books.  (Fair use is the legal doctrine of copyright law that allows excerpts to be shared noncommercially.  The first-sale doctrine prohibits the seller from controlling what a consumer does with a book or DVD after it is purchased, such as renting it, lending it or giving it away.)  ISCE claims, in addition, that libraries are entitled to special-use privileges under copyright law, which apply in this instance.

read more

A Welcome Focus on Latin American Commons

The Journal of Latin American Geography has dedicated an entire issue (vol. 12, no. 1) to surveying the state of commons on that continent. The special issue (in English) consists of nine essays, the first of which provides a helpful overview of the state of Latin American commons and commons research. (A listing of abstracts here.)  This academic treatment gives some welcome visibility and depth to the study of the commons in that vast region of the world, much of which is besieged by aggressive neoliberal policies that seek to extract vast natural resources in the name of "development." 

The Journal focuses on a range of commons-related themes in various countries, including the effect of rural out-migration from Mexico on commons there; new efforts in Costa Rica to treat biodiversity as a commons; the struggle of indigenous peoples in Brazil to secure tenure rights to their communal resources; and use of commons by marginalized people in Argentina to manage wild guanacos, a large, llama-like ungulate valued for their meat, skins and fibers.

The overview essay on current trends in Latin American commons research, by James Robson and Gabriela Lichtenstein, shines a light on the development agenda of oil and mining industries while noting the many legal and political changes that have reinstated communal property regimes.  Many countries, such as Brazil, Honduras, Venezuela and Nicaragua, have formally recognized the communal rights of indigenous communities to their traditional territories.  Overall, there is a “upturn in communal land tenure over time,” write Robson and Lichtenstein. 

read more

forthcoming in 2013: Civic Studies (the book)


This is a video of me (having a bad hair day) and some good friends making the case for the civic mission of higher education.

It is also an advertisement for the Civic Series, a set of short books on themes related to active citizenship and higher education. I am co-editing the volume entitled Civic Studies with Karol Soltan. It should be available by the end of 2013. The Table of Contents follows:

I. Overview

1. Peter Levine, “The Case for Civic Studies”
2. Karol Soltan, “The Emerging Field of a New Civics”
3. (multiple authors) “Framing Statement on Civic Studies”

II. The art and science of association: the Indiana Workshop

4. Filippo Sabetti, “Artisans of the Common Life: Building a Public Science of Citizens”
5. Paul Aligica, “Citizenship, Political Competence, and Civic Studies: the Ostromian Perspective”

III. Deliberative participation

6. Tina Nabatchi and Greg Munno, “Deliberative Civic Engagement: Connecting Public Voices to Public Governance”
7. Ghazala Mansuri and Vijayendra Rao, “The Challenge of Promoting Civic Participation in Poor Countries”

IV. Public work

8. Harry C. Boyte and Blase Scarnati, “The Civic Politics of Public Work”

V. Research engaged with citizens

9. Sanford Schram, “Citizen-Centered Research for Civic Studies: Bottom-Up, Problem-Driven, Mixed methods, Interdisciplinary”
10. Philip Nyden, “Public Sociology, Civic Education, and Engaged Research”

The post forthcoming in 2013: Civic Studies (the book) appeared first on Peter Levine.

a debate about the President’s higher ed proposals

The National Journal’s Fawn Johnson writes, “President Obama landed on some sweet talking points in his recent, somewhat rehashed, proposals to make colleges more affordable and more targeted on graduation and employment. “Higher education should not be a luxury,” Obama said in Syracuse, N.Y. “If a higher education is still the best ticket to upward mobility in America–and it is–then we’ve got to make sure it’s within reach.”

Johnson asks, “What’s not to like?”

I begin my invited response:

We in higher education deserve criticism for high costs and low graduation rates. But I have grave doubts about the goals and the solutions that President Obama proposes.

Consider two colleges. The first, which I will call “Harvard” (because that’s its real name), places almost all of its graduates in jobs or graduate schools. … The second college, which I will call “Local State,” enrolls students who live at home or off campus.  …. Completing a degree typically takes many years, if one manages to graduate at all. … Local State is the college we should subsidize and support.

The rest is on the National Journal’s Education Insider’s blog.

The post a debate about the President’s higher ed proposals appeared first on Peter Levine.

top ten signs you are an academic careerist

In The New Republic, Russell Jacoby names Stanley Fish as the academic who “raised careerism to a worldview.” “His writings incarnate the cheerful, expedient self-involvement that is part and parcel of contemporary life: everyone is out for himself. Fish has burnished this credo for the professoriate.”

I do not know if that is fair to Fish, but I do observe plenty of academic careerism. Here are ten signs of it:

  1. You want famous academics to know what you’ve done, but you don’t know or care what laypeople think about the topics you study.
  2. You can recite the professional achievements and setbacks of colleagues but don’t quite remember their arguments and findings.
  3. If you could continue to accumulate praise and rewards without learning anything new, you would stop learning.
  4. If you had a choice between a job where you could do better work and a job that had higher prestige, you would pick the latter.
  5. You are primarily interested in who holds each theory, not whether it is right. And you mainly select topics to study because prominent scholars are currently interested in them.
  6. You are most impressed by scholarly work that requires especially difficult techniques. You do not consider impact when you assess scholarship.
  7. You can explain what you know and how you know it, but not why it’s worth knowing.
  8. For you, a “good” university is one that attracts students and faculty who are already accomplished before they arrive.
  9. You think that fully successful students are those who become professors in your field.
  10. Like Fish, you don’t think taxpayers, students, and other laypeople have any right to judge your work.

It is a privilege to be paid to read, talk, and write. Many talented young people strive for a chance to join the academy but can’t find jobs. If you hold an academic position and have turned into a careerist, I believe you should quit and get out of the way.

The post top ten signs you are an academic careerist appeared first on Peter Levine.