For those interested in tech-based citizen reporting tools (such as FixMyStreet, SeeClickFix), here’s a recent interview of mine with Jeffrey Peel (Citizen 2015) in which I discuss some of our recent research in the area.
picture by MyTudut on flickr
(NB: article originally published in Capital Finance International)
As citizen engagement gains traction in the development agenda, identifying the extent to which it produces tangible results is essential. Participatory budgeting, a process in which citizens decide upon and monitor budget allocation, offers promising results, including increased local government revenues and reduced infant mortality.
Promoting citizen engagement in the development community: a quest for evidence
In recent years there has been a growing interest in citizen engagement as a means to promote better development outcomes. The Open Government Partnership (OGP), for instance, is a multilateral platform where governments from 66 countries commit, among other things, to promote governments that are more open, participatory and accountable to their citizens. Similarly, Making all Voices Count is an international initiative supported by private donors and development agencies that provides funding to projects that promote “citizen engagement and open, responsive government.”
The rationale behind this renewed enthusiasm for civic engagement is seemingly simple: citizens know best what their needs are and how to address them. Or, as spelled out in the OGP declaration, public engagement “increases the effectiveness of governments, which benefit from people’s knowledge, ideas and ability to provide oversight.” Yet, the evidence on the benefits of citizen engagement often seems fuzzy, scattered and – sometimes – contradictory. However, a clearer picture emerges when we examine some particular practices that fall under the general “citizen engagement” umbrella, of which participatory budgeting is one. Originating from the Brazilian city of Porto Alegre in 1989, participatory budgeting (PB) can be broadly defined as the participation of citizens in the decision-making process of budget allocation and in the monitoring of public spending. Experts estimate that up to 2,500 local governments around the world have implemented PB, from major cities such as New York, Paris, Seville, and Lima, to small and medium cities in countries as diverse as Poland, South-Korea, India, Bangladesh, and the Democratic Republic of Congo.
Over the years, PB has attracted significant attention from scholars and development professionals. As it reaches over a quarter-century of existence, it is generating a substantial amount of evidence of the benefits of involving citizens in budgeting decisions. Here, we briefly examine some of this evidence.
Some argue – and there is growing evidence – that citizen participation increases government tax revenues
At the beginning of the 2000s, researchers studying participatory budgeting began to see an unexpected result, with some municipalities reporting substantive increases in their tax revenues. In 2004, for instance, a comparative study [PDF] of 25 municipalities in Latin America and Europe found a significant reduction in levels of tax delinquency after the adoption of participatory budgeting. But, in reality, how surprising were these findings?
Mostly unknown even among seasoned public engagement advocates, a growing body of evidence in the field of “tax morale” suggests a relationship between citizen participation and tax compliance. The argument, in an oversimplified manner, is as follows: citizens are more willing to pay taxes when they perceive that their preferences are properly taken into account by public institutions. This argument finds ever-growing empirical support. For instance, a number of studies in Switzerland – notably those by the economists Bruno Frey and Benno Torgler – show that Swiss cantons with higher levels of democratic participation present lower tax evasion rates, even when controlling for other factors. Suggesting that this is not simply a Swiss exception, a cross-national study by Friedrich Schneider and Désirée Teobaldelli found that “the effect of direct democratic institutions on the shadow economy is negative and quantitatively important.” These observational findings are increasingly supported by a growing number of controlled experiments across a variety of cultural settings. At odds with conventional economic reasoning, some evidence in the field of “tax morale” suggests that participation may be even more effective at curbing tax evasion than traditional and commonly adopted deterrence measures, such as fines and controls.
In the specific case of participatory budgeting, more robust data is also emerging. For example, a recent working paper by the Inter-American Development Bank presents similar effects of participatory budgeting on revenues in a randomized controlled trial in Russia. As noted by the authors, Diether Beuermann and Maria Amelina, these results are by no means negligible:
Implementing the planning cycle of participatory budgeting increased local revenues per capita by US$30.22 in regions without previous decentralized experience and by US$37.34 in regions with previous decentralized experience […] These are sizeable effects as they represent differences of around 70 percent with respect to the control group mean.
So participatory budgeting is good for tax revenues, but how good is it for citizens themselves?
Participatory budgeting promotes pro-poor spending, better access to services and may even reduce infant mortality
The available evidence suggests that participatory budgeting leads to significant shifts in priorities and policies, towards expenditures that directly benefit the poor. A 2008 World Bank report demonstrated that participatory budgeting has a statistically significant impact on a number of social indicators. Among others, the report highlights that PB is positively and strongly associated with improvements in poverty rates and access to water services.
Despite producing evidence of its effectiveness on a number of fronts over the years, only 25 years after its initial implementation in Brazil do we start to see systematic evidence of sound development outcomes. This is mainly due to two recently released, major studies of participatory budgeting in Brazil. The first, published by Sonia Gonçalves in World Development, finds that municipalities that adopted participatory budgeting in Brazil “favoured an allocation of public expenditures that closely matched the popular preferences and channeled a larger fraction of their total budget to key investments in sanitation and health services.” As a consequence, the author also finds that this change in the allocation of public expenditures “is associated with a pronounced reduction in the infant mortality rates for municipalities which adopted participatory budgeting.” Barely a year later, a study by Michael Touchton and Brian Wampler in Comparative Political Studies generated similar findings, demonstrating that the adoption of participatory budgeting in Brazil is strongly associated with increases in health care spending and decreases in infant mortality rates.
These studies also highlight another important takeaway for those working with development and public sector reform: the need to consider the fact that participatory institutions may take time to produce noticeable effects. As shown by Touchton and Wampler, for instance, the effects of PB adoption become significantly more visible after the fourth year of implementation.
As citizen engagement draws increasing interest in the development agenda, staying focused on which types of processes work and which do not will become particularly relevant. Participatory budgeting offers some promising evidence for policy reformers who want to see tangible impact on the ground, but it might take more than enthusiasm to get there. Determination, and a certain amount of patience, remain essential ingredients when it comes to delivering results.
The other day during a talk with researcher Tanya Lokot I heard an interesting story from Russia. Disgusted with the state of their streets, activists started painting caricatures of government officials over potholes.
In the case of a central street in Saratov, the immediate response to one of these graffiti was this:
Later on, following increased media attention – and some unexpected turnarounds – the pothole got fixed.
That reminded me of a recurrent theme in some conversations I have, which refers to whether praising and shaming matters to civic tech and, if so, to which extent. To stay with two classic examples, think of solutions such as FixMyStreet and SeeClickFix, through which citizens publically report problems to the authorities.
Considering government takes action, what prompts them to do so? At a very basic level, three hypothesis are possible:
1) Governments take action based on their access to distributed information about problems (which they supposedly are not aware of)
2) Governments take action due to the “naming and shaming” effect, avoiding to be publically perceived as unresponsive (and seeking praise for its actions)
3) Governments take action for both of the reasons above
Some could argue that hypothesis 3 is the most likely to be true, with some governments leaning more towards one reason to respond than others. Yet, the problem is that we know very little about these hypotheses, if anything. In other words – to my knowledge – we do not know whether making reports through these platforms public makes any difference whatsoever when it comes to governments’ responsiveness. Some might consider this as a useless academic exercise: as long as these tools work, who cares? But I would argue that the answer that questions matters a lot when it comes to the design of similar civic tech initiatives that aim to prompt government to action.
Let’s suppose that we find that all else equal governments are significantly more responsive to citizen reports when these are publically displayed. This would have importance both in terms of process and technological design. In terms of process, for instance, civic tech initiatives would probably be more successful if devoting part of their resources to amplify the visibility of government action and inaction (e.g. through local media). Conversely, from a technological standpoint, designers should devote substantive more effort on interfaces that maximizes praising and shaming of governments based on their performance (e.g. rankings, highlighting pending reports). Conversely, we might find that publicizing reports have very little effect in terms of responsiveness. In that case, more work would be needed to figure out which other factors – beyond will and capacity – play a role in government responsiveness (e.g. quality of reports).
Most likely, praising and shaming would depend on a number of factors such as political competition, bureaucratic autonomy, and internal performance routines. But a finer understanding of that would not only bear an impact on the civic tech field, but across the whole accountability landscape. To date, we know very little about it. Yet, one of the untapped potential of civic technology is precisely that of conducting experiments at lowered costs. For instance, conducting randomized controlled trials on the effects on the publicization of government responsiveness should not be so complicated (e.g effects of rankings, amplifying visibility of unfixed problems). Add to that analysis of existing systems’ data from civic tech platforms, and some good qualitative work, and we might get a lot closer at figuring out what makes politicians and civil servants’ “tick”.
Until now, behavioral economics in public policy has been mainly about nudging citizens toward preferred choices. Yet it may be time to start also working in the opposite direction, nudging governments to be more responsive to citizens. Understanding whether praising and shaming works (and if so, how and to what extent) would be an important step in that direction.
***
Also re-posted on Civicist.
This blog has been slow lately, but as I mentioned before, it is for a good cause. With some great colleagues I’ve been working on a series of papers (and a book) on civic technology. The first three of these papers are out. There is much more to come, but in the meantime, you can find below the abstracts and link to each of the papers. I also add the link to a presentation which highlights some other issues that we are looking at.
Does online voting mobilize citizens who otherwise would not participate? During the annual participatory budgeting vote in the southern state of Rio Grande do Sul in Brazil – the world’s largest – Internet voters were asked whether they would have participated had there not been an online voting option (i-voting). The study documents an 8.2 percent increase in total turnout with the introduction of i-voting. In support of the mobilization hypothesis, unique survey data show that i-voting is mainly used by new participants rather than just for convenience by those who were already mobilized. The study also finds that age, gender, income, education, and social media usage are significant predictors of being online-only voters. Technology appears more likely to engage people who are younger, male, of higher income and educational attainment, and more frequent social media users.
Read more here.
What effect does government responsiveness have on political participation? Since the 1940s political scientists have used attitudinal measures of perceived efficacy to explain participation. More recent work has focused on underlying genetic factors that condition citizen engagement. We develop a ‘Calculus of Participation’ that incorporates objective efficacy – the extent to which an individual’s participation actually has an impact – and test the model against behavioral data from FixMyStreet.com (n=399,364). We find that a successful first experience using FixMyStreet.com (e.g. reporting a pothole and having it fixed) is associated with a 54 percent increase in the probability of an individual submitting a second report. We also show that the experience of government responsiveness to the first report submitted has predictive power over all future report submissions. The findings highlight the importance of government responsiveness for fostering an active citizenry, while demonstrating the value of incidentally collected data to examine participatory behavior at the individual level.
Read more here.
In this project, we analyzed whether mobile phone-based surveys are a feasible and cost-effective approach for gathering statistically representative information in four low-income countries (Afghanistan, Ethiopia, Mozambique, and Zimbabwe). Specifically, we focused on three primary research questions. First, can the mobile phone survey platform reach a nationally representative sample? Second, to what extent does linguistic fractionalization affect the ability to produce a representative sample? Third, how effectively does monetary compensation impact survey completion patterns? We find that samples from countries with higher mobile penetration rates more closely resembled the actual population. After weighting on demographic variables, sample imprecision was a challenge in the two lower feasibility countries (Ethiopia and Mozambique) with a sampling error of /- 5 to 7 percent, while Zimbabwe’s estimates were more precise (sampling error of /- 2.8 percent). Surveys performed reasonably well in reaching poor demographics, especially in Afghanistan and Zimbabwe. Rural women were consistently under-represented in the country samples, especially in Afghanistan and Ethiopia. Countries’ linguistic fractionalization may influence the ability to obtain nationally representative samples, although a material effect was difficult to discern through penetration rates and market composition. Although the experimentation design of the incentive compensation plan was compromised in Ethiopia and Zimbabwe, it seems that offering compensation for survey completion mitigated attrition rates in several of the pilot countries while not reducing overall costs. These effects varied across countries and cultural settings.
Read more here.
Read more here.
For those willing to learn more about citizen engagement, here are two opportunities worth checking out.
The first one is the World Bank’s MOOC on Citizen Engagement. Even though the course has already started it is still possible to enroll. Here’s a brief description of the course:
The 4-week course brings together a diverse range of experts to provide students with a comprehensive overview of citizen engagement. It begins by synthesizing the theories and concepts that underlie citizen engagement, and goes on to explore how citizens can be engaged in both policymaking and public service delivery. Finally, it investigates how recent innovations are shaking up the field, through detailing both successes and failures of these new approaches. Our presenters, leaders in academia, government, and civil society, provide a wide range of perspectives and real-world experience to give participants a deeper understanding of whether citizen engagement can truly enhance the process of development. Participants will also have the opportunity to collaborate with one another and design their own citizen engagement initiatives, thereby putting theories learned in the course into practice.
This week starts module 2, with Matt Leighninger, Tina Nabatchi, Beth Noveck and myself:
This week explores the role that citizens can play in actively shaping public policy. We start by examining how citizens participate, analyzing the differences between ‘thick’ and ‘thin’ forms of engagement and asking strategic questions such as who should participate, how should participants interact with decision makers, what information do participants need, and how will participation impact policy decisions. Next, we survey examples of crowdsourcing and open innovation that are helping governments and citizens better interact. Finally, we unpack why citizens participate, moving beyond the mere calculation of costs and benefits described in the rational choice model to an analysis of broader factors that influence participation.
The second opportunity is the coaching program on Citizen Engagement by the GovLab Academy, with Beth Noveck and myself. Here’s the description of the program:
This program is designed to help those wishing to integrate citizen engagement into ongoing projects. Whether policymaking or service delivery in nature, we start from the assumption that, engaging citizens is both more effective and more legitimate as a way of working. Engagement may be offline as well as on and local or widely distributed. But, in every case, teams should have a clear sense of the problem they are trying to solve, the rationale for why they believe greater openness to and collaboration with citizens can have a positive impact, and a willingness to measure impact. Convened by two practitioners/theorists of citizen engagement, the program with emphasize peer-to-peer coaching and introductions to relevant mentors and experts from around the world working on related problems or applying similar methods. Our goal? To have take more citizen engagement projects from idea to implementation. Everyone is invited to apply. There will be an admissions preference for those working at the city-level.
There’s a number of other awesome courses provided by the GovLab: you can check all of them here.
pic by Tax Credits on flickr
Quite a while ago, drawing mainly from the literature on tax morale, I posted about the evidence on the relationship between citizen engagement and tax revenues, in which participatory processes lead to increased tax compliance (as a side note, I’m still surprised how those working with citizen engagement are unaware of this evidence).
Until very recently this evidence was based on observational studies, both qualitative and quantitative. Now we have – to my knowledge – the first experimental evidence that links citizen participation and tax compliance. A new working paper published by Diether Beuermann and Maria Amelina present the results of a randomized experiment in Russia, described in the abstract below:
This paper provides the first experimental evaluation of the participatory budgeting model showing that it increased public participation in the process of public decision making, increased local tax revenues collection, channeled larger fractions of public budgets to services stated as top priorities by citizens, and increased satisfaction levels with public services. These effects, however, were found only when the model was implemented in already-mature administratively and politically decentralized local governments. The findings highlight the importance of initial conditions with respect to the decentralization context for the success of participatory governance.
In my opinion, this paper is important for a number of reasons, some of which are worth highlighting here. First, it adds substantive support to the evidence on the positive relationship between citizen engagement and tax revenues. Second, in contrast to studies suggesting that participatory innovations are most likely to work when they are “organic”, or “bottom-up”, this paper shows how external actors can induce the implementation of successful participatory experiences. Third, I could not help but notice that two commonplace explanations for the success of citizen engagement initiatives, “strong civil society” and “political will”, do not feature in the study as prominent success factors. Last, but not least, the paper draws attention to how institutional settings matter (i.e. decentralization). Here, the jack-of-all-trades (yet not very useful) “context matters”, could easily be replaced by “institutions matter”.
You can read the full paper here [PDF].
Glasses for reading (1936) – Nationaal Archief
(I should have posted this on the 31st, but better late than never)
Below are some of the most read posts in 2014. While I’m at it, I’ll take the opportunity to explain the reduced number of posts in the last few months. Since mid-2014 I have been working with a small team of political and data scientists on a number of research questions at the intersection of technology and citizen engagement (I presented a few preliminary findings here). Following the period of field work, data collection and experiments, we have now started the drafting and peer-review stage of our research. This has been an extremely time-consuming process, which has taken up most of my weekends, when I generally write for this blog.
Still, one of my new year’s resolutions is precisely to better discipline myself to post more regularly. And I am hopeful that the publication of our upcoming research will make up for the recent reduction in posts. We will start to disseminate our results soon, so stay tuned.
In the meantime, here’s a selection of the five most read posts in 2014.
The Problem with Theory of Change
Technology and Citizen Engagement: Friend or Foe?
A Brilliant Story of Participation, Technology and Development Outcomes
When Citizen Engagement Saves Lives (and what we can learn from it)
Social Accountability: What Does the Evidence Really Say?
The Journal of Field Actions, together with Civicus, has just published a special issue “Stories of Innovative Democracy at the Local Level: Enhancing Participation, Activism and Social Change Across the World.” When put together, the 13 articles provide a lively illustration of the wealth of democratic innovations taking place around the world.
A little while ago I wrote about Jonathan Fox’s work on the evidence of social accountability initiatives. Initially in the format of a PDF slide presentation, it has now been turned into a magnificent paper, the first of the GPSA working paper series. Below is the abstract:
Policy discussion of social accountability initiatives has increasingly has increasingly focused on questions about their tangible development impacts. The empirical evidence is mixed. This meta-analysis rethinks some of the most influential evaluations through a new lens: the distinction between tactical and strategic approaches to the promotion of citizen voice to contribute to improved public sector performance. Field experiments tend to study bounded, tactical interventions that rely on optimistic assumptions about the power of information alone both to motivate collective action and to influence public sector performance. More promising results emerge from studies of multi-pronged strategies that encourage enabling environments for collective action and bolster state capacity to actually respond to citizen voice. This reinterpretation of the empirical evidence leads to a proposed new series of grounded propositions that focus on state-society synergy and sandwich strategies through which ‘voice’ and ‘teeth’ can become mutually empowering.
You can download the paper here: Social Accountability: What does the Evidence Really Say [PDF]. You can also read my take on the main lessons from Jonathan’s work here. Enjoy the reading.
***
PS: I have been away for a while doing field work, but hope to start posting (more or less) regularly soon.
A major argument for democratic governance is that more citizen participation leads to better outcomes through an improved alignment between citizens’ preferences and policies. But how does that play out in practice? Looking at the effects of the introduction of electronic voting (EV) in Brazil, a paper by Thomas Fujiwara (Princeton) sheds light on this question. Entitled “Voting Technology, Political Responsiveness, and Infant Health: Evidence from Brazil” (2013), it is one of the best papers I’ve read when it comes to bringing together the issues of technology, participation and development outcomes.
Below is an extract from the paper:
This paper provides evidence on how improving political participation can lead to better service outcomes. It estimates the effects of an electronic voting, or EV, technology in reducing a mundane, but nonetheless important, obstacle to political participation: difficulty in operating ballots. The results indicate that EV caused a large de facto enfranchisement of less educated voters, which lead to the election of more left-wing state legislators, increased public health care spending, utilization (prenatal visits), and infant health (birth weight).
While filling out a ballot may be a trivial task to educated citizens in developed countries, the same is not true in Brazil, where 23% of adults are “unable to read or write a simple note” and 42% did not complete the 4th grade. Moreover, before 1994 Brazilian paper ballots required voters to write a candidate’s name or electoral number and involved only written instructions. This resulted in a substantial quantity of error-ridden and blank ballots being cast, generating a large number of residual votes (not assigned to a candidate and discarded from the tallying of results).
In the mid-1990’s, the Brazilian government developed an EV technology as a substitute for paper ballots. While its introduction aimed at reducing the time and costs of voting counting, other features of the technology, such as the use of candidates’ photographs as visual aids, the use of “error” messages for voters about to cast residual votes, and guiding the voting process step by step, facilitated voting and reduced errors.
(…) Estimates indicate that EV reduced residual voting in state legislature elections by a magnitude larger than 10% of total turnout. Such effect implies that millions of citizens who would have their votes go uncounted when using a paper ballot were de facto enfranchised. Consistent with the hypothesis that these voters were more likely to be less educated, the effects are larger in municipalities with higher illiteracy rates. Moreover, EV raises the vote shares of left-wing parties.
The paper will go on to argue that this enfranchisement of the less educated citizenry did indeed affect public policy. (…) I focus on state government spending, in particular on an area that disproportionately affects the less educated: health care. Poorer Brazilians rely mostly on a public-funded system for health care services, which richer voters are substantially more likely to use the co-existing private services. The less educated have thus relatively stronger preferences for increased public health care provision, and political economy models predict that increasing their participation leads to higher public spending in this area.
Using data from birth records, I also find that EV raised the number of prenatal visits by women to health professionals and lowered the prevalence of low-weight births (below 2500g), and indicator of newborn health. Moreover, these results hold only for less educated mothers, and I find no effects for the more educated, supporting the interpretation that EV lead to benefits specifically targeted at poorer populations.
Fujiwara’s findings are great for a number of reasons, some of which I highlight below:
All of these points, added to the methodological approach adopted by Fujiwara, are good reasons to read the paper. You can find it here [PDF].
