A little while ago I wrote about Jonathan Fox’s work on the evidence of social accountability initiatives. Initially in the format of a PDF slide presentation, it has now been turned into a magnificent paper, the first of the GPSA working paper series. Below is the abstract:
Policy discussion of social accountability initiatives has increasingly has increasingly focused on questions about their tangible development impacts. The empirical evidence is mixed. This meta-analysis rethinks some of the most influential evaluations through a new lens: the distinction between tactical and strategic approaches to the promotion of citizen voice to contribute to improved public sector performance. Field experiments tend to study bounded, tactical interventions that rely on optimistic assumptions about the power of information alone both to motivate collective action and to influence public sector performance. More promising results emerge from studies of multi-pronged strategies that encourage enabling environments for collective action and bolster state capacity to actually respond to citizen voice. This reinterpretation of the empirical evidence leads to a proposed new series of grounded propositions that focus on state-society synergy and sandwich strategies through which ‘voice’ and ‘teeth’ can become mutually empowering.
A major argument for democratic governance is that more citizen participation leads to better outcomes through an improved alignment between citizens’ preferences and policies. But how does that play out in practice? Looking at the effects of the introduction of electronic voting (EV) in Brazil, a paper by Thomas Fujiwara (Princeton) sheds light on this question. Entitled “Voting Technology, Political Responsiveness, and Infant Health: Evidence from Brazil” (2013), it is one of the best papers I’ve read when it comes to bringing together the issues of technology, participation and development outcomes.
Below is an extract from the paper:
This paper provides evidence on how improving political participation can lead to better service outcomes. It estimates the effects of an electronic voting, or EV, technology in reducing a mundane, but nonetheless important, obstacle to political participation: difficulty in operating ballots. The results indicate that EV caused a large de facto enfranchisement of less educated voters, which lead to the election of more left-wing state legislators, increased public health care spending, utilization (prenatal visits), and infant health (birth weight).
While filling out a ballot may be a trivial task to educated citizens in developed countries, the same is not true in Brazil, where 23% of adults are “unable to read or write a simple note” and 42% did not complete the 4th grade. Moreover, before 1994 Brazilian paper ballots required voters to write a candidate’s name or electoral number and involved only written instructions. This resulted in a substantial quantity of error-ridden and blank ballots being cast, generating a large number of residual votes (not assigned to a candidate and discarded from the tallying of results).
In the mid-1990’s, the Brazilian government developed an EV technologyas a substitute for paper ballots. While its introduction aimed at reducing the time and costs of voting counting, other features of the technology, such as the use of candidates’ photographs as visual aids, the use of “error” messages for voters about to cast residual votes, and guiding the voting process step by step, facilitated voting and reduced errors.
(…) Estimates indicate that EV reduced residual voting in state legislature elections by a magnitude larger than 10% of total turnout. Such effect implies that millions of citizens who would have their votes go uncounted when using a paper ballot were de facto enfranchised. Consistent with the hypothesis that these voters were more likely to be less educated, the effects are larger in municipalities with higher illiteracy rates. Moreover, EV raises the vote shares of left-wing parties.
The paper will go on to argue that this enfranchisement of the less educated citizenry did indeed affect public policy. (…) I focus on state government spending, in particular on an area that disproportionately affects the less educated: health care. Poorer Brazilians rely mostly on a public-funded system for health care services, which richer voters are substantially more likely to use the co-existing private services. The less educated have thus relatively stronger preferences for increased public health care provision, and political economy models predict that increasing their participation leads to higher public spending in this area.
Using data from birth records, I also find that EV raised the number of prenatal visits by women to health professionals and lowered the prevalence of low-weight births (below 2500g), and indicator of newborn health. Moreover, these results hold only for less educated mothers, and I find no effects for the more educated, supporting the interpretation that EV lead to benefits specifically targeted at poorer populations.
Fujiwara’s findings are great for a number of reasons, some of which I highlight below:
Participation and policy preferences: The findings in this paper support the argument for democratic governance, showing that an increase in the participation of poorer segments of society ultimately leads to better service results.
Institutions and context: The paper indirectly highlights how innovations are intrinsically linked to institutions and their context. For instance, as noted by Fujiwara, “the effect of EV is larger in the proportional representation races where a paper ballot requires writing down the name or number of the candidate (lower chamber of congress and state legislature) than in the plurality races where a paper ballot involves checking a box (senate, governor, and president).” In other words, the electoral system matters, and the Brazilian outcomes would be most likely to be replicated in countries with similar electoral processes (and levels of ballot complexity), rather than those adopting plurality voting systems. (If I remember well, this was one of the findings of a paper by Daniel Hidalgo (unpublished), comparing the effects of e-voting in Brazil and India: the effects of e-voting for elections in the lower house in India [plurality vote] were smaller than in Brazil). In a similar vein, the effects of the introduction of similar technology would probably be lower in places with higher levels of educational attainment within poor segments of society.
Technology and elections: Much of the work on technology and accountability evolves around non-electoral activities that are insulated from existing processes and institutions, which tends to mitigate the chances of real-life impact. And, whether you like it or not, elections remain one of the most pervasive and consequential processes involving citizen participation in public affairs. There seems to be untapped potential for the use of technology to leverage electoral processes (beyond partisan campaigns). Finding ways to better inform voters (e.g. voting advice applications) and to lower the barriers for entry in electoral competition (why not a Rock the Votefor unlikely candidates?) are some of the paths that could be further explored. Fujiwara’s paper show how technology can enhance development outcomes by building on top of existing institutions.
Technology and inclusion: For a number of people working with development and public policy, a major concern with technology is the risk of exclusion of marginalized groups. While that is a legitimate concern, this paper shows the opposite effect, reminding us that it is less about technology and more about the use that one makes of it.
Unintended effects: The use of technology in governance processes is full of stories of unintended effects. Most of them are negative ones, epitomized by the case of digitization of land records in Bangalore [PDF]: instead of transparency and efficiency, it led to increased corruption and inefficiencies. Fujiwara’s paper shows that unexpected benefits are also possible. While the primary goal of the introduction of e-voting in Brazil was related to costs and time, another major unanticipated impact was better service outcomes. If unintended effects are often overlooked by practitioners and researchers alike, this paper highlights the need to look for effects beyond those originally intended.
All of these points, added to the methodological approach adopted by Fujiwara, are good reasons to read the paper. You can find it here [PDF].
Rio Grande do Sul Participatory Budgeting Voting System (2014)
Within the open government debate, there is growing interest in the role of technology in citizen engagement. However, as interest in the subject grows, so does the superficiality of the conversations that follow. While the number of citizen engagement and technology events is increasing, the opportunities for in-depth conversations on the subject do not seem to be increasing at the same rate.
This is why, a few weeks ago, I was pleased to visit the University of Westminster for a kick-off talk on “Technology and Participation: Friend or Foe?”, organized by Involve and the Centre for the Study of Democracy (Westminster). It was a pleasure to start a conversation with a group that was willing to engage in a longer and more detailed conversation on the subject.
My talk covered a number of issues that have been keeping me busy recently. On the preliminary quantitative work that I presented, credit should also go to the awesome team that I am working with, which includes Fredrik Sjoberg (NYU), Jonathan Mellon (Oxford) and Paolo Spada (UBC / Harvard). For those who would like to see some of the graphs better, I have also added here [PDF] the slides of my presentation.
I have skipped the video to the beginning of my talk, but the discussion that followed is what made the event interesting. In my opinion, the contributions of Maria Nyberg (Head of Open Policy Making at the Cabinet Office) Catherine Howe (Public-i), as well as those of the participants, were a breath of fresh air in the current citizen engagement conversation. So please bear with me and watch until the end.
I would like to thank Simon Burral (Involve) and Graham Smith (Westminster) for their invitation. Simon leads the great work being done at Involve, one of the best organizations working on citizen engagement nowadays. And to keep it short, Graham is the leading thinker when the issue is democratic innovations.
Below is also an excellent summary by Sonia Bussu (Involve), capturing some of the main points of my talk and the discussion that ensued (originally posted here).
***
“On technology and democracy
The title of yesterday’s event, organised by Involve and Westminster University’s Centre for the Study of Democracy, posed a big question, which inevitably led to several other big questions, as the discussion among a lively audience of practitioners, academics and policymakers unfolded (offline and online).
Tiago Peixoto, from the World Bank, kicked off the debate and immediately put the enthusiasm for new technologies into perspective. Back in 1795, the very first model of the telegraph, the Napoleonic semaphore, raised hopes for – and fears of – greater citizen engagement in government. Similarly the invention of the TV sparked debates on whether technology would strengthen or weaken democracy, increasing citizen awareness or creating more opportunities for market and government manipulation of public opinion.
Throughout history, technological developments have marked societal changes, but has technological innovation translated into better democracy? What makes us excited today about technology and participation is the idea that by lowering the transaction costs we can increase people’s incentives to participate. Tiago argued that this costs-benefits rationale doesn’t explain why people continue to vote, since the odds of their vote making a difference are infinitesimal (to be fair voter turnouts are decreasing across most advanced democracies – although this is more a consequence of people’s increasing cynicism towards political elites rather than their understanding of mathematical probabilities).*
So do new technologies mobilise more people or simply normalise the participation of those that already participate? The findings on the matter are still conflicting. Tiago showed us some data on online voting in Rio Grande do Sul participatory budgeting process in Brazil, whereby e-voting would seem to bring in new voters (supporting the mobilisation hypothesis) but from the same social strata (e.g. higher income and education – as per the normalisation hypothesis).
In short, we’re still pretty much confused about the impact of technology on democracy and participation. Perhaps, as suggested by Tiago and Catherine Howe from Public-i, the problem is that we’re focusing too much on technology, tempted by the illusion it offers to simplify and make democracy easy. But the real issue lies elsewhere, in understanding people and policymakers’ incentives and the articulation (or lack thereof) between technologies and democratic institutions. As emphasised by Catherine, technology without democratic evolution is like “lipstick on a pig”.
The gap between institutions and technology is still a big obstacle. Catherine reminded us how participation often continues to translate into one-way communication in government’s engagement strategies, which constrains the potential of new technologies in facilitating greater interaction between citizens and institutions and coproduction of policies as a response to increasing complexity. As academics and practitioners pitch the benefits of meaningful participation to policy makers, Tiago asked whether a focus on instrumental incentives might help us move forward. Rather than always pointing to the normative argument of deepening democracy, we could start using data from cases of participatory budgeting to show how greater participation reduces tax evasion and corruption as well as infant mortality.
He also made a methodological point: we might need to start using more effectively the vast array of data on existing engagement platforms to understand incentives to participation and people’s motivation. We might get some surprises, as findings demystify old myths. Data from Fix My Street would seem to prove that government response to issues raised doesn’t increase the likelihood of future participation by as much as we would assume (28%).** But this is probably a more complicated story, and as pointed out by some people in the audience the nature and salience of both the issue and the response will make a crucial difference.
Catherine highlighted one key problem: when we talk about technology, we continue to get stuck on the application layer, but we really need to be looking at the architecture layer. A democratic push for government legislation over the architecture layer is crucial for preserving the Internet as a neutral space where deeper democracy can develop. Data is a big part of the architecture and there is little democratic control over it. An understanding of a virtual identity model that can help us protect and control our data is key for a genuinely democratic Internet.
Maria Nyberg, from the Cabinet Office, was very clear that technology is neither friend nor foe: like everything, it really depends on how we use it. Technology is all around us and can’t be peripheral to policy making. It offers great opportunities to civil servants as they can tap into data and resources they didn’t have access to before. There is a recognition from government that it doesn’t have the monopoly on solutions and doesn’t always know best. The call is for more open policy making, engaging in a more creative and collaborative manner. Technology can allow for better and faster engagement with people, but there is no silver bullet.
Some people in the audience felt that the drive for online democracy should be citizen-led, as the internet could become the equivalent of a “bloodless guillotine” for politicians. But without net neutrality and citizen control over our own data there might be little space for genuine participation.
*This point was edited on 12/07/2014 following a conversation with Tiago.
** This point was edited on 12/07/2014 following a conversation with Tiago.”
—————————
I am also thankful to the UK Political Studies Association (PSA), Involve and the University of Westminster for co-sponsoring my travel to the UK. I will write more later on about the Scaling and Innovation Conference organized by the PSA, where I was honored to be one of the keynote speakers along with MP Chi Onwurah (Shadow Cabinet Office Minister) and Professor Stephen Coleman (Leeds).
A little while ago I mentioned the launch of the Portuguese version of the book organized by Nelson Dias, “Hope for Democracy: 25 Years of Participatory Budgeting Worldwide”.
The good news is that the English version is finally out. Here’s an excerpt from the introduction:
This book represents the effort of more than forty authors and many other direct and indirect contributions that spread across different continents seek to provide an overview on the Participatory Budgeting (PB) in the World. They do so from different backgrounds. Some are researchers, others are consultants, and others are activists connected to several groups and social movements. The texts reflect this diversity of approaches and perspectives well, and we do not try to influence that.
(….)
The pages that follow are an invitation to a fascinating journey on the path of democratic innovation in very diverse cultural, political, social and administrative settings. From North America to Asia, Oceania to Europe, from Latin America to Africa, the reader will find many reasons to closely follow the proposals of the different authors.
While my perception may be biased, I believe this book will be a major contribution for researchers and practitioners in the field of participatory budgeting and citizen engagement in general. Congratulations to Nelson Dias and all the others who contributed their time and energy.
So what does the evidence about citizen engagement say? Particularly in the development world it is common to say that the evidence is “mixed”. It is the type of answer that, even if correct in extremely general terms, does not really help those who are actually designing and implementing citizen engagement reforms.
This is why a new (GPSA-funded) work by Jonathan Fox, “Social Accountability: What does the Evidence Really Say” is a welcome contribution for those working with open government in general and citizen engagement in particular. Rather than a paper, this work is intended as a presentation that summarizes (and disentangles) some of the issues related to citizen engagement.
Before briefly discussing it, some definitional clarification. I am equating “social accountability” with the idea of citizen engagement given Jonathan’s very definition of social accountability:
“Social accountability strategies try to improve public sector performance by bolstering both citizen engagement and government responsiveness”
In short, according to this definition, social accountability is defined, broadly, as “citizen participation” followed by government responsiveness, which encompasses practices as distinct as FOI law campaigns, participatory budgeting and referenda.
But what is new about Jonathan’s work? A lot, but here are three points that I find particularly important, based on a very personal interpretation of his work.
First, Jonathan makes an important distinction between what he defines as “tactical” and “strategic” social accountability interventions. The first type of interventions, which could also be called “naïve” interventions, are for instance those bounded in their approach (one tool-based) and those that assume that mere access to information (or data) is enough. Conversely, strategic approaches aim to deploy multiple tools and articulate society-side efforts with governmental reforms that promote responsiveness.
This distinction is important because, when examining the impact evaluation evidence, one finds that while the evidence is indeed mixed for tactical approaches, it is much more promising for strategic approaches. A blunt lesson to take from this is that when looking at the evidence, one should avoid comparing lousy initiatives with more substantive reform processes. Otherwise, it is no wonder that “the evidence is mixed.”
Second, this work makes an important re-reading of some of the literature that has found “mixed effects”, reminding us that when it comes to citizen engagement, the devil is in the details. For instance, in a number of studies that seem to say that participation does not work, when you look closer you will not be surprised that they do not work. And many times the problem is precisely the fact that there is no participation whatsoever. False negatives, as eloquently put by Jonathan.
Third, Jonathan highlights the need to bring together the “demand” (society) and “supply” (government) sides of governance. Many accountability interventions seem to assume that it is enough to work on one side or the other, and that an invisible hand will bring them together. Unfortunately, when it comes to social accountability it seems that some degree of “interventionism” is necessary in order to bridge that gap.
Of course, there is much more in Jonathan’s work than that, and it is a must read for those interested in the subject. You can download it here [PDF].
I have been doing some research on works related to the evaluation of citizen engagement initiatives (technology mediated or not). This is far from exhaustive, but I thought it would be worth sharing with those who stop by here. Also, any help with identifying other relevant sources that I may be missing would be greatly appreciated.
Whenever discussing participation in Africa (as well as in other developing contexts) the issue of “who participates” often emerges. A constant in these conversations is an assumption that participation in the continent is strongly driven by material and immaterial resources (e.g. money, time). In other words, there seems to be a widespread belief, particularly among development practitioners, that the most privileged sectors of society are disproportionately more likely to participate than the least well-off.
In part, such an assumption is empirically grounded. Since the early 70s, studies have shown inequality in political participation, with the most advantaged groups being disproportionately more likely to participate. Considering that policy preferences between groups differ, unequal participation leads to the troubling possibility that public policy is skewed towards favoring the better off, thus further deepening societal differences and access to public goods and services.
However, often ignored is the fact that most of these studies refer to participation in traditional western democracies, notably the United States and European countries. But do these results hold true when it comes to participation in Africa? This is the question that Ann-Sofie Isaksson (University of Gothenburg) tackles in a paper published in Electoral Studies “Political participation in Africa: The role of individual resources”.
By looking at an Afrobarometer dataset of 27,000 respondents across 20 African countries, Isaksson’s findings challenge the standard thinking on the relationship between resources and participation:
(…) it seems the resource perspective does a poor job at explaining African political participation. If a resource is relevant for meeting the costs of participating, more of that resource should mean more participation. If anything, however, the estimations suggest that having little time (i.e. working full-time) and little money (i.e. being poorer) is associated with more participation.
Isaksson’s analysis is not confined to participation in elections, also examining non-electoral participation, i.e. attending community meetings. With regard to the latter only, there are modest effects associated with exposure to information (e.g. radio, TV, newspapers) and education. Yet, as the author notes, “the result that community attendance is higher among the poor remains”.
To conclude, as underlined by Isaksson in her paper, she is not alone in terms of findings, with other studies looking at Asia and Latin America pointing in a similar direction, slowly dispelling the myth of the role of resources for participation in developing countries. Development practitioners should start to take note of these encouraging facts.
***
P.s.: An earlier ungated version of the paper can be found here [PDF].
“A Review of the Evidence on Open Budgeting” is a recent report by the World Bank Institute’sCapacity Development and Results team. It explores key questions and existing evidence around the impact of open budgeting. Despite the growing body of literature, there remains limited substantiation for whether and how open budgeting contributes to reductions in poverty and improvements in the lives of the poor. This report pieces together the results chain presenting evidence for and against from the literature. It explores links between open budgeting and indicators of impact such as human development and public service delivery. The findings highlight the importance of measuring budget transparency, accountability, and participation. The findings show that the impact of institutional changes differ under varying conditions in specific contexts. The conclusions of the report point to the need for further investigation into impact and establishing effective measurement practices for monitoring related institutional change under varying conditions and different contexts.”
A report by Brian Wampler for the Global Initiative for Fiscal Transparency (GIFT):
Citizen participation in budgetary and other fiscal processes has been expanding at international, national, and local levels over the past 15 years. The direct participation of citizens, it is hoped, will improve governance, limit misuse of public funds, and produce more informed, engaged citizens. At the national level, reformist governments now encourage the direct engagement of citizens during multiple moments of the policy cycle—from initial policy formulation to the oversight of policy implementation. Reformist governments hope to take advantage of increased citizen participation to increase their legitimacy, thus allowing them to change spending and policy priorities, increase state effectiveness by make public bureaucrats more responsive to citizens and elected officials, and, finally, ensure that the quality of public services improves. During the 1980s and 1990s, many subnational governments took advantage of policy decentralization to experiment with new institutional types. Direct citizen participation has been most robust at subnational levels due to the decreased costs and the greater direct impact of citizens on policymaking.
(….)
The main purpose of this report is to examine how three countries, South Korea, Brazil, and the Philippines, have made extensive efforts to create new institutions and policies that encourage the participation of citizens and CSOs in complex policy processes. South Korea developed an institutional arrangement based on policy experts, CSOs, and the Korean Development Institute. Brazil uses a model that relies extensively on the participation of citizens at multiple tiers of government. Finally, the Philippines use a mixed model that incorporates citizens and CSOs at national and subnational levels
(….)
Political reformers seeking to incorporate greater numbers of people into policymaking venues face a series of challenges. These include: (1) asymmetrical access to information as well as differing skills base to interpret information; (2) the difficultly of decision-making when groups grow in size; (3) a reduction in the importance of any single participant due to the greater number of participants; (4) political contestation over who has the right to participate; (5) who are the legitimate representatives of different groups; and (6) higher organizational costs (time, money, personnel). This report maps out how new participatory institutions and programs that are designed to help governments and their civil society allies draw citizens directly into decision-making processes.To explain the variation in the type of participatory experiences now used by different countries,we identify four factors that most strongly affect the types of participation-oriented reforms as well as the results. These four factors include: (a) presidential-level support for reform, (b) the configuration of civil society, (c) state capacity and (d) the geo-political direction of reform (topdown/center –periphery vs. bottom-up/periphery/center. It is the combination of these four factors that most strongly explains the type of institutions adopted in each of these countries.
When it comes to the relationship between participatory institutions and development outcomes, participatory budgeting stands out as one of the best examples out there. For instance, in a paper recently published in World Development, Sonia Gonçalves finds that municipalities that adopted participatory budgeting in Brazil “favoured an allocation of public expenditures that closely matched the popular preferences and channeled a larger fraction of their total budget to key investments in sanitation and health services.” As a consequence, the author also finds that this change in the allocation of public expenditures “is associated with a pronounced reduction in the infant mortality rates for municipalities which adopted participatory budgeting.”
Evolution of the share of expenditures in health and sanitation compared between adopters and non-adopters of participatory budgeting (Goncalves 2013).
Now, in an excellent new article published in Comparative Political Studies, the authors Michael Touchton and Brian Wampler come up with similar findings (abstract):
We evaluate the role of a new type of democratic institution, participatory budgeting (PB), for improving citizens’ well-being. Participatory institutions are said to enhance governance, citizens’ empowerment, and the quality of democracy, creating a virtuous cycle to improve the poor’s well-being. Drawing from an original database of Brazil’s largest cities over the last 20 years, we assess whether adopting PB programs influences several indicators of well-being inputs, processes, and outcomes. We find PB programs are strongly associated with increases in health care spending, increases in civil society organizations, and decreases in infant mortality rates. This connection strengthens dramatically as PB programs remain in place over longer time frames. Furthermore, PB’s connection to well-being strengthens in the hand of mayors from the nationally powerful, ideologically and electorally motivated Workers’ Party. Our argument directly addresses debates on democracy and well-being and has powerful implications for participation, governance, and economic development.
When put together, these findings provide compelling evidence for those who – often unfamiliar with the literature – question the effectiveness of participatory governance institutions. Surely, more research is needed, and different citizen engagement initiatives (and contexts) may lead to different results.
But these articles also bring another important takeaway for those working with development and public sector reform. And that is the need to consider the fact that participatory institutions (as most institutional reforms) may take time to produce desirable/noticeable effects. As noted by Touchton and Wampler:
The relationships we describe between PB and health and sanitation spending, PB and CSOs, and PB and health care outcomes in this section are greater in magnitude and stronger in statistical significance for municipalities that have used PB for a longer period of time. Municipalities using PB for less than 4 years do exhibit lower infant mortality rates than municipalities that never adopted PB. However, there is no statistically significant difference in spending on health care and sanitation between municipalities using PB for less than 4 years and municipalities that never adopted the program. This demonstrates the benefits from adopting PB are not related to low-hanging fruit, but built over a great number of years. Our results imply PB is associated with long-term institutional and political change—not just short-term shifts in funding priorities .
If throughout the years participatory budgeting has produced evidence of its effectiveness on a number of fronts (e.g. pro-poor spending), it is only 25 years after its first implementation in Brazil that we start to see systematic evidence of sound development outcomes such as reduction in infant mortality. In other words, rushing to draw conclusions at early stages of participatory governance interventions may result in misleading assessments. Even worse, it may lead to discontinuing efforts that are yet to bear fruit in the medium and longer terms.
