The 29-page toolkit, 15 Key Metrics for Evaluating Participatory Budgeting: A Toolkit for Evaluators and Implementers, was created in collaboration with Public Agenda
(PA), the Participatory Budgeting Project
(PBP), and the North American Research Board. The toolkit includes 15 metrics to capture major elements of a PB process, to support the evaluation of a specific PB site and use the data to inform the larger PB movement.
Below is an excerpt from the toolkit, which you can find in full PDF, both in English and in Spanish on Public Agenda’s site here.
From Public Agenda…
Evaluation is a critical component of any PB effort. Systematic and formal evaluation can help people who introduce, implement, participate in or otherwise have a stake in PB understand how participatory budgeting is growing, what its reach is, and how it’s impacting the community and beyond.
We developed the 15 Key Metrics for Evaluating Participatory Budgeting
toolkit for people interested in evaluating PB efforts in their communities. It is meant to encourage and support some common research goals across PB sites and meaningfully inform local and national discussions about PB in the U.S. and Canada. It is the first iteration of such a toolkit and especially focused on providing practical and realistic guidance for the evaluation of new and relatively new PB processes.
Anyone involved in public engagement or participation efforts other than participatory budgeting may also be interested in reviewing the toolkit for research and evaluation ideas.
To develop the 15 PB metrics, the North American Research Board, Public Agenda (PA) and the Participatory Budgeting Project (PBP) drew on previous evaluations of PB in the U.S. and around the world, the academic literature on PB as a democratic innovation and the experience of local evaluators in the U.S. and Canada. To create the research instruments, Public Agenda and PBP adapted surveys originally developed and used by local evaluators in various PB sites across North America.
The toolkit includes:
-15 Key Metrics for Evaluating Participatory Budgeting:
15 indicators (“metrics”) that capture important elements of each community-based PB process and the PB movement in North America overall
-Key PB Metrics Research Instruments:
A set of Research Instruments (all customizable) to support local evaluation and facilitate the collection of data that address the key PB metrics: Idea Collection Participant Survey Template, Voter Survey Template, and Questionnaire for Evaluators and Implementers
-Introduction to the Instruments and Evaluation Timeline:
An introduction to the above instruments, which also includes a timeline for how evaluation can fit into PB roll-out
15 Key Metrics for Evaluating Participatory Budgeting
Measuring PB’s Impact on Civic and Political Life:
To what extent does PB engage a significant and growing number of residents, including those who cannot or do not participate in mainstream political life?
1. Number of PB participants and percentage of eligible residents who participate.
Indicates PB’s reach and ability to engage the targeted population.
2. Number and percentage of PB voters who are eligible to vote but did not vote in the most recent local election.
Indicates PB’s potential to engage residents who don’t participate in the mainstream political process.
3. Number and percentage of PB voters who are ineligible to vote in local elections.
Indicates PB’s potential to engage people who are excluded from standard forms of political participation owing to age, immigration status or other reasons.
4. Number and percentage of participants who report prior civic engagement or participation.
Indicates PB’s potential to attract otherwise less civically engaged residents.
5. Number and percentage of participants who report being new or returning to PB.
Indicates both growth and retention of PB participants and various patterns of participation over time.
To what extent does PB foster collaboration between civil society organizations and government?
6. Number of nongovernmental and community-based organizations involved in PB.
Indicates the extent to which PB engages civil society. Also an indicator of variation in how processes are implemented.
Is PB associated with elected officials’ political careers?
7. Number and percentage of elected officials re-elected.
Helps to assess over time PB’s association with officials’ political careers.
Impact on Inclusion and Equity:
Is PB engaging traditionally marginalized communities?
8. Number and percentage of participants who are of low socioeconomic status (SES) and/or people of color; and relative to demographics in the jurisdiction and in the most recent local election.
Indicates PB’s potential to engage communities that are marginalized in the traditional political process.
Through what means does PB facilitate participation?
9. Accessibility indicators for idea collection phase, project development phase and voting.
Captures aspects of the process implementation that increase access during the idea collection phase, the project development phase and the voting phase.
Is PB fostering equitable distribution of resources?
10. Allocation of PB funds by project type (to be compared with the allocation of comparable funds prior to PB). Describes how PB funds get allocated across types of projects. Informs study of differences in allocation and of equity in the distribution of PB funds.
Impact on Government:
How are the number of PB processes and dollar amounts allocated to PB changing from year to year?
11. Number of new, continued and discontinued PB processes from year to year.
Tracks growth and sustainability in PB processes over time.
12. Amount and percentage of funds allocated to PB projects.
Tracks the money allocated to PB projects in any one year.
What is the implementation rate of winning PB projects?
13. Project completion rates and final project costs.
Highlights the # and % of winning ballot projects that are completed and how much money was spent on them (compared with how much was allocated).
Are additional resources being allocated to projects or needs identified through PB?
14. Amount of additional money allocated to projects and needs identified through PB.
Indicates PB’s potential to bring additional funds to communities and/or to allocate funds differently by raising the importance of an issue.
What is the cost to government of implementing PB?
15. Dollar amount spent on PB implementation. Makes transparent how much money is spent on implementation and how that compares with the funds allocated to projects, with quality indicators of the process and with outcomes.
Above is an excerpt of the toolkit. It can be found in full PDF form on Public Agenda’s site here.
About the Participatory Budgeting Project
The Participatory Budgeting Project (PBP) is a non-profit organization that helps communities decide how to spend public money, primarily in the US and Canada. Their mission is to empower community members to make informed, democratic, and fair decisions about public spending and revenue.
Follow on Twitter @PBProject
About Public Agenda
Public Agenda is a nonprofit, nonpartisan organization that helps diverse leaders and citizens navigate divisive, complex issues and work together to find solutions.
Follow on Twitter @PublicAgenda
Resource Link: www.publicagenda.org/pages/research-and-evaluation-of-participatory-budgeting-in-the-us-and-canada