Note from Beth: My colleagues at Network Impact have released an important report about measuring and evaluating “civic tech” based on their experience working with Knight Foundation grantees. It goes beyond just analytics tools to exploring tools, techniques, and data for measuring deep civic engagement, open governance, and inclusive decision-making. My colleague, Anne Whatley, wrote this summary of what you’ll find when you dig into these two stellar and highly practical reports and resource lists.
Methods and Metrics for Assessing Civic Tech, Guest post by Anne Whatley, Network Impact
Digital technologies are connecting residents in neighborhoods, catalyzing community discussions, changing the way governments and citizens interact, and making government more transparent. This growing field of “civic tech” is aiming to harness technology to spur citizen engagement, improve cities and make government more effective. The Network Impact team spent 18 months consulting with civic tech thought leaders and offering technical assistance to Knight Foundation grantees focused on assessing the impact of their technologies. One thing that stood out in our conversations is that the civic tech field is ready for a deeper dive on outcomes. Designers of these civic tech platforms and apps are already measuring results, such as the number of users, and are very familiar with tools like Google Analytics. But these instruments don’t explore deeper changes like increases in civic engagement, more inclusive community decision-making, and open governance that may be occurring as a result of their initiatives. A big topic of discussion as the field is maturing is how to track on-the-ground outcomes for people and communities, in addition to counting clicks, views, downloads and tweets.
Code for America, one of the organizations that we worked with, is already implementing outcome assessments. Code for America tracks their impact on open governance ecosystems by monitoring a range of results, including the number of apps created by fellows that are sustained by municipal administrations. In addition to tracking app deployments, Code for America has developed metrics that help them understand their impact on the governments they work with, the communities they engage, and the larger civic technology ecosystem. When we asked Code for America team to summarize the need they see in the field, they responded, “The civic tech field is growing rapidly. Demonstrations of what is possible are up and running in communities of every size across the United States. Now we need to find out not only what works, but what works best over time.”
A few lessons for assessing civic tech from our work include:
A Team Approach
It’s important to engage your full team, especially tech development staff, early in evaluation planning to ensure that systems are in place to capture the data you’ll need to track progress and assess results. For example, if location information will prove useful in tracking place-based impacts, best to build this metric into registration from the start, rather than try to capture the information later.
Hone in on your true scope of influence
Start by focusing your assessment on the outcomes that your app or platform is most directly designed to produce (such as increased trust between neighbors or resident contributions to a specific local decision-making process) rather than on very broad overarching goals like “strengthening participatory democracy.” Strong evidence of success in reaching a targeted objective will create a firmer foundation for future efforts and send a clearer signal to supporters, users and the field than a broader claim for which you have limited evidence.
Learn and borrow from others
Seeing how others are doing this work can help you develop or improve your assessment plan and lead to improved results. The guides we created based on our work with civic tech teams include many examples of outcome assessments linked to overarching objectives that include increasing civic engagement, promoting deliberative democracy, and supporting open governance.
Guides to Civic Tech Assessment
We summarized what we learned in two guides that are designed to help civic tech innovators monitor progress towards their intended outcomes and evaluate the impact of their efforts.
- Assessing Civic Tech: Case Studies and Resources for Tracking Outcomes is a publication of the Knight Foundation with Network Impact that focuses on measuring the impact of civic tech platforms on people, places, and processes.
- How to Measure Success: A Practical Guide to Answering Common Civic Tech Assessment Questions is a Network Impact publication that offers examples and advice for monitoring a platform’s ongoing performance using tools and approaches that are effective and practical.
Our website also includes Additional Tools and Resources with links to more articles and useful references. If you have any suggestions for inclusion please send us a note.
Anne Whatley is a consultant with Network Impact providing advice, research and tools to support social change networks, foundations and the emerging field of network builder
