Next week, I’m headed to Seattle for a couple of “Measuring the Networked Nonprofit” book events generously sponsored by Microsoft. I’m very excited to participate in one of the evening events, a curated discussion on Data, Transparency, and Impact with Jane Meseck (Microsoft), Paul Shoemaker (Social Venture Partners) and Eric Stowe (Splash) and moderated by: Erica Mills, Claxon Marketing. (The event is sold out, but they are live-streaming it) They are crowdsourcing questions for the conversation here.
Paul Shoemaker shared this post from CEP, “Data Point: Foundations’ Information Needs VS Nonprofits’ Needs” that shares one of the key findings from a recent study called “Room for Improvement” that looks at foundation’s support of performance assessment. There is a lot of pressure for nonprofits to “demonstrate impact” and performance assessment is one method. The study aimed to get the nonprofit perspective on these two important questions: How important do nonprofits believe performance assessment is to demonstrating impact? And if understanding nonprofit performance matters so much to foundations, just how much help are they offering to nonprofits to assess performance? The survey fundings are based on a sample nonprofit CEOs whose organizations are receiving funding from foundations giving at least $5 million annually in grants called the “Nonprofit Voice,” with 170 respondents answering the questions on this survey.
The report offers a couple of key findings:
- Nonprofits very much want to be able to understand their performance and are taking steps to do so
In this survey sample of 170 nonprofits receiving $5 million or more in foundation funding, they use performance measurement data as a management tool. The report does not include any examples or mini-case studies what data these nonprofits are using to inform their understanding of their progress and efforts to improve on an ongoing basis. I can’t help but wonder if their methods are similar to how VolunteerMatch measures its impact and results, although VolunteerMatch does not receive the same scale of foundation funding as nonprofits in this sample.
This finding is also different from another study that looked at how nonprofits were using data conducted by NTEN and Idealware, that found that while many nonprofits were collecting data, they were not using it for decision-making and improvement. The sample in the NTEN/Idealware study is larger and broader than the CEP study.
- Nonprofits want more help in performance assessment efforts than they are currently receiving from their foundation funders.
The study also found that despite perceptions that nonprofits are not doing enough to demonstrate the progress they are making, foundations do not appear to be making significant efforts to help nonprofits in this area. Only 32 percent of respondents to our survey say their funders have been helpful to their ability to assess their progress in achieving their goals. More than 60 percent would like more help from their foundation funders in these efforts.
This is similar to the finding in the NTEN/Idealware study that discovered the barriers that keep nonprofits from collecting and using data: expertise, time, and money. These barriers are precisely why I wanted to write ”Measuring the Networked Nonprofit: Using Data to Change the World” with KD Paine – and continue to blog about opportunities for nonprofits to find help – for example where they can find a data nerd to help them.
The CEP survey asked the nonprofits, “What is the most important step funders could take to help you assess your organization’s progress?” According to the report, one nonprofit leader reflected that it would be helpful if funders were to “provide additional funds with each grant specifically to measure outcomes rather than just require that we identify and measure the outcomes as though it can be done without cost.” Another wished that funders would “include funding for proper assessment tools to systematically capture and analyze data and other pertinent information.”
The report ends with some reasons why there might be a disconnect. First a different interpretation of what the actual measures (or Key Performance Indicators) should be to measure progress – and that nonprofits and their funders don’t necessarily agree. Another reason might be that nonprofits do not have skills and capacity to do performance assessment. The report cites Mario Morino 2011 book, Leap of Reason: Managing to Outcomes in an Era of Scarcity, “I know many nonprofit leaders who are not managing to outcomes today but are strongly predisposed to do so. They inherently know what their outcomes are and very much want to assess and manage to them. But they are severely hamstrung by the lack of funding available to do this hard work.”
NTEN and Idealware recently released a new workbook, “Getting Started With Data-Driven Decision-Making.” (I have some issues with the approach of being “data-driven” and it is better to be “data-informed” based on a measurement process – because you could be driven by the data and it could be the wrong data). While I would re-order some of the questions and logic to be more results or outcome-based vs data-centric, there are some excellent flow charts for the sense-making piece of using data.
I like the worksheet that allows you rate the usefulness and ease of collection for proposed metrics to collect. The framework that is used in the worksheet is to rate usefulness based on whether the data can answer a particular question. This would be different if you using performance measurement — you’d rate your metrics according to how well it can inform your progress towards outcomes — as Bill Gates describes in his annual letter which focused on measurement. Still, adapting this grid as part of thinking through selection and prioritization of KPIs is useful. And relating it back to the earlier study on the disconnect between funders and nonprofits on data – perhaps using it to have a discussion with a funder.
Is your organization using performances assessment or outcome based measurement to measure progress towards goals? How? Do you feel there is a disconnect between what data you’re collecting for measurement and what data your funder needs?
Leave a Reply