Nonprofits Collect Lots of Data, But Most Don't Use It Says NTEN/Idealware Report

The State of Nonprofit Data report was released this week  (hat tip to Ted Fickes).   Idealware and NTEN prepared the report based on a  survey in April 2012 with nearly 400 nonprofit organizations about how they manage and use data.   The report found that nonprofits are either doing a lot with their metrics or not much at all.
 

These barriers that keep nonprofits from collecting data and identified in the report include:  data collection, expertise, time, and money.  These barriers are precisely why  I wanted to write  “Measuring the Networked Nonprofit:  Using Data to Change the World” with KD Paine.    In Chapter 3 of our book, we talk about the skills and practices of a data-informed organization.   The reports shares the infographic below to illustrate the “Data Machine.”   The report notes that a “well-functioning nonprofit data decision-making process provides numerous opportunities for nonprofits to improve their programs and get better results.”   But far too often the processes of nonprofits did not turn out useful data or in some cases prevented it.

The report goes on to describe the type of data that nonprofits collect which include:  Financial and internal operations data; marketing, communications, and fundraising data and tracking programs and outcomes.  The report offers some insights about the state of data collection in nonprofits, especially with respect to tracking the impact of their programs:

When it comes to tracking the actual impact of their programs, we heard many possibilities for looking at the direct effect of programs—for instance, client satisfaction surveys, number of repeat clients, client recidivism, and the percent of issues resolved for clients, all of which can help to inform nonprofits’operations. many of the organizations in our focus groups wanted to move beyond this level to look at their actual impact in the community, but were struggling to do so. While many nonprofits look at metrics that would actually measure the long term outcomes of their work as the “holy grail” of data-based decision-making, it’s unclear that this level of measurement is practical or even possible for many nonprofits. many orgs need to either rely on metrics which are indirect measures of long term impact—like the number of repeat clients, client satisfaction, or percent of client issues resolved—or invest in expensive, long-term longitudinal research. for instance, one of the nonprofits in our focus group was interested in measuring to what degree they were improving high school students’ success in school by engaging them in creative writing. To directly measure a high-level impact like this, however, one would need years of high quality data summarizing success in school (grades and attendance, for example)and, ideally, a control group with very similar demographics and attributes. This kind of data would be difficult to get from schools, and the research design, data tracking and analysis would be time-consuming and expensive.

In our book,  “Measuring the Networked Nonprofit” one of our key messages is not to start collecting data or think about tools until your organization has defined success.    We lay out a simple framework, “The 7 Steps of Measurement” that can be used to measurement type of nonprofit program.    So, it is eye-opening to see that between 25-42% of respondents have a plan for defining success and having a plan to collect data that will help determine if they are successful.

It is also interesting to see that between 26-50% of nonprofits surveyed use their data to make decisions.    Jocelyn Harman in her review of our book put it another way.  Nonprofits need to “loose their gut and get serious about measurement.”
The survey also provides some data to answer the question, “What is the funders role in supporting good measurement?”   The report identified challenges, including the varied funding and data collection requirements of different funders and reporting demands.   Finally, the report ends with some high level advice about how to get started with measurement and data collection.   These recommendations are very closely aligned with the in-depth and practical how to steps we outline in Measuring the Networked Nonprofit.

While this report focuses on the challenges, barriers, and problems that nonprofits face in using good measurement practices and collecting data,  a session at last weeks Independent Sector called Data 360 focused on the best practices.      The session was a poster session where about a dozen of the leading practitioners of being data-informed in the nonprofit sector lead small group discussions to share best practices.
 
While report reveals that the nonprofit has a lot of room for improvement in using measurement and data, there is a great opportunity for us to continue the dialogue and share best practices on sites like Markets for Good, these resources from IS Sector and, of course, at the NTC. But remember, data is only part of the puzzle -it’s how you make sense of it and apply to constantly improve your organization’s impact and results. This requires a balance of what I call “left brain” (number crunching) and “right brain” (creative thinking) that can lead to innovative approaches to solving the wicked problems that our social sector wants to solve.