Say So What To Your Data Three Times

Last week KD Paine and I were in Washington, DC for a whirlwind of book discussions, parties, and workshops – together and individually – for the Measuring the Networked Nonprofit.   Global Giving generously hosted a book discussion with their staff and partners at their offices.    It was great to see Alison Carlman and Kevin Conroy from Global Giving and Meg Biallis who were participants in the testing group of 60 Packard grantees who tested the book frameworks and contributed to the insights and case studies in the book.     I love doing book discussions where participants do a lot of the talking with me and one another – it makes for great learning to share.
One of the messages in the book is “Collect less data, think more about it.”     The sense-making part is so important!      And, one of the questions that keeps coming up is “How do we get our board or senior staff off their addiction to the wrong metrics, like views?”      You have to show the connection between the metric and what matters!
During the discussion, one of the partners asked about how they track whether engaging with their audience on Facebook or other social channel  might lead to donations.   You, of course, need a formal ladder of engagement or conversion funnel and need to be tracking conversions against what your ultimate definition of success is.     One of the participants asked, “So how do you do this with your metrics program?”    Kevin Conroy, the “data scientist” at Global Giving, answered, “Set up goals and tracking URLs  in Google Analytics.”

Kevin came up and gave a mini-lesson on how to use this technique.  When he brought up the google analytics for Global Giving, there was a huge spike from last week.   Kevin demonstrated the “Say So What To Your Data Three Times” principle.    Here’s how he did it:

• A big spike in traffic to the Global Giving site.  Asks so what?  Drills down into the referral traffic – where did it come from?
• The referral traffic came from Reddit.    So What?  The news piece that was on the top of the Reddit site was about the server outages for major online services.  The comments were discussing what people did when Reddit was done.   One person mentioned they did a fundraiser for Global Giving and included the link.   This cause the traffic spike, so what?
• Kevin has goals and conversion tracking for all traffic to the site to look at the conversion rates to donations.    While there a huge spike in traffic, it did not result a high conversion rate to donations.  So what?

Perhaps not worth investing outreach and engagement time on this platform!
For the So What rule to work, you have to follow the 7 steps of measurement and have worked out a ladder of engagement.  We will be covering these in an upcoming NTEN Webinar on November 28th.

The Honest Loser Award
One of the points that resonated was how to create an organizational culture where failure is accepted so you can reap the learning from it.   Donna Callejon shared how Global Giving does this.   They have an “Honest Looser Award” where a staff member gets to tell the story of something they tried in their program, something new, that didn’t work and what they learned from it..    They do this several times a year to encourage innovation and take the stigma out of failure.
Have you asked so what of your data three times?  What did you learn?