Evaluating Networks and their Effectiveness

Guest Post by Tom Kelly
Lists. I am a list maker. To do. To followup. To call. And at a conference like Growing Social Impact in a Networked World (http://www.geofunders.org/networksconference.aspx), I made several lists–in my notebook, on my iPad, emails to myself, and on those very cool hexagonal Post-its. I will work through all of them over the next week or so but the list that will keep growing is the one of insights and new ideas I gained from hearing fellow participants share their knowledge, experiences, and tools.
I was very fortunate to be a part of the early Network of Network Funders community of practice. And it was amazing to be at this meeting and see and connect with real examples of foundations struggling and succeeding with networks in ways that were just notions (yes, hunches) a few years ago. I entered the community as a foundation evaluator wanting to find the best strategies and tools for documenting and learning about networks but also for measuring those difficult-to-pin-down outcomes and impact.
Catalyzing Networks for Social Change: A Funders Guide http://www.geofunders.org/document.aspx?oid=a0660000008Gpuk laid out a framework for assessing networks and their effectiveness and the discussion at this meeting confirmed the usefulness of the frame for looking at impact at 3 levels: connectivity, network formation, and field-level outcomes.
The funders here this week also validated the different purposes and approach in evaluating networks–the importance of patience and timing, the perspective inside (versus outside) a network, the balance of attribution and contribution, the network’s value and use of learning and assessment, and the developmental nature of both the network and the evaluation.
My list of new ideas and followup items gained from colleagues this week challenges me to think harder about not only WHAT to evaluate with networks but also HOW and WHY.
Action: In the struggle to define and measure networks as means or ends focused on process or outcomes, several people offered examples of networks that took smaller actions that could be observed as evidence of progress and network health and as interim steps towards creating the conditions for greater impact later.
Value and Values: We’ve often defined network participation in terms of the value that network members identify for themselves. And we’ve underscored the importance of networks articulating their values and keeping themselves accountable to those shared values. Being result and outcome-oriented is also a value that networks can hold for themselves and evaluators can look for evidence of networks holding themselves accountable internally and externally.
Space and Boundaries: As foundations (funders, investors), we struggle with how much control, authority, influence, accountability, and expectations we exert with networks that need to grow and sustain themselves. But funders can contribute more than just check-writing by seeding the environment and space with the basic infrastructure and impetus to get started (without micromanaging) as well as suggesting boundaries that can help focus networks who struggle with too many opportunities and ideas. Evaluation can also help foundations get the feedback they need to modulate their role and influence.
Communication: The words we use. People are struggling with how to describe the messy processes and life cycles of networks in ways that non-network members (the resource decision makers called trustees) understand but also in ways that analyze network function and experience that contribute to real-time learning. We need case studies that are prospective, real time, and that analyze the different choices and components of network action (and not simply a story). And as Eugene Kim http://www.twitter.com/eekim pointed out, we can’t simply read the case study about violin playing, we have to use case studies as the prompt to actively reflect, experiment, and learn.

Tom Kelly is the Associate Director for Evaluation at the Annie E. Casey Foundation in  Baltimore, Maryland