Social Sector

Learning for
social impact
What foundations can do
McKinsey & Company is a global management consulting firm. Our Social
Sector Office helps the world’s leading institutions develop and deliver
solutions to chronic, complex societal challenges in the areas of global health,
economic development, education, and climate change. Our mission is to
bring an objective, fact-based approach to all of our work in order to strengthen
our clients’ ability to deliver meaningful and sustainable change.
Learning for social impact
What foundations can do 1

Preface

In early 2009, McKinsey & Company launched a project to identify best practices for social
impact assessment. Thirteen leading foundations acted as partners, sharing their time and
experience. We also enlisted more than 100 thought leaders from nonprofits, academic
institutions and beyond to provide their insights. We thank these many contributors, who are
identified in the acknowledgments.

This paper presents our most relevant findings for foundation leaders and other funders.
Additional material, including a workbook for program officers and other practitioners, is
available at sso.mckinsey.com/socialimpact.

A few notes on scope and intent:

• T his paper is intended primarily for leaders of grantmaking foundations. We believe it

will also be useful for heads of other institutions, including government funders, multi-
stakeholder partnerships and large nongovernmental organizations (NGOs). Our goal
is to equip these leaders with a framework for thinking about impact assessment and
helping their organizations adopt proven best practices.
• While assessment is extremely important to improving the execution and design of
programs, the primary focus of this work is on the role of assessment in strategy, which is
the main responsibility of foundation leaders.
• We view assessment here at the program cluster or portfolio level, rather than at the
individual project level.
• We address the social sector as a whole and do not factor in sector-specific or geographic
considerations, which may be significant.
• Our sources have been largely U.S.-based, with a few contributions from the U.K. and
Europe. As a result, we use terms that are common in the U.S.

The work described here takes one step towards a better understanding of social impact
assessment. We value the perspectives of social sector colleagues on these efforts and are
eager to hear about their priorities for further work in this area.

April , 2010
2

Introduction

In February 2009, Philanthropy for Social Justice and Peace,

a group of funders, met in Cairo. The gathering brought What is Social Impact Assessment?
together more than 100 leaders in philanthropy, from
organizations such as the African Women’s Development Social impact is a meaningful change in
Fund, the Brazil Human Rights Fund, the Russian Law and economic, social, cultural, environmental,
Justice Foundation, and the Arab Human Rights Fund. and /or political conditions due to specific
Sessions were held on constituent engagement, structural
actions and behavioral changes by
transformation and grantmaking strategies, but what really
grabbed everyone’s attention was a skit entitled “Kirsty and individuals and families, communities
the Evaluators: Getting a Grip on Evaluation.”1 and organizations, and / or society
and systems. Assessment evaluates
The plot involves Kirsty, an experienced grantmaker who characteristics, practices, results, and / or
has been asked for the first time to prepare an evaluation value of activities.
plan for her foundation’s grantees. She consults experts,
who brandish placards with alien terms such as “log frame”
and “least squares regression.” She surfs the Internet and innovation or missing what is truly meaningful for social
is deluged with buckets of jargon. Then program officers impact because it cannot be measured easily.
descend on her, each demanding evaluations tailored to
make their programs look good. In the finale, an exhausted Our learning driven approach to assessment involves a
and exasperated Kirsty begs the audience for help. set of five best practices, described in Chapter 1, “Making
Assessment Work”:
Subtle it was not. But the skit drew knowing nods from
philanthropic leaders at the conference, who are also 1. H
 ear the constituent voice – Grantees and
struggling to find the right metrics to apply to social sector target constituents are a foundation’s most important
activities. Like Kirsty, they are asking for simple solutions stakeholders. Anchor the assessment process with the
and common approaches to determine what works and why. perspective of the constituents on the ground – the
individuals and communities that are implementing and
To answer this need, McKinsey & Company launched an benefiting from the programs.
initiative called Learning for Social Impact. We believe that 2. A ssess to learn and do – Assessment should be
the greatest value of assessment lies in learning. An emphasis undertaken in a spirit of inquiry. At every step, ask
on learning does not eliminate accountability; if anything it “What do we want to learn?” and build an assessment to
raises the bar. answer that question. Highly effective foundations build
evaluative thinking into their planning, and develop
The goal of social impact assessment is to drive strategy and assessment in tandem.
improvements that increase the value of programs to the 3. Apply rigor within reason – Match the right
people they serve. We know that organizations have to assessment tool to the job. An initiative’s assessment plan
account for funds spent, and describe the scope and reach of must be tailored to its strategic objectives.
work carried out. But, there is a risk that counting outputs 4. Be practical – Calibrate the investment in assessment
becomes a backwards-looking exercise. Our priority on to the program being measured. Find ways to get the
learning makes measurement forward-looking: the focus is information you need without placing undue burdens on
on making programs work better in the future. grantees or program managers. When possible, use tools
that already exist.
Measurement is essential for helping grantmakers 5. Create a learning culture – Create a learning culture
understand where their funding has the most impact. But that provides a safe space for honest discourse about
using the right form of measurement is critical. Indeed, some performance issues and embeds assessment and feedback
types of measurement can have undesired effects: stifling in all routine tasks.

1
Philanthropy for Social Justice and Peace website, http://www.p-sj.org/node/1164
Learning for social impact
What foundations can do 3

Best practices for social impact assessment, including In Chapter 3, “Leading the Sector,” we issue a challenge
those explored in Chapter 1, have been discussed by the to foundation colleagues to take the lead in making
social sector for many years. Foundation leaders know assessment more useful sector-wide. Foundations are
assessment is central to demonstrating that a mission-driven uniquely positioned to spearhead this effort. They have been
organization is on the right path. They already appreciate instrumental in getting the sector this far on social impact
that it is hard to do, but crucial and possible. So why should assessment and have the convening power, bully pulpit and
foundation leaders pay particular attention to social impact resources to push the issue over the finish line. We believe
assessment today? Because the current situation creates the three coordinated steps could be significant.
environment for a more productive look at assessment.
 Develop protocols for partnering with grantees
In Chapter 2, “Seize this Moment,” we discuss why now is Agree on how funders cooperate with grantees on
a critical moment offering an opportunity for real progress assessment, what resources they will provide, and
in measuring what works and why. Three significant trends what grantees should expect from assessment. Setting
shaping the sector can drive adoption of best practices, and these protocols would push the sector forward in a
spur creative and collaborative solutions to the challenge of meaningful way.
measuring social impact. If the sector does not embrace this  Agree on a limited number of common metrics
moment, it may not come again. If funders and grantees can align on relevant metrics,
minimize duplicate effort, and share data efficiently, both
 New skills and perspectives: An influx of new talent assessment and reporting can be greatly streamlined.
is entering all corners of the social sector from fields like  Build a system to share assessment learning
technology and venture capital, where measuring results Funders can set the example for the sector by broadly
is expected. sharing assessment reports through a common
 Rise in social investing: The demand from social repository. This will encourage grantees, social
investors to see credible metrics of impact before enterprises and others to follow suit.
accepting a below-market rate on their capital is
leading to common standards and platforms for impact In these three chapters, the goal is to establish a clear view of
measurement. best practice for assessing social impact and show why and
 Government demand for innovation and how those practices now have the potential for widespread
evidence: The U.S. government’s push for innovative adoption. This is a moment when need and solution are
solutions to social problems from nonprofits that can coming closer together, for the benefit of the entire sector.
demonstrate results provides a real opportunity for
programs to get to scale – but only if there is proof of
their impact.
4

1 Making assessment work

Foundations are built on values and Making assessment a learning process

aspirations. They work with their grantees involves the five best practices described YouthTruth: Constituent
to create social change that reflects their below. voice on education
perspectives and goals. But how can boards,
donors and taxpayers know that foundations 1. H
 ear the constituent voice YouthTruth is a program
are creating social impact? How can For all their resources and capabilities, developed by The Center of
foundations learn what to do next time to get foundations are not on the front lines of
better results? Can the experience of a single Effective Philanthropy with
social change. But their grantees are. So are
foundation and its grantees benefit the social other stakeholders, including individuals support from the Bill and Melinda
sector as a whole? and communities served by grantees, Gates Foundation to survey
the public at large, and the news media. students to understand how to
To answer these questions we designed To gain the greatest insight from social improve school effectiveness
an approach called Learning Driven impact assessment, foundations should and impact. Comparative
Assessment to help foundations, their define stakeholders broadly and engage as
grantees and other stakeholders discover assessments of students’
many of them as they can to get different
ways to increase impact. “The evaluation of perspectives. Engagement with stakeholders high school experiences are
a program or initiative to measure its social comes in two phases – as partners in shared with funders, district
impact is a critical element of grantmaking assessment and as partners in learning. leaders, schools leaders and
– it identifies what was successful, what other students. At an Atlanta
didn’t work, and what lessons can be applied
high school that participated
to future efforts. It is a learning tool for
both the organization and the funder – not in YouthTruth 2008-2009,
a stick with which to beat grantees,” as school leaders learned that
Gara LaMarche, president and CEO of The ninth graders were consistently
Atlantic Philanthropies told us.2 having a less positive experience
than upperclassmen. The
school revamped the freshman
orientation program in the fall
Constituent voice of 2009 to help kids connect
better with the new school
Quantified summaries of constituents’ feedback can
community. In an intermediate
provide credible performance data to managers and school-administered survey,
funders on the things that matter most. When social students reported being more
purpose organizations’ goals depend on behavioral engaged in school, suggesting
change by other constituents, then social purpose that higher achievement may
ensue. The school is repeating
organizations achieve more when they build effective YouthTruth in 2010 to further
relationships with those constituents. gauge its progress. After a pilot
David Bonbright, Keystone Accountability
in 2008-09, YouthTruth has
been expanded to survey more
than 15,000 students in 70 high
schools in 2009-2010.3

2
McKinsey interview, February 2010.
3
For further information, see http://www.youthtruthsurvey.org/
Learning for social impact
What foundations can do 5

Partners in assessment
Let the “locals” show you around. To get a Community voice during the
complete picture of how and to what extent tsunami recovery
programs are delivering social impact,
learn what relevant constituents have to During the response to the 2004 tsunami
say – the grantees and the individuals and
in Sri Lanka, World Vision invested in
communities targeted by programs. “Billions
have been spent on programs that look good a Humanitarian Accountability Team to
to outsiders, but don’t work because they engage with affected communities. The
don’t speak to local people’s concerns and team provided information, listened to
realities,” observes Alex Jacobs, director community concerns, connected with other
of research for Keystone Accountability. stakeholders, and gave residents a greater
“The good news is that a simple, practical
voice in the Sri Lanka Tsunami Response
solution is taking shape: Ask them. Ask
intended beneficiaries what they think Team’s programming. This ensured that
about plans, performance and reports. It’s a projects fit the community’s needs. As a
similar approach to customer feedback in the result, the team saved $5 million by avoiding
commercial sector. Amazingly, it’s not always unsuitable or unnecessary construction. For
widely practiced in social change.”4 example, the accountability team learned
that a proposed project for 400 apartments
Involving grantees and other constituents
in the design and implementation of in Hambantota was not needed, so no
assessment, and in the interpretation of the building took place.5
results, helps ensure that you are measuring
what’s relevant and valuable for them. It puts
the facts in context, so the story is complete
and better conclusions are possible.

4
 lex Jacobs, “Constituency Voice: Making Sure the Right Voices Count,” 2009.
A
5
See: Julian Srodecki, “Improving efficiency and effectiveness through increased accountability to communities: a case study of World Vision’s
tsunami response in Sri Lanka,” World Vision (June 2008) available at: http://www.odihpn.org/report.asp?id=2926
6

Anchoring the assessment process in the with the youth it was trying to serve. Global
field provides additional benefits. Beyond Giving was able to attack the problem by
building some assessment capacity for sending conflict resolution volunteers to the
constituents, the process develops other organization and the community. Eventually,
abilities and strengthens relationships. however, the community requested that
Constituents learn how to communicate Global Giving shift its support to a new
and negotiate priorities, and they organization, and it obliged.6
develop sufficient trust to permit greater
collaboration in future. It is also useful to include additional
stakeholders – other funders and
Global Giving, an on-line fund-raising collaborators, policymakers, academic
marketplace serving organizations experts – in assessment design, data
worldwide, took the key assessment question collection and data analysis, if possible. Each
directly to constituents with stickers that will have different perspectives and needs,
read: “What does your community need? which may not be easily reconciled. But
Tell us: Globalgiving.org/ideas.” Among the using multiple inputs will provide the most
useful insights the program gleaned was complete picture of what works and why –
feedback on a project in Kisumu, Kenya. and will seed broad buy-in to conclusions
Community members alerted Global Giving based on the assessment’s results.
to the poor relationship a local nonprofit had

Grantee perceptions

In 2006, the William and Flora Hewlett Foundation commissioned the Center
for Effective Philanthropy to survey its grantees. Based on grantee feedback,
the foundation launched a year-long project to streamline its processes,
including simplifying the grant application without sacrificing its strategic value.
“The Grantee Perception Report has helped us learn how to improve relations
with our most important group of stakeholders,” Paul Brest, the foundation’s
president, announced.7

6
 ttp://www.globalgiving.org/pfil/1713/TechnologyAided_RealTime_Feedback_Loops_in_International_Philanthropy.doc
h
7
http://www.hewlett.org/what-we-re-learning/our-approach-to-philanthropy/grantee-perception-report/2006-grantee-perception-report
Learning for social impact
What foundations can do 7

Partners in learning
Once you have results, share them. The Communities and networks
stakeholders who helped design the
assessment process should be rewarded  iScale is a networked social enterprise committed to developing and
with the knowledge and insights the sharing innovations for scaling impact. iScale facilitates the Impacts
assessment yields. Bear in mind that Community of Practice, bringing together evaluation experts and
different stakeholders will want different
practitioners to share experiences, explore specific challenges that
types of information: policymakers seek
hard facts and figures, while grantees networks face in measuring their impact, and develop solutions.
may want how-to guides or comparative  In May 2009, New Philanthropy Capital and Bertelsmann Stiftung
experience papers.8 And don’t forget to sponsored a meeting to explore the establishment of an association
enlist stakeholders in developing solutions of social impact analysts, which would organize networking events
to overcome the obstacles you uncover and learning opportunities, share practical tools, and hold forums to
through the assessment.
discuss issues related to the assessment of nonprofits. One goal of
By sharing your knowledge broadly the association would be to elevate nonprofit analysis to professional
you can contribute to the development status, perhaps using the association as an accrediting body.10
of best practices, common metrics
and benchmarks related to your work.
Communities of practice, such as the
iScale9 network that develops and shares
innovations for scaling impact, can help you
spread the word.

2. A
 ssess to learn and do
Two key choices shape an assessment that
will support successful programs. First, set
learning as the assessment’s primary goal. Start with the right question
To do this, begin with as much knowledge
as you can develop about the target
Evaluation planning should always begin with the
problem and the possible solutions; then questions to be explored. But that is not yet the norm. In
understand how your program works and part, this is because people assume that the question of
how it can work better; then test whether
it will work at scale. Second, integrate interest is “Did it work?” Well, that could be the most useful
assessment goals and results into all your question to explore, but it also could be “How did it work?”
program decisions: build your assessment
plan as you plan your strategy and
or “How will it work?” or “Does it work every time?” or
design your program. When you do this, “Why did it work?”
assessment guides your actions; it’s not an
academic review or an isolated exercise. Jackie Williams Kaye, The Atlantic Philanthropies

8
A comparative experience paper shares various approaches to the same problem and identifies commonalities and challenges between the
approaches. In addition, it suggests when a certain approach is best suited for a given situation
9
iScale is a “networked social enterprise committed to creating, developing, applying, promoting, and sharing the innovations for scaling impact
to address the world’s most pressing challenges.” The organization expresses this commitment in a variety of ways, such as social media
platforms, communities of practice, action-learning processes, impact planning and assessment, and strategic mapping. For more information,
please visit http://www.scalingimpact.net/
10
http://www.philanthropycapital.org/how_we_help/big_ideas/association_nonprofit_analysts.aspx
8

Let learning drive the assessment their expected impact and continuously assessment provides the specific information
We use the term Learning Driven fine-tune program strategy. Also, Learning needed to support each type of decision. The
Assessment to describe an approach Driven Assessment captures knowledge feedback loop connecting assessment data to
that seeks insights about the program from expectation “failures” as well as these decisions is critical, ensuring that results
being measured, applies lessons from the successes, harnessing the full range of what are interpreted and applied (Exhibit 1). This is
assessment to improve program strategy, can be learned to benefit future work. a fundamental change for many foundations,
and drives results. It is an assessment which often separate direction-setting and
undertaken with a spirit of inquiry, not of Plan assessment and action together assessment activities.
judgment. “It all starts with What do we “It is a mistake to view evaluation as
want to learn?”, says Jackie Williams Kaye, something embodied in isolated, after-the- We emphasize learning, though not for its
strategic learning executive of The Atlantic fact reports. Highly effective foundations own sake. The goal of a good assessment is
Philanthropies.11 “Useful questions lead build evidence and analytical thinking ultimately to drive better results. Therefore,
to answers that get used.” These Learning into their planning, and use assessment to the lessons from the assessment must
Questions include: What problems do you drive an ongoing cycle of learning, revising, be timely, specific and actionable. If the
need to address? What opportunities do you re-assessing and improving. “It is essential assessment is planned by the same team
want to pursue? What information do you for the donor and the recipient organization defining the strategy and programs, it is
need to plan and take effective action? to have candid discussions about the more likely to be practical and relevant.
progress measures they will use, and to do so
This approach differs from other at the time the donation is made (not when Our focus here is on how assessment and
assessments, which audit results of a the first report comes due!),” says Edward strategy develop in tandem, which is most
program once it’s over, determine which Pauly, director of research and evaluation relevant for foundation leaders. Asking
programs are “best” primarily for purposes at The Wallace Foundation. “Agreeing on the right questions during the planning
of asset allocation, or simply gather stories progress measures at the beginning of process ensures that the strategy will be
to communicate the foundation’s work. Such the philanthropic work generally makes it specific and goal-oriented, with signposts to
assessments may fail to identify what works possible for both the donor and the recipient chart progress and provide early warnings
and why. They don’t consider unintended to work for progress.”12 if programs are falling short – because
consequences (both positive and negative). assessment is built in. The alternative
They often ignore environmental influences Foundations make three primary types – trying to retrofit a strategy with an
that enhance or undermine a program’s of decisions in pursuit of their missions: assessment plan – is like trying to install a
success. These assessments can fail to ask strategic direction, program design and speedometer once the car is racing down the
“does the community support it?” and “what program execution. A well-planned highway at 100 miles an hour.
do the intended beneficiaries think about it?”
A Learning Driven Assessment captures all
these facets.
“Everyone has a plan… til they’re hit.” Mike Tyson

A Learning Driven Assessment not only

Regardless of the level of effort to learn prior to grant
takes in all the relevant information needed
to draw lessons from specific programs, initiation, the real opportunity to increase the probability
it also captures critical information at all of social impact presents itself after the grant is made.
stages – as programs are conceived, while
they are being designed, and during their It is at this point that assumptions regarding grantee or
execution. By asking Learning Questions foundation effectiveness collapse in the face of the very real
throughout the grantmaking process,
managers get real-time input, enabling them circumstances of the work supported.
to correct programs that aren’t yet delivering
Patricia Patrizi, Patrizi Associates

11
 ackie Williams Kaye, “Enhancing Evaluation: The Answer Is in the Form of a Question,” 2009.
J
12
Edward Pauly, “Philanthropy with Impact: A Guide to Evaluative Thinking for Foundations and Donors,” 2010, Forum for Active Philanthropy.
Learning for social impact
What foundations can do 9

Assessment in action Exhibit 1

Chance UK, the mentoring Foundation strategy lifecycle
charity, changed its
approach as a result of what
it found through impact Learning driven decision making

measurement: Female
mentors tended to be more
effective at helping young
FOUNDATION STRATEGY LIFECYCLE
people overcome social
and emotional problems; VALUES
Strategy Program design Program execution

male mentors tended to be AND

MISSION Learning questions Assessment design Assessment execution External
more effective at addressing communication

behavioral problems. LEARNING DRIVEN ASSESSMENT

The charity was able to

change the way it matched 3. Exercise rigor within reason
mentors and young people,
The most rigorous assessment design is
making its mentoring more not always feasible or appropriate for the
effective and helping more program you are evaluating. The best
assessments are tailored to a program’s
young people address the objective. We have defined 24 generalized
challenges they face. Until social sector objectives as a starting point
we as a field collect these for assessment planning. These objectives
correspond to six Types of Intervention
stories, and weave them into (knowledge development, services and
our promotion of measuring products, capacity enhancement and skills
development, behavior change, enabling
social impact, we’ll fail to systems and infrastructure, policy) and four
get people as excited as they Stages of the Solution (framing, prototyping,
demonstration, scale up). This framework,
should be. the “Universe of Initiative Objectives” is
described in greater detail in the following
Tris Lumley, New Philanthropy Capital
special section, which also provides guidance
on how to match the assessment approach to
the Stage of the Solution, and how to design a
Learning Driven Assessment.
10

Exhibit 2
Universe of initiative objectives
6 types of social intervention
We define a universe of Initiative
Objectives that describe goals for social
Knowledge Service / Capacity Behavior Enabling Policy
sector programs, based on Type of development product enhancement change systems and development
development and skills programs infrastructure and
Intervention and Stage of the Solution. and delivery development development implementation
These Initiative Objectives apply across
program areas, geographies and Description Discovering, Providing goods Helping Sharing Establishing Promoting or
populations. developing, and services to organizations or information and systems and resisting a
interpreting or fulfill unmet individuals providing infrastructure that change in
sharing needs of strengthen their motivation to facilitate social government,
knowledge to constituents capabilities assist individuals change multi-lateral, or
Social interventions – the levers we pull solve existing or to change their corporate policy
expected behavior for
to effect social change – fall into the six problems positive social
categories: Knowledge Development; benefits

Product and Service Development and Examples ▪ Medical ▪ Soup kitchens ▪ Vocational ▪ Seatbelt ▪ Networking ▪ Grassroots
Delivery; Capacity Enhancement and Skills research ▪ Performing arts training campaigns opportunities campaigns
▪ Policy ▪ Anti-malaria ▪ Technical ▪ Drunk-driving ▪ Convenings ▪ Lobbying
Development; Behavior Change Programs; research bed nets assistance awareness ▪ Definition of ▪ Litigation
Enabling Systems and Infrastructure ▪ Traditional ▪ Hybrid seeds ▪ Handwashing common
wisdom standards
Development; and Policy Development and ▪ IT systems
Implementation (Exhibit 2).

We define four Stages of the Solution for a

social sector problem, which describe the
phase of planning or implementation that
Exhibit 3
the intervention has reached (Exhibit 3). The
progression is often iterative, nonlinear and 4 stages of the solution
unpredictable, because external events,
preliminary results, or developments in
Description Examples
other programs can force the organization
Define and fully ▪ Conduct analysis to understand
to go back and rethink a previous step. The Frame the understand the problem key drivers of childhood obesity
in the United States
assessment design, data collection and problem

data analysis will be different for each of the

four stages. See “Match the assessment Brainstorm ideas and ▪ Design alternative models for
Develop an design solution approach vocational training programs
approach to the Stage of the Solution.” approach including implementation plan,
costs, benefits and drawbacks

Some problems are new and require Implement approach at ▪ Launch a new after-school program
research to develop a basic understanding. Demonstrate limited scale to test and and refine its operations
and refine the refine the solution ▪ Assess effectiveness of a bed net
This is the Frame the Problem phase. solution distribution program that has been
operating for 5 years
When problems are better understood
and solutions are being prototyped, it is Bring to scale and ▪ With federal government
embed social change in support, develop Children’s
time to Develop an Approach to solve Scale and
the status quo Zones in top 20 US cities based
sustain
the problem. Next the approach is tested on the Harlem model

to Demonstrate and Refine the Solution.

Finally, there are problems that are well
understood, strategies have been proven,
and these solutions need to be Scaled
and Sustained.
Learning for social impact
What foundations can do 11

As illustrated in Exhibit 4, each Type of Intervention and For each initiative objective, we have defined a set
Stage of the Solution has an initiative objective. So, a of generalized Learning Questions to help define a
vaccination program (a Service or Product Development customized assessment plan. To learning more about
and Delivery intervention) that is in pilot phase designing a Learning Driven Assessment, see “A Learning
(Demonstrate and Refine the Solution) has the initiative Driven Assessment in three steps”.
objective to “target constituents’ needs for service/product
met in pilots.” Taken together, the Initiative Objectives
reflect the broad universe of social impact work.

Exhibit 4

Universe of initiative objectives

Type of social intervention

Knowledge Service / Capacity Behavior Enabling Policy

development product enhancement change systems and development
development and skills programs infrastructure and
and delivery development development implementation

Frame the
▪ Problem’s ▪ Target ▪ Target ▪ Problem’s ▪ System gaps in ▪ Context of
context, root constituents’ constituents’ context, causes, coordination, issue and
problem
causes, effects need and capabilities/skills effects and trust and access potential role
and scale are demand for gap and demand target fully understood for policy fully
fully service/ product for program fully constituents understood
understood fully understood understood fully understood

Develop an ▪ Evidence-based ▪ Proof of concept ▪ Program options ▪ Program options ▪ Gap-closing ▪ Policy options
approach hypotheses for service/ designed and designed, and options designed and
Stage of the solution

developed and product most promising most promising designed, and most promising
research method demonstrated model selected model selected most promising solution
designed solution selected selected

Demonstrate ▪ Hypothesis ▪ Target ▪ New ▪ New behavior ▪ Effective ▪ New policy

and refine proven constituents’ knowledge/ exhibited by solution applied position
the solution needs for skills applied target by early adopted by
service/ by target constituents in adopters vanguard
product met in constituents in pilots leaders
pilots pilots

Scale and ▪ New ▪ Need and ▪ Capabilities/ ▪ Problem solved ▪ System gaps ▪ Issues improved
sustain knowledge demand for all skills gap for through eliminated due to full
adopted and target all target mainstream role through implementation
broadly constituents constituents modeling of new mainstream use and broad
applied to fully met by fully met by behavior by of effective enforcement of
address service/ product program target solution new policy
problem constituents
12

Match the assessment approach to the stage preliminary review of results is appropriate. In the more
of solution mature pilot phase, once the program is operating
as intended and has had time to produce results,
 When framing the problem, an observational experiments (including randomized controlled trials)
assessment helps in understanding the current and and quasi-experiments can be used. A wide range of
historical situation. Existing data can put the problem methods for gathering and assessing data should be
in context, and interviews or surveys can gather more employed to demonstrate the effectiveness of the pilot
targeted information. before scaling up.
 To evaluate alternative prototypes, use simulations or  When replicating successful programs, observational
models, which are low-risk, cost-effective means for or quasi-experimental designs should be used to
comparing how approaches will work. Solicit feedback determine whether a program continues to yield
on these ideas from constituents and experts, too. expected benefits at broad scale. Assessment
 In early pilot testing, the goal is simply to execute should focus on how the model performs in different
the program successfully. At this stage, assessment geographies, for different populations of constituents,
should focus on implementation and process. Only a and over time.

A learning driven assessment

in 3 steps

 In Step 1, we provide Learning Questions and a

recommended short list of assessment options
that are based on the Initiative Objective. Exhibit 4
illustrates the Universe of Initiative Objectives, based
on the six Types of Intervention used in the social
sector and the four Stages of development of the
Solution to social problems.
 Step 2 of the process helps the practitioner
customize the Learning Questions to the specific
portfolio of grants, and develop relevant metrics.
 In Step 3, the practitioner factors in organization-
level priorities, such as mission, values or cost, to
finalize the assessment plan.

For a complete explanation of this process and

guidance on how to apply it, please consult
sso.mckinsey.com/socialimpact.
Learning for social impact
What foundations can do 13

4. Be practical suffice. But, by all means, do make the big findings against the cost of the time it takes,”
assessment investments where it counts advises Gara LaMarche “There is nothing
Foundations often set the rules when it – the program with potential to scale; more demoralizing for a grantee than to do
comes to program assessment. Because the innovation that might be a real game- all the work only to have it ignored.”14
they influence the scale and nature of the changer; for flagship programs operated by
assessment for a given program, they need the foundation itself. A thoughtful, comprehensive assessment
to judge wisely when a major investment plan that is based on the right questions
in assessment is appropriate. Foundations As Edward Pauly says, “evaluations are not and crafted with grantee participation
should understand the burden they place on appropriate when donors are supporting an avoids a “fishing expedition” that yields vast
grantees that apply for and “service” their established, well-managed organization to quantities of unusable data. Too often, says
grants – and think carefully about what continue its good works. Careful progress David Hunter, former director of assessment
information is really needed for a meaning- measurement should be the core of evaluative for the Edna McConnell Clark Foundation,
ful assessment. Foundations also can avoid thinking in those cases. Only when a donor is funders demand “reams of data concerning
overburdening their own staffs by looking supporting a previously untried innovation, their finances, activities, and products in
for existing assessment tools and methods, a tricky reform, or an activity whose effects formats and for purposes that often are
rather than building something new every are highly uncertain. . . . [O]ne of their most idiosyncratic to each funder, arbitrary, and
time a program strategy is launched. By valuable resources is more knowledge about virtually impossible to explain or justify
making meaningful assessment as easy on what works and why.13 – and certainly have nothing to do with
grantees as possible and re-using proven evaluation or, for that matter, with helping
tools, foundations can help maximize the While it’s important to gather all the facts people.”15
resources available to drive social impact. to evaluate whether a program is successful
and to understand the context of the work, Often, a grantee winds up tracking different
Don’t measure everything remember that assessment is resource- metrics on the same program for different
Not every program requires a Cadillac intensive. “Funders need to be cognizant not funders. By collaborating with one another to
assessment plan. If a global foundation is to make grantees jump through hoops on define common metrics, funders can relieve
providing a small grant for a local program trivial things that take up valuable staff time their grantees from such duplication of effort
in its headquarters city; if a grant represents and distract them from their core mission. and will likely get better data. By asking the
minor follow-on support, rather than lead Evaluation is a valuable tool, but it also takes grantees what metrics matter most to them,
funding; if the program is already winding time and effort and funders should weigh the funders can ensure the assessment yields
down – the Volkswagen assessment will value of the exercise and the relevance of its maximum benefits.

Useless paperwork breeds mistrust

Grantees know that their reports are used by grantmakers primarily as a way of checking
compliance and perceive this as further evidence that grantmakers do not trust them,
according to our focus group participants. Grantseekers also suspect that many of their
reports do not receive much attention. They wonder why they are required to provide such
detailed and lengthy reports just to prove that they complied with the grant terms.
Project Streamline Report

13
Edward Pauly, “Philanthropy with Impact: A Guide to Evaluative Thinking for Foundations and Donors,” 2010, Forum for Active Philanthropy.
14
McKinsey interview, February 2010.
15
David Hunter, “Performance Management – Navigating Between a Rock and a Hard Place,” December 28, 2007.
14

Borrow, don’t reinvent

The social sector is rich in resources for Can we learn about assessment from video games?
designing social impact assessments.
Funders can tap the Center for Effective The social sector is finding new ideas for assessment in lots of different
Philanthropy,16 Grantcraft,17 and places—including the video gaming world. One example, Darfur is Dying,
Grantmakers for Effective Organizations.18 assesses the impact of participatory game experiences based on measurable
Networks like iScale and communities
civic engagement in the real world. In this online game, the player assumes the
of practice such as the Network on
Development Evaluation19 of the OECD- role of a member of Darfur refugee family. As a result of role-playing a refugee
DAC can connect practitioners with others in the game, more than 40,000 players have taken real-world actions, such as
who have successfully faced with the same sending a message to universities urging divestment in Sudan or donating to an
challenges. NGO working in Darfur. By tracking actions that players take outside the game,
it’s possible to assess the impact of the game.
Tools and Resources for Assessing Social
Impact (TRASI)20 is a new resource, which
was developed by McKinsey and the A second example introduces the notion of real-time impact assessment.
Foundation Center.21 It is the first cross- E-Line Media, a digital entertainment publisher, is currently working on a game
sector repository of social impact assessment called Talkers and Doers to teach entrepreneurial skills to low-income youth.
tools. Now hosted by the Foundation By completing missions in the game, players unlock real-world opportunities
Center, TRASI contains 150 examples of to make money. Nonprofits that work with low-income youth partner with
how social impact is being measured by
the game developers to assess and optimize the impact of the game daily,
foundations, nonprofits, social investors,
social enterprises and others seeking social maximizing the game for both fun and impact. For example, if players are not
change. While social impact assessment is taking desired actions in the game – like connecting with on-line and real-world
complex and different sectors, geographies mentors - designers will tweak the game design, experimenting with various
and constituents have unique needs, our approaches until mentorship signups meet targets.
research indicates that the field is very
fragmented. Our motivation in creating
A third model embeds assessment in the game itself. The ability to complete
TRASI was to make existing approaches
readily available so that fewer resources are all of the levels and finish the game means that a certain level of proficiency has
spent on reinventing the wheel. been attained. This model is most relevant to games that focus on education
and knowledge development.

- Alan Gershenfeld, E-line Media

16
The Center for Effective Philanthropy is a nonprofit organization with the mission to provide data and create insight so philanthropic funders can better define,
assess, and improve their effectiveness and impact. For more information, please visit http://www.effectivephilanthropy.org/index.php
17
GrantCraft is a project of The Ford Foundation designed to promote discussion about strategic and tactical lessons in philanthropy. For more information,
please visit http://www.grantcraft.org/
18
Grantmakers for Effective Organizations is a coalition of more than 2,000 individual members representing 350 grantmaking organizations committed to
building strong and effective nonprofit organizations. GEO helps grantmakers improve practices: Learning for Improvement, Leadership Development,
Money and Stakeholder Engagement. For more information please visit http://www.geofunders.org/home.aspx
19
The Network on Development Evaluation (NDE) is a subsidiary of the Development Assistance Committee, which functions under Organization for Economic
Cooperation and Development. NDE works to increase the effectiveness of international development programs by supporting “robust, informed and
independent evaluation.” NDE gathers together evaluation directors and specialists from OECD development cooperation agencies and multilateral
development institutions. For more information, visit http://www.oecd.org/site/0,3407,en_21571361_34047972_1_1_1_1_1,00.html
20
For more information visit http://trasi.foundationcenter.org/
21
The Foundation Center is a national nonprofit service organization recognized as the nation’s leading authority on organized philanthropy, connecting
nonprofits and the grantmakers supporting them to tools they can use and information they can trust. Its audiences include grantseekers, grantmakers,
researchers, policymakers, the media, and the general public. The Center maintains the most comprehensive database on U.S. grantmakers and their grants;
issues a wide variety of print, electronic, and online information resources; conducts and publishes research on trends in foundation growth, giving, and
practice; and offers an array of free and affordable educational programs. For more information go to http://foundationcenter.org/
Learning for social impact
What foundations can do 15

5. Create a learning culture

Not according to plan
A Learning Driven Assessment can
only come to life in an environment There is as much to learn from foundation programs that fail to meet
where inquiry, feedback and continuous expectations as from programs that have met or exceeded them
improvement are valued – and where – perhaps more. The Robert Wood Johnson Foundation defines
honest dissent is encouraged. Foundations
programmatic failure as not meeting desired ends, and organizational
need metrics and benchmarks to design
and implement assessment, but they also
failure as not learning from programmatic failure or success. In
need a learning culture. This culture must its 2010 report, To Improve Health and Healthcare (vol. 13), the
exist within the foundation and extend to foundation reflects on some of its less successful programs and the
all stakeholders engaged in the assessment related lessons. Comparing learning in philanthropy with learning in
process. science, the foundation reminds us of the healthy tension between
a passionate belief in hypothesized explanations and eagerness for
Organizations that have a learning culture:
 Value honest appraisal, open dissent and
objective evidence to test those beliefs.
constructive feedback
 Undertake intelligent risk-taking in A successful learning culture requires the described how a program failed and invited
pursuit of insight and impact full support and participation of foundation readers to pay attention to “the story of a
 Engage in strategic decision-making leadership – everyone has a role. Foundation philanthropic initiative that did not meet
 Exhibit curiosity; seek innovation and leaders must show by their actions that they, the expectations of its many stakeholders.”
excellence too, are learning and that learning is always Brest acknowledged that the decade-long
 Pursue informed exploration with an valued, even when it is painful. Paul Brest, community improvement program that
orientation to the future president of the William and Flora Hewlett had cost more than $20 million “fell far
 Consider the relevant context, and make Foundation set a good example in March, short of achieving the hoped-for tangible
difficult decisions based on evidence and 2007 when he published a statement on the improvements in residents’ lives.” He went
new understanding foundation’s website, in which he frankly on to share some of the valuable lessons.
 Provide a space where staff can share
stories and lessons from their least
successful efforts: what happened, What is a learning culture?
why it happened, and what they will do
differently next time. It is important that people charged with helping
nonprofit organizations plan evaluations position these
evaluations to provide credible, useful information that
has the potential to have an impact on decision-making.
Evaluation approaches that are coupled with what
we know about how people learn in organizational
contexts improve the likelihood that evidence will
inform choices. At Atlantic, we have evaluation experts,
an organizational development expert, and an internal
facilitator on staff to help establish a continuum from
evaluating to learning.
John A. Healy, The Atlantic Philanthropies
16

2 Seize this moment

Foundations have played a major role and federal government support for social The challenge for the sector is to apply
in developing the field of social impact sector innovation in the U.S. the principles that underlie traditional
assessment. Practicing what they preach, business assessments – measuring, tracking,
many foundations assess how they 1. New skills and perspectives understanding sources of success – in ways
themselves measure up. More than 170 that make sense in the social context. “When
funders have submitted to assessment by Over the past 10 years, the demographics investors evaluate a business, they ultimately
their stakeholders in Grantee Perception of the social sector have changed. Dot-com need to answer only one question – how
Reports22 and 30 have shared the results founders, serial entrepreneurs, venture much money will it make?” notes Michael
publicly. These are some of the signs of very capital investors and hedge fund managers Edwards in Just Another Emperor? “The
real progress. have become influential donors and leaders equivalent for civil society is the social
of foundations and nonprofits. Freshly impact that organizations might achieve,
Yet the conversation about how to measure minted M.B.A.s, who have studied social alone and together, but that is much more
social sector results that has been under way entrepreneurship, social innovation, and the difficult to evaluate, especially at the deeper
for 20 years began to stagnate prior to the nonprofit capital markets, are taking jobs in levels of social transformation.”27 Efforts
global economic crisis. Now that foundation the sector. by these new practitioners to create hybrid
endowments and donations are falling23 and metrics, such as the “social rate of return”
demand for services is surging,24 it is more These “philanthrocapitalists”25 are eager to on investments in citizen action, may be
important than ever for resources to flow to measure social returns. Where they come doomed, he concludes.
high-performing nonprofits and programs. from, everyone uses common standards,
shared benchmarks and widely-accepted The wholesale adoption of investment
In working with foundations and thought measures of value. So, it is only natural that measures won’t capture what is most unique
leaders to identify the best practices we they look for the same sort of tools to guide and important in the social sector. But more
describe in Chapter 1, it became increasingly their efforts in the social sector. These smart heads contributing to the solution
clear that the time is right to push real sector- new players, notes Gara LaMarche, have can only help solve the persistent problems
wide reforms in social impact assessment. brought about “a heightened emphasis on of social impact assessment. This talent
There are three long-term trends shaping the measurability and incorporating lessons migration offers an opportunity to harness
social sector that we believe will move the from the business world.” So, even though experience from technology, medicine,
needle on assessment: a flood of new talent many lessons from business assessment business and investing, and build off that
with fresh perspectives entering the sector; are not directly applicable to philanthropy, experience to develop solutions that truly fit
the rise of social investors willing to back “the increased focus on evaluation and the social sector.
ventures that generate a social bottom line; measurement is a good thing,” he says.26

22
 he Center for Effective Philanthropy’s Grantee Perception Report is used by more than 170 funders to gain comparative, candid feedback
T
collected through confidential surveys from more than 50,000 grantees. The report provides feedback in areas such as: quality of interactions,
application and reporting processes, and perceptions of impact.
23
The Foundation Center estimated that 2009 giving by the nation’s more than 75,000 grantmaking foundations would decrease by up to 13
percent. For 2010, 26 percent of foundations expect that their giving will be lower than in 2009. http://foundationcenter.org/gainknowledge/
research/pdf/researchadvisory_economy_200911.pdf
Hunger in America 2010 reports a staggering increase of 46 percent in the number of Americans receiving annual emergency food aid through
24 

the nation’s network of food banks since 2006. More than one in three client households are experiencing very low food security – or hunger
– an increase of 54 percent in the number of households compared to 4 years ago. http://feedingamerica.org/faces-of-hunger/hunger-in-
america-2010.aspx
25
Coined by Bill Gates, the term “philanthrocapitalism” refers to the call for a new capitalist system with the dual mission of making profits
and fixing the world’s inequalities. (See “Gates Pushes Creative Capitalism,” Financial Times, 25 January 2008.) According to the idea of
philanthrocapitalism, companies should think more broadly about the ways in which their actions can benefit both the greater good and their
own bottoms lines. Others have expanded this definition to include the adoption of business-like practices in the social sector.
26
McKinsey interview, February 2010.
27
Michael Edwards, Just Another Emperor? The Myths and Realities of Philanthrocapitalism (New York: Demos, 2008), p. 66.
Learning for social impact
What foundations can do 17

2. Rise in social investing

Understanding risk as part of social impact assessment
Access to capital is a strong motivator
Once we have measured a program’s social return, how do we know if for developing common approaches for
the return was “enough”? measurement in the social sector. Socially
responsible investors28 offer that access – but
In for-profit investment management, there is a basic concept of risk- they want reliable metrics of social impact
to guide their investing. According to a
adjusted rate of return. Riskier investments require higher returns to
recent report by the Monitor Group, “impact
compensate for their risk. Financial investors even have a tool – the investing has the potential to grow to about
Sharpe Ratio – to compare investments on the basis of risk and return. 1 percent of total managed assets, which
would result in about $500 billion of capital
In the social sector, risk is not clearly defined and we lack tools to channeled toward social and environmental
measure it. Many who think about measuring impact never consider impact.”29 This is significant. As a point of
comparison, assets of all U.S. and European
measuring risk. Yet risks abound: concept risk, design risk, execution
foundations, along with U.S. donor advised
risk, discovery risk, and the risk of failing to stick to one’s moral and funds, total approximately $862 billion.
political convictions. Annual global grantmaking is approximately
$110 billion.30
And who bears these risks? The donor receives the tax advantage and
the warm and fuzzy feeling that comes from being altruistic, regardless A number of efforts are under way to help
social enterprises share information on the
of whether real social impact is generated. It is the constituent who
social impact they generate.These include
bears the full cost – at least an opportunity cost – if the donation is common standards and platforms like
used ineffectively and inefficiently, and fails to generate a social return. the B Rating System,31 the Global Impact
Investing Ratings System (GIIRS),32 and the
Risk is an unavoidable dimension of innovation and social change Impact Reporting and Investment Standards
work, and a relevant – though missing – part of the conversation on (IRIS),33 which help social investors and
social enterprises communicate and agree on
social impact assessment.
measures of value.

28
 ocially responsible investors seek social impact as well as financial return when investing capital. They consider environmental, social and
S
corporate governance criteria in addition to standard financial analysis when making investment decisions.
29
“Investing for Social and Environmental Impact,” Monitor Group, January 2009.
30
U.S. foundation source: http://foundationcenter.org/gainknowledge/research/pdf/fy2009_highlights.pdf ; U.S. donor advised funds source:
http://www.cof.org/files/Bamboo/programsandservices/research/documents/dafjuly09.pdf ; European foundation source: http://www.efc.be/
NewsKnowledge/Documents/EFC-RTF_EU%20Foundations-Facts%20and%20Figures_2008.pdf
31
The B Rating System is a survey created by B Labs that helps an organization assess its impact on each of its stakeholders and allows the
organization to use tools and best practices embedded in the survey to improve its social and environmental performance. It can be used
formally by companies that wish to become “Certified B Corporations,” or as a general benchmarking tool for any group that is interested in
measuring impact. For more information go to http://www.bcorporation.net/become/BRS
32
GIIRS, or Global Impact Investing Ratings System, is an open-source rating system that provides impact investors with independent and
objective tools to assess the social and environmental impact of individual companies and of their larger investment portfolios. GIIRS
development is being overseen by B Lab, a non-profit dedicated to building a marketplace of socially impactful companies, funds and investors,
with support from The Rockefeller Foundation. For more information go to http://www.rockefellerfoundation.org/
33
IRIS, or Impact Reporting and Investment Standards, is a group of financial, operational, and impact metrics that helps increase the
transparency of social and environmentally-oriented investments through a “user-configurable, open-source XBRL-enabled toolset.” IRIS is a
joint initiative of The Rockefeller Foundation, Acumen Fund, B Lab and others. For more information go to http://iris-standards.org
18

Tools like Pulse34 from the Acumen Fund, 3. Government demand for Corporation for National and Community
are now used to collect and report results innovation and evidence Service’s new Social Innovation Fund.
of social enterprises at the portfolio level. While the Social Innovation Fund funding
Communities such as the Global Impact The Obama Administration has created is relatively small, its influence will likely
Investing Network (GIIN)35 and the Aspen several innovation funds37 that will source be large. It will help to showcase innovative
Network of Development Entrepreneurs proven solutions to social problems from nonprofits that are proving their effectiveness,
(ANDE)36 support collaboration, develop non-profits. The White House Office of Social which may attract other sources of federal
infrastructure and build capacity for social Innovation, meanwhile, intends to promote funding to finance scale-up.
investing. Foundations have helped seed social innovation throughout the federal
and support all of these efforts, using government, including through the
their convening power, capital and ability
to publicize.

Social impact is the same, whether it is

Innovation:
generated through a social investment or a How to measure a moving target
grant, and can be measured in a similar way.
Innovation is all about navigating uncertainty
Therefore, the work of the social investment
community in assessment should benefit the and we’ve found this to be a sticking point in
social sector as a whole. By setting common conversations about evaluation, where you are
metrics, shared benchmarks, and collecting
data in a sector-wide repository, the social presumed to know what you need to measure
investment community has started to build in advance. There is huge value in determining
an important platform for understanding
and measuring social impact.
what to measure early on so that you will
know if you are on track. But it is also critical to
regularly assess whether you need to alter your
measurement criteria as you learn more. This
practice of prototyping and iteration are key
tools to guide us through foggy projects.
Aaron Sklar, IDEO

34
 ulse is a portfolio data management platform that helps social investors track, measure and evaluate financial, operational, and social data to
P
better understand the global impact of their portfolios. For more information go to http://www2.app-x.com/pulse/
35
GIIN is a non-profit dedicated to increasing the effectiveness of impact investing. GIIN has three main areas of focus: the GIIN Investors’ Council
brings together and supports leading impact investors; IRIS is a group of financial, operational, and impact metrics that help increase the
transparency of impact investing; and GIIN’s education initiative provides research and analysis to support institutions as they begin or expand
an impact investing program, informs policymaking related to impact investing, and conducts and distributes new research to the larger impact
investing community. For more information go to http://www.globalimpactinvestingnetwork.org/cgi-bin/iowa/home/index.html
36
ANDE, the Aspen Network of Development Entrepreneurs, is a global network of organizations dedicated to propelling entrepreneurship in
emerging markets. Housed in the Aspen Institute, ANDE works to increase the amount and effectiveness of investment and business assistance
provided to entrepreneurs in developing countries. For more information go to http://www.aspeninstitute.org/policy-work/aspen-network-
development-entrepreneurs/about-ande
37
Innovation funds have been created as part of the Corporation for National and Community Service, the Department of Education, the
Department of Health and Human Services, the Department of Housing and Urban Development, and the White House Office of Faith-Based and
Neighborhood Partnerships.
Learning for social impact
What foundations can do 19

Another way the Social Innovation Fund The Office of Management and Budget, evidence of impact, has enlivened the
is expected to punch above its weight is by which has oversight responsibility for discussion of assessment. Not all the talk
helping advance social impact assessment. the social innovation funds along with is positive, to be sure. Many in the social
According to the enabling legislation, thousands of other government programs, sector are excited by the possibility of a new
applicants to the Social Innovation Fund has also signaled the importance of funding source. But there is also unease
must “propose to focus on improving assessment. “Rigorous ways to evaluate about how success will be measured. Some
measurable outcomes” and prove program whether programs are working exist,” OMB of the methods of evaluating programs to
effectiveness with data, “including, where Director Peter Orszag has written on his which Orszag alludes, such as randomized
available, well-implemented, randomized blog. “But too often such evaluations don’t controlled trials, are not the most
controlled trials.” If an applicant does not happen. They are typically an afterthought. appropriate approaches for assessing nascent
have sufficient evidence of success, it may . . . This has to change, and I am trying to programs (at the proof of concept or early
apply to work with a research organization put much more emphasis on evidence-based pilot stages) and could wind up choking off
to evaluate its program. In addition, up to 5 policy decisions here at OMB.”39 innovation. At a minimum, the attention
percent of the Social Innovation Fund’s $360 from Washington undeniably has the
million may be used for research to improve The prospect of significant government potential to push the debate on best practices
evaluation, benefiting the social sector as resources to bring effective programs to in social impact assessment further.
a whole.38 scale, coupled with a demand for rigorous

Randomized controlled trials: for mature programs only

Randomized controlled trials are an important tool, but they are

most suitable for measuring programs at the mature pilot phase.
Even then, experimental design should be used alongside other
types of measures to ensure a comprehensive understanding of the
results in the proper context. David Bonbright, chief executive of
Keystone Accountability, summed up the concerns of social sector
colleagues in a 2008 article on using experimental evaluation:

“First it really is expensive. This might be OK if there were not

much cheaper alternatives. Second, while a properly conducted
experimental evaluation will prove that X program will produce Y
outcome, it does not tell you why. Consequently, it is not particularly
helpful for scaling up program X to achieve Y in other settings
(context turns out to matter, as many studies have shown).”40

38
Serve America Act, 2009. For more information go to http://www.nationalservice.gov/about/serveamerica/index.asp
39
 ttp://www.whitehouse.gov/omb/blog/09/06/08/BuildingRigorousEvidencetoDrivePolicy/
h
40
“Against Experimental Evaluation: Proving or Improving, Alliance Magazine, March 2008.
20

Foundations have contributed to all three

of the trends reshaping the social sector. Proven innovation that works
They provide an avenue for new donors
and leaders to enter the social sector; they The programs described below have been cited as proven social innovators
have supported the research and creation of and are under consideration to be scaled up with federal funding.
networks and platforms that facilitate social
investing; and they have offered thought
partnership and matching funds for the
Nurse-Family Partnership. This well-established home-visitation program
government’s Social Innovation Fund. has proven to be effective in improving the long-term health and well-
being of poor first-time mothers and their children. Over three decades,
Foundations can take the lead going forward randomized controlled trials were conducted with three diverse populations
as well. In Chapter 3, we describe three in New York, Tennessee, and Colorado, demonstrating the benefits to
specific steps for foundations to move social
first-time, low-income mothers. Follow-up research continues today,
impact assessment ahead: partnering with
grantees, agreeing to common measures,
documenting the long-term outcomes for mothers and children.41
and sharing assessment lessons.
Harlem Children’s Zone. Harlem Children’s Zone offers a comprehensive
Foundations have the opportunity to set of programs that engage local residents to provide a safe learning
seize the moment. If they do not, the same environment and positive opportunities for children and families in New
stagnant conversation about assessment will
York City’s Harlem community. Over 10,500 youth are served each year
be rattling through the conference halls and
meeting rooms 10 years from now – and the
through early childhood education, charter schools, after-school and
social sector can expect to lose the benefits summer enrichment programming for youth, and help for teens to gain
of these three trends as human capital, access to college or other post-secondary education opportunities.42 The
financial capital and intellectual capital organization’s Promise Academies seek to improve academic achievement
move on. and life outcomes in high poverty areas. This approach has produced
enormous gains that eliminated the black-white achievement gap,
according to research by two Harvard economists.43

Carrera Adolescent Pregnancy Prevention Program. This program of

the Children’s Aid Society helps children aged 10 to 18 avoid pregnancy
and risky sexual behavior. Emphasizing their potential, rather than their
risks, the program helps children develop a desire for a productive, risk-
free future and provides opportunities for them to discover interests and
develop talents, while emphasizing education and employment. Philliber
Research Associates found the CAS-Carrera program yielded a 50 percent
reduction in teen birth rates in communities served, along with a multitude
of positive outcomes for both boys and girls, including higher graduation
and college admission rates, and increased adoption of healthy living
practices. The program is the only fully evaluated teenage pregnancy
prevention program in the country with statistically proven effectiveness.44

41
 ee their website: http://www.nursefamilypartnership.org/
S
42
See their website: http://www.hcz.org/
43
“Are High-Quality Schools Enough to Close the Achievement Gap?” Willie Dobbie and Roland G. Fryer, Jr., Harvard University, 2009.
44
See their website: http://www.stopteenpregnancy.com/
Learning for social impact
What foundations can do 21

3 Leading the sector

Foundations should remain in the lead, The first step is to develop a common
promoting improved social impact taxonomy to guide data gathering.
assessment. They are ideally positioned to Even setting metrics for operational
drive best practices that will benefit their effectiveness as a means of getting this
own organizations, their grantees and system in place is a start. To walk the
the entire social sector. If the top 25 U.S. talk, foundations must also collaborate
foundations agreed to cooperate on the on a basic set of metrics to report on
following three initiatives, think what they their own work – and post those annual
could accomplish. results, too.

 Develop protocols for partnering  Build a system to share assessment

with grantees. learning.
Codify how foundations should Commit to contributing all portfolio- and
collaborate with grantees on planning, strategy-level assessment reports to a
implementing and learning from master clearinghouse so the whole sector
assessment. Stipulate levels of technical can benefit. Encourage grantees to
assistance and financial support for contribute their major evaluation reviews
assessment in every grant. Create a as well. Build off existing capacities
“grantee bill of rights,” describing what such as PubHub,46 a collection of almost
grantees can expect from their funders, 5,000 research reports and case studies
including feedback on what’s working – about work supported by foundations.
and the ability to provide funders with Tap affinity groups of funders such as
feedback as well. Grantmakers for Effective Organizations
and the Evaluation Roundtable,47 which
 Agree on a limited number of seek to improve assessment practices,
common metrics. and groups that bring together funders,
Adopt a basic set of metrics that grantees and others, such as Independent
foundations agree to accept from their Sector,48 which have the capacity to
grantees. Grantees would post their design the system. These efforts can
annual results to a common repository. bring forth the first social sector-wide
A neutral party, such as the Foundation repository for learning by funders,
Center, can collect the assessment data NGOs, social enterprises, activists and
and develop benchmarks, perhaps others seeking social change.
following the example set by IRIS.45

45
IRIS, or “Impact Reporting and Investment Standards” is an effort initiated by The Rockefeller Foundation, Acumen Fund, and B Lab with the
goal of establishing a common framework for defining, tracking, and reporting impact investment performance. IRIS builds on already-existing
efforts in the sector to create one language that allows for comparison across the wide range of organizations that seek to have social impact.
This framework includes descriptor indicators that focus on an organization’s missions and specific initiatives, common financial indicators,
operations indicators, sector-specific indicators, and a glossary of key terms and metrics. For more information go to http://iris-standards.org/
46
PubHub showcases the knowledge generated or supported by foundations, and fosters the exchange of best practices and lessons learned
in the field of philanthropy. PubHub’s searchable catalog of annotated links to foundation-sponsored reports, allows access to the latest
publications on specific issues from across the full spectrum of philanthropic activity in the United States. For more information go to http://
foundationcenter.org/gainknowledge/pubhub/about_pubhub.html
47
The Evaluation Roundtable is a group of evaluation and program executives at North American foundations with a deep commitment to
evaluation and learning.
48
Independent Sector is the leadership forum for 550 charities, foundations, and corporate giving programs committed to advancing the common
good in America and around the world. For more information go to http://www.independentsector.org/index.htm
22

So, if the most powerful foundations in the

United States got together, and did these
Feedback: There’s an app for that
things, what would be the result? The
U.S. social sector would soon have a very One nascent example of a system to share learning is
powerful platform for impact assessment Keystone Accountability’s Feedback App. Organizations
that could be expanded and adapted globally. survey their constituents on some common questions.
An investment in assessment would be an
Feedback is shared anonymously with the organization, and
investment in learning and driving social
change. The timing is perfect as the need is
aggregated in a repository. Over time, the repository will enable
universal. Who will take the lead? benchmarking, allowing an individual organization to see
how its own constituent feedback compares to the average
feedback of peer organizations working in the same sector or
region. The more organizations that use the app, the better the
benchmarks become. More information can be found at
http://feedback.keystoneaccountability.org/

We hope this report is helpful to foundation leaders and others. It is only a beginning and we are exploring additional
opportunities to deepen our work on Learning for Social Impact. We have already identified some ideas for future work:

 Concrete examples of how social impact assessment findings are used to improve programs
 Special challenges of assessment in the developing world
 Project assessment from the perspective of nonprofits and social enterprises
 Sector-specific and geography-specific assessment frameworks.

Please share suggestions and comments by emailing socialimpact@mckinsey.com

Acknowledgements

Foundations Grantmakers for Effective Organizations: Kathleen P. Enright

The Atlantic Philanthropies: John A. Healy, Jackie Williams Kaye, GreatNonprofits: Perla Ni
Gara LaMarche, Rebecca Rittgers Growth Philanthropy Network: Cynthia W. Massarsky
Case Foundation: Erich Broksas, Michael Smith HelpAge International: Mark Gorman
The Annie E. Casey Foundation: Tom Kelly Human Rights Watch: Laura Boardman, Ian Gorvin
The Children’s Investment Fund Foundation: Sean Carney, Hunter Consulting LLC: David E. K. Hunter
Phyllis Costanza, Peter McDermott Icosystem: Paolo Gaudiano
The Edna McConnell Clark Foundation: Kelly Fitzsimmons, IDEO: Aaron Sklar
Woodrow McCutchen Independent Sector: Andrea Affeltranger, Erica Greeley
Ford Foundation: Linetta Gilbert, Christopher Harris, Jan Jaffe, Institute for Philanthropy: Dr. Salvatore LaSpada
Richard M. McGahey, Suzanne Siskel International Planned Parenthood Federation: Kevin Osborne
Bill & Melinda Gates Foundation: Kendall Guthrie, Philip Setel iScale; Harvard Business School: Sanjeev Khagram
The William and Flora Hewlett Foundation: Paul Brest, Jacob Harold, Keystone Accountability: David Bonbright, Alex Jacobs, Natalia
Jennifer Ratay Kiryttopoulou
Robert Wood Johnson Foundation: David C. Colby, Robert G. Hughes Limmat Stiftung: Juan J. Alarcon
W.K. Kellogg Foundation: Tony Berkley, Ted Chen Microsoft: Michael Galgon
John S. and James L. Knight Foundation: Mayur Patel Dr. Joyce Lewinger Moock
The Rockefeller Foundation: Margot Brandenburg, Nancy MacPherson New Philanthropy Capital: Padraic Brick, Martin Brookes, Tris
The Wallace Foundation: Richard Laine, Edward Pauly Lumley, Belinda Vernon
Oxford Poverty & Human Development Initiative: Sabina Alkire,
Other Contributors John Hammock
Acumen Fund: Brian Trelstad Pacific Community Ventures: Beth Sirull
Ashoka: Karabi Acharya Patrizi Associates: Patricia Patrizi
B Corporation: Bart Houlahan, Andrew Kassoy Physic Ventures: William Rosenzweig
Base of the Pyramid Initiative, University of Michigan: Ted London PLAN International UK: Marie Staunton, Irko Zuurmond
Blended Value: Jed Emerson REDF: Anna C. Martin
Blueprint Research & Design: Lucy Bernholz Rockefeller Philanthropy Advisors: Melissa Berman
BOND: Sue Campbell, Andy Featherstone Root Cause: Anand Dholakia, Andrew Wolk
Calvert Foundation: Shari Berenbach Save the Children: Duncan Trotter
Center for Advancement of Social Entrepreneurship, Fuqua SeaChange Capital Partners: Charles Harris
Business School, Duke University: Catherine H. Clark Skoll Centre for Social Entrepreneurship; Saïd Business School,
The Center for Effective Philanthropy: Phil Buchanan, Lisa R. Oxford University: Pamela Hartigan
Jackson, Valerie Threlfall Skoll Foundation: Ben Binswanger, Lakshmi Karan
The Center for High Impact Philanthropy: Katherina M. Rosqueta SoCap: Amy Benziger
The Center for What Works: Debra B. Natenshon Social Solutions: Jeff Mason
Charity Navigator: Ken Berger SVT Group: Sara Olsen
Clark Foundation: Douglas Bauer Katy Spencer
Commonwealth Development Corporation: Marie Rosencrantz Stewart Satter Program in Social Entrepreneurship; New York
Cynthesis Consulting: Cynthia Gibson University, Stern School of Business: Jill R. Kickul
DC Central Kitchen: Robert Egger Synergos Institute: John Heller, Chong-Lim Lee, Surita Sandosham
Demos, New York: Michael Edwards Tactical Philanthropy: Sean Stannard-Stockton
The Earth Institute, Columbia University: Shiv Someshwar TCC Group: Paul Connolly, Peter York
E-line Media: Michael Angst, Alan Gershenfeld Melinda Tuan
The Foundation Center: Nina Gantcheva, Joyce Infante, Maureen The Urban Institute: Elizabeth Boris, Maria-Rosario Jackson
Mackey, Lawrence T. McGill, Thomas Provan, Frederick K. Schoff, VisionSpring: Peter Eliassen, Graham Macmillan
Bradford K. Smith Wellspring Consulting LLC: Christopher Keevil
FSG Social Impact Advisors: Mark Kramer, Marcie Parkhurst White House Office of Social Innovation and Civic Participation:
GiveWell: Elie Hassenfeld, Holden Karnofsky Michele Jolin, Sonal Shah
GlobalGiving: Marc Maxson Xigi Media: Gary A. Bolles
Joyce & Irving Goldman Family Foundation: Sarah E. Meyer Young Foundation: Simon Tucker
Good Capital: Kevin Jones
Social sector
March 2010
McKinsey London
Copyright © McKinsey & Company