Evaluation is a crucial step in assessing the success of your efforts and providing you with the data to show the measurement impact you’ve had. Unfortunately, given the technical training required for certain evaluation efforts and the fact that donors historically fail to provide sufficient funding for evaluation, foundations and nonprofits both struggle with sufficient evaluation capacity.24 For example, only 36% of nonprofits receive financial and/or non-monetary support from foundations for performance assessment.25 By providing funding for evaluation capacity and training to grantees, donors can address this problem.
There are many people who can help you assess whether your funding produced its intended results. This talent includes staff at the nonprofits who received funding, external program evaluators or consultants, philanthropic staff you hire, and the beneficiaries themselves. The more you tap all these people, the more insightful your assessment activities will be and the more you will learn and improve.
Internal v. External Evaluation
While many people assume that an outside, third-party evaluator is always best, there is great benefit in ensuring the nonprofits you fund have sufficient capacity to assess for themselves whether their work is producing the intended results.
High-performing nonprofits understand the purpose of the work, the evidence behind their chosen activities, and the needs of their clients. When nonprofits have systems for collecting good data, along with staff who know how to use the data, evaluation results not only help everyone understand whether something succeeded or failed but can also directly inform and improve nonprofit work overall.
And when nonprofits have mechanisms for listening to and integrating beneficiary voice, you and the organizations you fund can be more confident that your work is aligned with the priorities and interests of those you seek to help. This is especially important given the power dynamic and demographic differences we discuss in Avoiding Philanthropy’s Talent Pitfalls. For example, The Fund for Shared Insight, a national funder collaborative based in the United States, promotes beneficiary feedback as a complement to more traditional monitoring and evaluation techniques. The Trust Based Philanthropy Project is a peer-to-peer funder initiative to address the inherent power dynamic between funder and grantee. For more on beneficiary voice, see resources.
The advantage of hiring an outside program evaluator or consultant is the specialized skill set in evaluation methodology, which may be needed for more complex programs such as those that target systems-level outcomes (i.e., changes at the broad policy, practice, or community infrastructure level). Outside consultants can also bring a certain level of neutrality and objectivity to evaluations that can be especially important when an effort is high profile, politically sensitive, and/or under consideration for scaling up to other communities, or even nationally.
Whether you rely on talent internal to your grantee, internal to your foundation (or other philanthropic organization), or external to both, the key to assessing results is to compare what happened to a “counterfactual” — i.e., a reference point that gives you a sense of what would have happened in the absence of the effort you funded. In randomized control-trial studies, that counterfactual is provided by a control group. However, your assessment efforts do not need to be structured as a scientific study to give you a comparison. There are estimates and available benchmarks in relevant social science literature and public data sources that a master’s degree student in a relevant field can access.
Good assessment is an iterative, cyclical process. The information gleaned by assessment activities provides opportunities to revisit and refine social impact goals, a chosen approach, how well an approach was implemented, and even the ways to assess going forward. It keeps you accountable to your plans and goals and helps those you fund stay accountable to their stakeholders and mission. Equally important, it provides information to help you and those you work with learn and improve so that you can achieve social impact that much better and faster.
Talent to Assess, Learn, and Improve: Advantages and Disadvantages
WHO CAN HELP
Staff at the nonprofits you fund
ADVANTAGES
- Understand grantees’ mission and rationale behind activities
- Closer to beneficiaries and their needs
- When systems exist for collecting and understanding results, can immediately incorporate lessons learned in a cycle of real-time, ongoing improvement
DISADVANTAGES
- Incentive to report good results to retain or secure funding if you don’t build trust (see Avoiding Philanthropy’s Talent Pitfalls)
- May be less credible than third- party assessment to others (e.g., government funders) who value presumed neutrality of external evaluator
WHO CAN HELP
Outside, third-party evaluation professionals
ADVANTAGES
- Bring specialized skillset in approaches to nonprofit performance and social impact measurement
- May provide capacity that staff responsible for programs and client services don’t have
- May lack the same incentive as staff to report good results, especially if they belong to a professional organization with a clear code of standards (e.g., American Association of Evaluation Professionals)
DISADVANTAGES
- No existing relationship with clients/beneficiaries
- Time and cost to understand mission and rationale behind grantees’ activities
- Findings may not be readily accepted or incorporated into grantee practice
WHO CAN HELP
Beneficiaries
ADVANTAGES
- Direct, firsthand experience of the grantees’ work — i.e., ultimate arbiter of “success”
- When perspective is systematically incorporated, it helps avoid common pitfalls resulting from differences in demographics, power, and perspectives between nonprofits, funders, and beneficiaries
DISADVANTAGES
- May lack interest or capacity to provide information for assessment activities
WHO CAN HELP
Philanthropic staff you hire
ADVANTAGES
- Most familiar with your intended goals and approach
- Can synthesize information from across grantee staff, outside evaluators, and beneficiary representatives to provide a more comprehensive assessment
- Available to integrate lessons learned into next cycle of philanthropic activities
DISADVANTAGES
- Neither proximate to beneficiaries nor presumed neutral