The sector’s accountability relationships and how it defines and accounts for “success” are in need of an overhaul, says contributor Nancy Pole. In the fifth article in our Rethinking Philanthropy series, she argues that philanthropic foundations can play a leading role in this transformation – and in so doing, think differently about their own accountabilities.
A number of promising changes are underway in the non-profit and charitable sectors and in their relationships with partners and communities. In the last few years, the needs, aspirations, and calls for justice of equity-seeking communities have begun to receive long-overdue attention. Leading voices are acknowledging the role that the non-profit and charitable sector has played in Canada’s colonial legacy, and work continues with a view to transforming these relationships and to entering into new ones based on reciprocity – transformations that imply redistribution of resources and a centring of Indigenous knowledge, aspirations, and traditions of philanthropy.
In parallel to this and after decades of advocacy, we are beginning to see long-hoped-for reforms to the charitable sector’s regulatory framework. Following the lifting of limits on charities’ political activities in 2018 and the Senate Special Committee Report in 2019, federal legislators are now grappling with reforms to donor–donee rules and requirements, and amendments to charitable foundations’ annual disbursement quota will begin to come into effect later this year.
Despite these promises of change and renewal, the sector’s accountability relationships and how it defines and accounts for “success” have, overall, remained constrained, fraught, and distorted. The reasons for this are bound up with the sector’s financing models and funding relationships. Non-profit organizations and their allies are well aware of these issues. This piece argues that funders – and philanthropic foundations in particular – can be key players in the righting of these relationships.
Accounting for success in an increasingly complex funding ecosystem
While charities and non-profits largely recognize the importance of evaluating their own work in order to know what is working and how, the funder–fundee relationship remains central to evaluation practice and norms (Lasby, 2019a; Tello-Rozas et al., 2022). Funder expectations about measuring and accounting for success, and the frameworks or measures that organizations are made to work with, tend to define evaluation purposes and activities for charities and non-profits – creating a potentially problematic distortion of program purpose and success (Phillips & Carlan, 2018).
Organizations struggle to meet the increasingly diverse reporting expectations of donors and investors.
As organizations diversify their financing sources, they struggle to meet the increasingly diverse reporting expectations of donors and investors and experience ongoing tension between satisfying their own evaluation and learning needs and meeting external accountability demands (OECD, 2021; Tello-Rozas et al., 2022).
Government agencies continue to be the single largest funding source within the non-profit sector, accounting for about 44% of core charitable-sector revenues (Lasby, 2011). As a consequence of neoliberal reforms to public service provision, the long-term, entrenched trend since the early 1990s has been toward contracting funding relationships with non-profit service providers (Phillips et al, 2010; Depelteau, 2013; Elson & Carmichael, 2022). Over this same period, the adoption of results-based management and performance-measurement practices within the public sector spilled over into reporting and accountability requirements for funded non-profits (Chouinard, 2013; Phillips & Carlan, 2018).
Non-profits feel pushed to report only on positive results and are discouraged from sharing openly about challenges and difficulties.
When evaluation is viewed by funders, particularly governments, as control and risk management rather than as a means to learning and improvement, non-profits feel pushed to report only on positive results and are discouraged from sharing openly about challenges and difficulties (Bare, 2010; Lalande & Cave, 2017; Lasby, 2019a & b).
In a parallel yet distinct trend, with the increasing prominence of social enterprise and the social economy within the sector, new financing models and vehicles have emerged to support the investment of private capital in projects and organizations. In contrast to more traditional funders, social impact investors lend or invest their capital, seeking to achieve social benefits but also a financial return if outcome targets are achieved. In connection with these forms of investment, requirements have risen for standardized metrics that enable impacts to be commodified and traded on social exchange markets, leading to the popularity of ratios such as social return on investment (SROI) (Phillips & Carlan, 2018).
What if we had an ecosystem that truly valued experimentation and adaptation over reporting prescribed “success,” that valued capturing, distilling, and sharing authentic learnings?
A number of risks have been noted in connection with the uptake of these “financialized” forms of outcome measurement, including the increased reporting burden that they represent and their tendency to discourage innovative practices. Most significantly, metrics that focus on short-term impacts tend to prioritize interventions for which there is a high certainty of cause and effect, failing to capture more complex phenomena (Ruff, 2021). Moreover, they are likely to bias funding toward more immediate “solutions,” rather than in support of long-term social progress (OECD, 2021).
The emerging push for alternative ways forward
Despite these strong pushes for standardization, it is increasingly recognized that a one-size-fits-all approach is inappropriate and that top-down and overly constraining readings of outcomes and impacts negatively affect the autonomy of non-profit organizations to articulate success on their own terms and to measure what matters most to them and to the communities they serve. In addition, when the primary focus is on measuring and communicating results to “prove” success to funders in exchange for their support, there are built-in incentives to distort what information is worthy of attention, what gets reported as “results,” and even what programs are prioritized (Dougherty, 2019).
What if we had an ecosystem instead that truly valued experimentation and adaptation over reporting prescribed “success,” that valued capturing, distilling, and sharing authentic learnings? What if we had an ecosystem that included all affected stakeholders in defining and assessing what counts as success and that supported accountability to all of these stakeholders?
The last seven years have seen a blossoming of initiatives that aim to help social purpose organizations and their allies achieve greater autonomy and power to define the terms by which their success should be measured, and to negotiate flexible approaches to reporting and evaluation that reconcile different stakeholders’ needs. This was the aim pursued by the Ontario Nonprofit Network’s Sector Driven Evaluation Strategy and was an upshot of the Quebec-based Territoires innovants en économie sociale et solidaire’s impact measurement initiative and the Montreal Declaration.
The Common Approach to Impact Measurement is a powerful tool to meet the needs of non-profit networks.
Kate Ruff, Common Approach
The Common Approach to Impact Measurement initiative, which grew out of Ontario’s Social Enterprise Impact Measurement Task Force and now works with a broad range of stakeholders across Canada, has similarly come to embrace a commitment to “centering impact measurement on the needs of operating charities, nonprofits and social-purpose businesses and those they serve.” Building upon its Common Foundations that distill a shared set of essential practices in measuring impact, the initiative has also developed and launched a common standard for representing impact data.
“This common standard is a powerful tool to meet the needs of non-profit networks,” say Kate Ruff, interim executive director at Common Approach. “Each participating non-profit continues to own and control their own data but labels and shares it in a way that allows data to be aggregated and compared across a whole sector or within a region. This ultimately helps networks to better understand and tell the story of the sector’s impact.”
A role for philanthropic foundations: Reconciling accountabilities
Philanthropic foundations accounted for 12% of sector revenue in the mid-2000s (Grasse & Lam, 2021) – a proportion that may have since increased as the number of foundations and amounts granted have also increased. Despite being a relatively small part of the funding ecosystem, foundations have a critical role to play in shifting this relationship and in fostering these new practices.
Why is this? Foundations are much freer than government funders, as they are more shielded from external accountability pressures (Leat, 2006; Rourke, 2014) – although perceptions and realities vary between private and public foundations (Williamson & Kingston, 2021). Because of this freedom, foundations can take risks, support innovation, and lay down new pathways for others to follow, helping practices to become mainstream – allowing them an influence on funding models and on funder practices that can outstrip their relative size in the funding ecosystem. This influence applies as much to evaluation practices and to reporting relationships as to anything else.
Foundations are much freer than government funders … and can take risks, support innovation, and lay down new pathways for others to follow, helping practices to become mainstream.
One way to think about the current and potential role of foundations in the evaluation and reporting ecosystem is through the lens of their accountabilities. As the foundation sector in Canada has grown and become more visible, doubling in size between 2008 and 2017 to represent $84.4 billion in assets (PFC, 2017), and as inequality has spiked during the COVID-19 pandemic, foundations’ legitimacy and ways of operating have come under renewed critical scrutiny. Beyond the traditional focus on their own boards and donors, foundations are facing calls to consider their accountability to the broader public, as their funds are underwritten by tax subsidies, as well as to the partners and communities with whom they work. Some have argued that an implicit social compact requires foundations to properly account for the contributions that they are making to positive social change through their programs and funding – in other words, their outcomes and impact (Berman et al., 2019; Rourke, 2014).
Many foundations are in fact engaging with pressing social, economic, and environmental challenges. In addition, with their knowledge of the work carried out by multiple organizations to address these challenges, philanthropic foundations are seen by many to have a special role in generating and sharing actionable knowledge about “what works” (Ebrahim & Rangan, 2014). Efforts are currently underway to design a data strategy for the philanthropic sector, which could include information-sharing about grant activities and outcomes, gathered at least partly through reporting and evaluation. This strategy aims to engage the foundation sector with a broader push within the sector to learn from evidence-informed policy-making and service delivery.
However, in thinking about outcomes and impact, foundations have very often shifted the burden of accountability onto grantees. In other words, rather than examine the impact of their own decisions and practices, more often than not foundations use grantee performance and outcomes as proxies for their own performance and impact (Leat, 2006; Beer et al, 2021). Left unexamined, this kind of thinking about outcomes and impact can reproduce some of the same problems created by government funding models and private investment practices.
Trust-based philanthropy and accountability to communities
A significant part of the philanthropic sector has begun to shift toward more trust-based forms of philanthropy, aligning with a philosophy and stance that recognizes community stakeholders as the experts of their own context. Through a range of practices, including unrestricted funding, this shift supports their ability to come up with the most appropriate responses to the needs and challenges that they are contending with (Ayer & Anderson, 2022). This trust-based framework centres the accountability relationship that foundations have with the partners and communities that they work with (Beer et al., 2021), and a significant measure of success becomes the quality of these relationships.
The trust-based philanthropy movement is increasingly seen as a way of shifting power over resources and decision-making to communities and to the organizations that represent and advocate for them. This is reflected in evaluation and learning approaches that respect and support grantees’ capacity to define and measure the outcomes that matter most to them, and that recognize the value of real-time learning to support ongoing improvement and adaptation (Tello-Rozas et al., 2022).
What would it take for foundations to attend to community relationships and outcomes while also assessing and accounting for the outcomes of their own actions?
Conversely, the trust-based approach presents some challenges for aggregation and comparison of outcomes within and across sectors, as the focus is not so much on “what works” but on the deep specificities of what works for whom, when, and under what circumstances (Bare, 2010). What would it take for foundations to work to reconcile both of these aims? While committing to practices that prioritize relationships and community partners’ agency and autonomy, how can they also generate valuable evidence about what works? And what would it take to attend to community relationships and outcomes while also assessing and accounting for the outcomes of their own actions?
Trilby Smith, an evaluator with long-standing connections to a number of foundations, believes these questions should be further explored. “A lot of funders have undertaken the move towards trust-based philanthropy since the beginning of the pandemic,” she notes, “but not many have thought through the implications that this has in connection with learning and evaluation. There is a lot of room for generative conversations to be had in the sector around this and for innovative practices to be brought in and shared more widely.”
There is a lot of room for generative conversations to be had in the sector and for innovative practices to be brought in and shared more widely.
Trilby Smith
Looking to the practices of various proponents and pioneers, there are already elements of a vision for what an evaluation and learning ecosystem could look like that brings together these accountabilities, and for what foundations can do to foster and act consistently with this vision:
1) Continue to give primacy to relationships, where communities and foundations as allies and collaborators co-produce a learning and evaluation agenda that is shared and managed by all. This is possible only if foundations and communities also work together to co-produce the visions and strategies for change that they agree to act on together – a dialogue process that, to be real, must invite and make generous space for dissension and criticism (Beer et al, 2021).
2) Continue to invest in evaluation for ongoing learning while maintaining a commitment to understanding how actions and strategies are producing change or contributing to it. A commitment to unpacking causal links is vitally important to the complex, uncertain work that seeks to advance equity and justice, where gaps in knowledge and faulty problem framings can perpetuate inequities. Foundations, non-profits, and evaluators can embrace the range of participatory methods that enable causal analysis in ways that honour and engage the knowledge of multiple stakeholders (Lynn et al, 2021).
3) Carve out a space in which foundations pay rigorous attention to their own role and assess the impacts – positive and negative – of their own decisions and practices, not just those of their grantees. This needs to be accompanied by a commitment to greater transparency, allowing for the results of these assessments to be shared beyond foundation walls. There is much room to improve here: the most recent figures show that only 17% of Canadian foundations shared the results of any type of evaluation externally (PFC, 2017).
Conclusion
Impact, improvement, and success will continue to be responsibilities that public-benefit organizations must grapple with. Accountability relationships with funders need to be rethought to remove sources of distortion and noise in this equation. Philanthropy is only one part of a resourcing landscape that also includes government funders and, increasingly, investors. As funders, foundations have many opportunities to be unhelpful, to contribute to unproductive and dishonest dynamics around how success and impact are defined and measured.
However, foundations are also well positioned to step up to a collective responsibility to create a different dynamic, one that is needed for the collaborative undertaking of understanding, sharing, and communicating our real successes and failures along the road to transformation. In the space of just a couple of years, fundamental changes appear to be underway in the philanthropic sector, and the conversation about trust-based philanthropy has rapidly come to dominate discourse. In embracing this shift and all the good that it brings for relationships with communities, foundations should also be reflecting on and communicating about how they want to work with their partners to produce useful and scalable knowledge and evidence.
References
Ayer, W. & Anderson, P. (2022). Trust & Impact: Funders’ Perspectives on Unrestricted Funding in Canada. Imagine Canada.
Bare, J. (2010). “Philanthropy, Evaluation, Accountability, and Social Change.” The Foundation Review, 1(4), 84–104.
Beer, T., Patrizi, P., & Coffman, J. (2021). “Holding Foundations Accountable for Equity Commitments.” The Foundation Review, 13(2). https://doi.org/10.9707/1944-5660.1565
Berman, M.A., Karibi-Whyte, R. & Tarasov, O. (2019). Social Compact in a Changing World:
How Philanthropies are Grappling with Growing Scrutiny and Critique. Rockefeller Philanthropy Advisors.
Chouinard, J.A. (2013). “The Case for Participatory Evaluation in an Era of Accountability.” American Journal of Evaluation, 34(2), 237–253.
Depelteau, J. (2013). Enjeux liés au financement et à la gouvernance des organismes communautaires québécois. Revue de littérature. Institut de recherche et d’informations socio-économiques.
Dougherty, C. (2019). “Trust and Transparency: Accreditation and Impact Reporting by Canadian Charities.” Canadian Journal of Nonprofit and Social Economy Research, 10(1), 6–25.
Ebrahim, A. & Rangan, V.K. (2014). “What Impact? A Framework for Measuring the Scale and Scope of Social Performance.” California Management Review, 56(3), 118–141.
Elson, P. & Carmichael, P. (2022). “A Short History of Voluntary Sector–Government Relations in Canada (revisited).” https://thephilanthropist.ca/2022/04/a-short-history-of-voluntary-sectorgovernment-relations-in-canada-revisited
Grasse, N. & Lam, M. (2021). “Financing Canadian Charities: The Conditional Benefits of Revenue Diversification.” In Phillips, S.D. & Wyatt, B. (eds.), Intersections and Innovations: Change for Canada’s Nonprofit Sector. Muttart Foundation.
Lalande, L. & Cave, J. (2017). Measuring Outcomes in Practice: Fostering an Enabling Environment for Measurement in Canada. Mowat Centre.
Lasby, D. (2011). “What T3010 Data Tell Us about Charity Financing.” The Philanthropist, 24:2, 155–160.
Lasby, D. (2019a). State of Evaluation in Canada: The Role of Funders. Imagine Canada.
Lasby, D. (2019b). The State of Evaluation: Measurement and Evaluation Practices in Canada’s Charitable Sector. Imagine Canada.
Leat, D. (2006). “Grantmaking Foundations and Performance Measurement: Playing Pool?” Public Policy and Administration, 21(3), 25–37.
Lynn, J., Stachowiak, S., & Coffman, J. (2021). “Lost Causal: Debunking Myths about Causal Analysis in Philanthropy.” The Foundation Review, 13(3). https://doi.org/10.9707/1944-5660.1576
OECD (2021). “Social Impact Measurement for the Social and Solidarity Economy: OECD Global Action Promoting Social & Solidarity Economy Ecosystems.” OECD Local Economic and Employment Development (LEED) Papers, No. 2021/05, OECD Publishing. https://doi.org/10.1787/d20a57ac-en
Philanthropic Foundations Canada (2017). A Portrait of Canadian Foundation Philanthropy: Canada National Report for the Global Philanthropy Project at Harvard Kennedy School. https://pfc.ca/wp-content/uploads/2022/01/portrait-cdn-philanthropy-sept2017-en.pdf
Phillips, S.D., Laforest, R., & Graham, A. (2010). “From Shopping to Social Innovation: Getting Public Financing Right in Canada.” Policy and Society, 29, 189–199. https://doi.org/10.1016/j.polsoc.2010.06.001
Phillips, S.D., & Carlan, V. (2018). “On Impact: Emerging Challenges of Evaluation for Canada’s Nonprofit Sector.” In K. Seel (ed.), Management of Nonprofit and Charitable Organizations in Canada. 4thed. LexisNexis.
Pole, N. & L. Rey (2022). Conceptions de l’impact chez les fondations subventionnaires : Quatre orientations types. In Fontan, J-M, Alalouf-Hall, D., & Bergeron, C. (eds.), États de lieux sur la philanthropie subventionnaire québécoise. Presses de l’Université du Québec.
Rourke, B. (2014). Philanthropy and the Limits of Accountability: A Relationship of Respect and Clarity. Philanthropy for Active Civic Engagement and Charles F. Kettering Foundation.
Ruff, K. (2021). “Social and Environmental Impact Measurement.” In Phillips, S.D. & Wyatt, B. (eds.), Intersections and Innovations: Change for Canada’s Nonprofit Sector. Muttart Foundation.
Tello-Rozas, S., Léonard, M., & Lussier-Lejeune, F. (2022). Mise à jour du portrait des pratiques d’évaluation dans les organismes communautaires [Research report]. Centre de formation populaire, Réseau québécois de l’action communautaire autonome, Relais-femmes, Service aux collectivités de l’Université du Québec à Montréal.
Williamson, A.K. & Kingston, K.L. (2021). “Performance Measurement, Evaluation and Accountability in Public Philanthropic Foundations.” Evaluation Journal of Australasia, 21(2) 101–119. https://doi.org/10.1177/1035719X211000880
