Charity Insights Canada, a rapid-response survey project run out of Carleton University’s School of Public Policy and Administration, has three main pillars: collect quality, near-real-time data; advocate for better public policy; and provide community education through data literacy. Will the sector buy in?

Starting this summer, a group of Canadian charities will participate in a pilot project that might transform how the sector uses data. This pilot is part of a $3.5-million five-year initiative called the Charity Insights Canada Project. It took a significant leap of faith from the donors – the Muttart, Lawson, Vancouver, and Metcalf foundations, plus one anonymous donor – to invest in creating a recurring near-real-time survey about charities and, eventually, non-profits.

“It is a big challenge,” admits Paloma Raggo, assistant professor at the School of Public Policy and Administration at Carleton University and director of the Charity Insights Canada Project.

Relevant information is already being collected about the charitable and non-profit sector. For instance, Imagine Canada’s Sector Monitor digs deep into many issues. But data from government sources are mostly published with an 18-month lag. “We plan for our results to be available within 48 hours. That is a major commitment,” Raggo says.

The Philanthropist Journal talked to Raggo to understand how this project could help non-profits fulfill their missions and the government better assist them.

Tell me a bit about yourself. What is your area of interest?

My PhD was about leadership and accountability. I asked the CEOs of 152 Canadian non-profits about their vision of accountability. Three visions emerged. The first group associates accountability with sound financial reporting. They focus on the use of their funding. CEOs of the second group define accountability as fulfilling their mission. The third group describes accountability as respecting the rules. They ensure they fill in all the required documents and check all the boxes. My main conclusion from these discussions is that a CEO’s vision of accountability influences her daily and long-term decisions. CEOs from the first group don’t manage like those from the second and the third group. It is a different way of studying accountability: instead of focusing on the donor/charity relationship, I shed light on the “politics of accountability.” When you hold power, what you decide depends on what you believe you are accountable for.

Let’s talk about the Charity Insights Canada Project. Where did the idea come from?

The Muttart Foundation wanted to fill a data gap. The sector stakeholders share the same feeling about the data they receive: “It is interesting, but I needed to access it earlier.” Bob Wyatt, Muttart’s executive director, discussed this problem with my colleague Susan Phillips. He wondered if a rapid response panel inspired by business associations’ surveys was possible for the charitable sector. Since Bob is a recurring lecturer in Carleton’s Master of Philanthropy and Nonprofit Leadership program, he knows our university’s strengths.

What is Carleton’s School of Public Policy’s expertise with charitable data collection?

The Master of Philanthropy and Nonprofit Leadership program includes a class on research methods. The idea came from my colleague Professor Susan Phillips, who is the graduate supervisor of the program. There is a recurring conversation about how critical data is to the charity sector, but the divide lingers between the “data literate” and the “data illiterate.” Our research-methods class aims to narrow that gap.

The Charity Insights Canada Project has three pillars. What are they?

First is the research part, including data collection. Every week, we will send two to three questions to a panel of 1,000 Canadian charities and non-profits; answering should take between two to three minutes. There will be perception questions (What is your take on the budget?), organizational questions (What is your turnover? Your most costly activity?), and strategy questions (What is your most pressing challenge?). We have yet to decide if questions will be sent by email or text message.

The second pillar is related to public policy. Each month, we will publish a policy brief inspired by the answers to the four weekly surveys. We will point to issues needing more research and start conversations about the possible content of that research. Those briefs should help non-profits get the info required to engage with governments and donors – and the latter to better target their contribution.

The third pillar is community education through data literacy. It was not an original request of the Muttart Foundation. We added it because, as a university, education is our core competency. We will produce podcasts and videos answering big and small questions. What is a variable? What does “error margin” mean in a survey? Can a photo or an interview be considered data? Data literacy also implies data organization. Therefore, the Charity Insights Canada microsite will explain how to maximize the use of Google Drive and how to structure data collection and store data. Too few non-profits have a standardized method to collect info about participants in their activities. And not enough employees know how to access this info.

The project includes a weekly survey of a panel of 1,000 charities and non-profits. How will you select the questions?

We are creating a 15-member advisory committee with different profiles: sector representatives, academics, and civil servants. It is a demanding mandate; 100 questions must be produced over one year. There will be some recurring questions – to test the state of mind of respondents at different times of the year – while others will reflect the current social, financial, and political context. Although the survey will prioritize breadth over depth, we want to cover various sector issues.

What is the ultimate goal of this project?

I want to simplify the daily life of the employees and the managers of non-profits. If they are better organized, with valuable data easily accessible to all and easy to share, they will get more time to fulfill their mission.

You feel this project could be transformational. How? 

Usually, when you shed light on something obscure, it gains value. Actively managing data empowers non-profits. It enriches their discussions with third parties and creates possibilities. For example, collaboration between non-profits and researchers is underdeveloped. We, the academics, could help in many concrete ways. Take data security, for instance: universities have the infrastructure and the knowledge to host data for small non-profits with scarce resources.

This project is a leap of faith. What are the challenges?

I foresee two challenges. First, to recruit a representative and engaged pool of respondents and keep them with us for five years. Second, to publish the results 48 hours or so after we receive them. Those 48 hours are needed to verify the results, control their quality, and draw some conclusions and add comments to provide context to the results.

How do you plan to overcome these challenges?

In terms of the recruiting, we bet on the rapid response. The survey industry is basically extractive: someone takes your info and, at best, you get feedback months or years later. At worst, you never hear about the results. Our survey promises rapid feedback, so respondents can benchmark their organization. That could be an incentive. As for the 48-hour publishing deadline, well – I rely on the expertise of my data scientist colleagues! They develop scripts and code to generate graphs from quantitative and qualitative answers.

You already have big plans for year two. What are they? 

I do have big plans! Core funding is rare, and core funding in infrastructure is even more rare. I will invest five years of my research life in this project – I do not want it to fail. The sector buy-in is critical; therefore, I plan to create the most interactive experience possible. I want users to propose survey questions on the project platform by year two. And I aim for a bilingual experience. Respondents should be able to select the survey’s language. As a francophone, this is dear to my heart; I hope for an authentic Canadian conversation.

How it all started: The Muttart Foundation ask

“The last comprehensive industry survey dates back to 2003, and people still use these stats!” says Bob Wyatt, executive director of the Muttart Foundation. The pandemic moved things to another level. “Suddenly, our decision time was two weeks, and we were relying on data that was a few months old and, at worst, a few decades old. It did not make any sense.”

So how did other organizations rise up to the data-collection challenge? “I noticed that business associations survey their members regularly about various issues,” Wyatt says. Could the non-profit sector do the same? To find out, Wyatt reached out to Carleton University. “They are leading academics in the non-profit field, and their IT department is solid. I thought these were two good reasons to discuss data collection and management with them.” 

We believe these are risks worth taking; we need more evidence-based decision-making from all stakeholders. This is what this project is all about.

Bob Wyatt, Muttart Foundation

The Carleton team (Paloma Raggo, Susan Phillips, and Nathan Grasse) came up with a proposition with a twist: it includes an education component. Users will receive data literacy through videos and podcasts. And since all the survey results will be open source, the platform visitors will be able to generate new data that suits their specific needs.

Teamwork makes the dream work

The Muttart Foundation board agreed to $1 million in financing for the project, subject to finding other donors. Wyatt contacted five foundations; four of them accepted, for a total of $3.5 million. The group has very different founding interests, and they operate in a wide range of areas. But they all came together around the same objective: to increase the charitable and non-profit sector’s capacity through data. Within two months, all documents were signed, and the project was ready to start. 

The first phase will last three years. Then, the donors and the academics will sit together to review the outcome and decide to pursue, adapt, or stop. The donors are aware of the risks, the main one being the charities’ collaboration. Wyatt bets on the fact that “most charities acknowledge that no organization is an island. They are part of an ecosystem. And if the government or the donors make decisions without data or based on unreliable data, then it affects all the sector.”

He adds, “We believe these are risks worth taking; we need more evidence-based decision-making from all stakeholders. This is what this project is all about.”