In November 2018, Mowat NFP and the Community Safety Knowledge Alliance hosted Innovation in Evidence, a conference on evidence-informed policymaking. Internationally, evidence institutions like What Works Centres have been transforming how evidence is used in the policymaking process to address “wicked” social problems, ranging from social isolation among seniors to policing interventions for crime reduction.
The “why” of evidence-informed policymaking is well-understood – government programs and policies produce better outcomes for citizens and allocate resources more efficiently by learning from success in other jurisdictions. Innovation in Evidence focused on the “how” – connecting the work of existing evidence institutions to existing challenges in Canadian social policy to make evidence available in a way that is timely, useful, and easy to understand.
The Regina conference had three key objectives to address this knowledge gap:
- Identify opportunities to strengthen Canada’s evidence ecosystem;
- Inform efforts that are currently underway in Saskatchewan and Alberta to improve the use of evidence in policymaking and program delivery; and
- Explore the potential for international collaboration on social policy issues.
This article explores some of the key opportunities, promising practices, and obstacles for the Canadian non-profit and charitable sector that were identified at the conference. Two pre-conference papers — Strengthening Canadian Efforts to Identify What Works (and What Doesn’t) in Social Policy, published on Mowat NFP, and Supporting Evidence-Informed Policymaking and Service Delivery in Canada’s Non-Profit Sector, published in The Philanthropist — provide additional context.
Opportunities and promising practices
- Focus limited resources on consolidating existing evidence, rather than conducting new research. One of the biggest challenges is harnessing existing evidence and applying it, rather than generating new evidence. If a non-profit is interested in improving its use of evidence-based practices, one of the easiest and most cost-effective approaches is to dedicate staff resources (or hire an external researcher or partner with an academic institution) to consolidate existing evidence. This can include reviewing systematic reviews, research reports and studies from other jurisdictions of what works in a particular issue area. This approach will be more efficient and less resource-intensive than conducting a new research project or randomized control trial.
- Evidence is only as useful as an organization’s ability to access, understand, and implement it in practice. Many conference participants emphasized that a lack of evidencecapacityamong frontline practitioners and policymakers remains a significant barrier for the adoption of evidence. In the UK, the National Institute for Health and Care Excellence has experimented with “field teams” that work directly with policymakers or practitioners to incorporate research findings into their day-to-day work. PolicyWise is one example of a Canadian capacity-building organization that provides technical assistance and training to non-profit organizations that are looking to improve their research and evaluation practices.
- Collaboration will maximize return on
investment. Duplication is an
ongoing challenge, particularly for organizations with limited resources. While
there are many established evidence institutions in Canada, conference
participants stressed the need for more collaboration across organizations and
sectors. Mowat NFP launched a discussion paper and crowdsourced platform for mapping Canada’s evidence ecosystem at the
conference to promote greater collaboration and awareness about existing
Canadian initiatives. Collaboration is particularly important when
organizations are considering building new evidence platforms or conducting new
research projects – both of which can be costly and time-intensive endeavours.
- Beneficiaries and service users have an important role in
guiding evidence-based program design and policymaking. Both should define the objectives and scope of a research
project, collect data and analyze key findings. Non-profit organizations can
encourage greater participation by establishing advisory committees, inviting
service users/beneficiaries to participate in research teams and committees, or
hiring individuals to act as peer researchers (e.g. focus group facilitators).
Diane Roussin of the Winnipeg
Boldness Project provided a useful example of how
community members identified culturally appropriate, easy-to-understand
alternatives to conventional research tools like the Early Development
Instrument.
- Non-profit organizations should explore qualitative research methods – in addition to quantitative methods – to understand the needs of the individuals they serve. Many organizations defer to quantitative research to evaluate outcomes through administrative data or surveys, but qualitative research methods can add enormous value. Sarah Schulman of InWithForward described their process of “journey mapping” with vulnerable individuals to better understand the gaps and challenges they experience in accessing services. These types of research methods can be particularly helpful in understanding issues in a deeper way and evaluating the impact of a program or service in a broader systems-level context.
Potential obstacles
- Deciding on standards of evidence will pose a challenge. Many conference presenters noted the importance of having a consistent standard of evidence to evaluate and compare evidence from different sources. Ideally, standards of evidence provide a framework to contrast evidence across sub-sectors (e.g. the EMMIE standard used by the What Works Centre for Crime Reduction). It is important to decide on a standard of evidence early in the process and, where possible, align with other organizations on the same standard.
- It is difficult to align research and policymaking timeframes. Measuring outcomes – particularly systems-level outcomes – takes time. The funding cycle for evidence is often disjointed from the research process and this can create pressure for non-profit organizations to demonstrate outcomes in too-short timeframes.
- Backbone organizations are often needed to develop the evidence ecosystem in a particular issue area. This is one of the most significant assets of the What Works Centre model – one organization can act as the “clearinghouse” or central repository of data and evidence and assist with the process of knowledge translation. In Canada, organizations like the Canadian Observatory on Homelessness serve this role and assist in educating the sector on research methods and promising practices for homelessness interventions.
- Funding remains a challenge for non-profit organizations that want to build their evidence capacity. While funders emphasize the importance of evidence-based programs, it can be challenging for organizations to build their internal capacity to work with evidence in a more sophisticated way. Staff training and professional development, funding for external research support, software, and website development can be costly.
Advancing the conversation: what happens next?
Mowat NFP hosted a second convening session on evidence-based policymaking in Toronto in February 2019. This session focused on issues specific to the Ontario context, and built on the key learnings from Regina to develop a strategy for building Canada’s evidence ecosystem. Mowat NFP will publish a paper summarizing key findings and next steps from both the conference and convening event later this spring.
Mowat NFP’s papers on building collaborative data infrastructure, measuring outcomes in practice, and reimagining organizational governance provide additional resources for non-profit organizations that are looking to build their evidence capacity.
