Introduction
Federal agencies often have experience with building and using evidence to design, manage, and improve their work on behalf of the public. All agencies, regardless of their experience level, can benefit from assessing their capacity to build and apply evidence in their work. The brief presents an evidence capacity framework that draws on research literature supplemented with interviews and focus groups with federal staff. Federal agencies could use this framework to inform their evidence capacity assessments required under the Foundations for Evidence-Based Policymaking Act of 2018 (the Evidence Act). Specifically, agencies could use this framework to lead qualitative self-assessment and reflection about their current capacity and to begin strategic conversations about ways to strengthen capacity within their agency. The framework can help agencies develop a shared understanding of the capacities they are trying to develop and the opportunities they could unlock by developing those capacities. Likewise, state and local government agencies, as well as private-sector human services organizations, can use the framework to assess their capacity to build and use evidence about their own work and contribute evidence as requested by federal agencies.
Purpose
The purpose of the evidence capacity framework presented in this brief is to help agencies and other organizations develop a shared understanding of the dimensions of capacity described in the literature. Developing this shared understanding is a step toward identifying organizational strengths and opportunities to develop or deepen specific evidence capacities. Organizations can use the framework to guide an assessment of their evidence capacity through discussions among staff.
Key Findings and Highlights
This brief describes a framework for evidence capacity. The brief introduces the framework’s five dimensions and the components that comprise each dimension. The dimensions are:
- Evidence culture, demonstrated as an organization that routinely uses evidence to support the organization’s mission, objectives, program and policy choices, and decision-making.
- Evidence infrastructure, shown through use of evidence-related tools, resources, routines, and processes that form an infrastructure to build and use evidence in a timely and efficient manner.
- Engagement, which involves having systems and processes to promote collaboration within and across internal and external audiences to advance informed and equitable evidence building and evidence use.
- Human capital, described as the knowledge, skills, and abilities of the organization’s staff to build evidence and communicate about it, as well as the opportunities for staff to develop this expertise.
- Leadership, depicted through leaders who foster an evidence culture, identify opportunities to build evidence, allocate time and resources to evidence activities, and make decisions transparently informed by evidence.
The framework then organizes components by the individual, interpersonal, or organizational levels. A graphic with three concentric circles depicts these levels and identifies the relevant components by dimension. Most components pertain to evidence capacity at the organizational level, with a few components at the interpersonal level and one component at the individual level. This suggests that much of evidence capacity is driven by organizational efforts and cannot be sustained by a few trained individuals in a silo.
Methods
To develop the evidence capacity framework, the team searched for literature published from 2006—2021 that addressed evidence capacity or evaluation capacity. The search identified 49 sources that described an evidence capacity framework or components of a framework. The sources consist of research studies of capacity building and toolkits and guidance to help organizations build evidence capacity.
We reviewed each source to identify the components of evidence capacity it described (that is, inputs, outputs, or activities involved in evidence capacity). Then, for each component, we extracted the component’s definition and associated measures. We grouped the components into dimensions of evidence capacity based on common themes. By grouping components described in the literature, we identified five dimensions of evidence capacity and 17 components. Each dimension has multiple related but distinct components.
Citation
Mastri, Annalisa, Heather Gordon, Ruth Neild, Elizabeth Alberty, Heather Zaveri, Megan McCormick, Veronica Sotelo Munoz, and Lance Bitner-Laird (2022). Evidence Capacity in Organizations: A Literature-Informed Framework. OPRE Report # 2022-303, Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Glossary
- Evidence:
- Facts, data, research, evaluation, or other information collected and used systematically to understand the effectiveness or efficiency of an organization’s work, or to understand the context or communities in which an organization conducts its work.
- Evidence capacity:
- Knowledge, skills, behaviors, and resources that support an agency’s ability to build and use evidence to make decisions and inform its work.