ICHQ Brief Series: Understanding Implementation and Costs to Support Quality in Early Care and Education

Publication Date: March 2, 2022

Introduction

Research Questions

  1. What implementation activities support each key function of center operations and can we reliably measure implementation of each function?
  2. How do staff members use their time in support of key functions within the center?
  3. What are the costs associated with the implementation of key functions?
  4. Are differences in center characteristics related to implementation and costs?
  5. Can implementation and cost measures be aligned to inform decisions to guide quality improvement?

A growing body of research demonstrating the benefits of high quality early care and education (ECE) programs for young children, particularly those in families with low incomes, has inspired a strong commitment from both federal and state officials to improve the quality of ECE programs nationwide. Policymakers, administrators, and program and center directors have limited guidance about how to fulfill this goal and would benefit from actionable information about the cost of high-quality care and the use of center-level resources to guide decision-making.

The Office of Planning, Research, and Evaluation (OPRE) in the Administration for Children and Families contracted with Mathematica to develop two sets of center-level measures to capture (1) implementation of activities that can support quality in ECE centers serving children from birth to age 5 (not yet in kindergarten) and (2) the costs to provide care and services. By summarizing how services are provided, how center resources are used, and the associated costs of care and services, the combined ICHQ implementation and cost measures will help policymakers, administrators, and program and center directors connect decisions about day-to-day operations to the larger question of how to allocate limited resources to provide high quality ECE.

The ICHQ implementation and cost measures are framed around five key functions, or areas, of ECE center operations that contribute to high quality care. The five key functions of center operations are: (1) Structural Supports for Instruction and Caregiving; (2) Instructional Planning, Coordination, and Child Assessment; (3) Center Administration and Planning; (4) Workforce Development; and (5) Child and Family Support. Each of the five key functions are defined by a specific set of activities and practices that allow us to measure implementation and costs for each function distinctly. All ECE centers carry out the key functions to varying degrees to provide services to young children and their families. Constructing measures around each of the five key functions will help the ECE field better understand how implementation and costs in specific areas relate to quality.

This series of briefs highlights and adds to findings presented in an associated methods paper that are based on a multi-case study of 30 ECE centers. The multi-case study, conducted in 2017-2018, was a key step to develop draft measures and explore how well they are working to summarize implementation, estimate costs, and identify ways centers can support quality. The measures are being further tested and validated based on results from a 2021 field test with a larger sample of centers.

ICHQ Brief Series:

Purpose

The goals of ICHQ are to (1) produce technically sound, systematic measures of the implementation and costs of education and care in center-based settings that serve children from birth to age 5 (not yet in kindergarten); (2) produce implementation and cost measures to examine the variation in ECE center capacities and resources that can make a difference in quality and the experiences of children; and (3) develop a feasible and useful instrument to guide the collection, development, and reporting of the measures. The measures are intended for use in research to examine and explain variations in quality, in practice to inform quality improvement, and in policy to inform funding and technical assistance.

Key Findings and Highlights

Early testing with a purposive sample of 30 centers suggests that the ICHQ measures capture important information about implementation and costs and hold promise for working together to inform pathways to quality. Each of five briefs focuses on different aspects of the measures development work and present findings about the draft measures:

  • Measuring Implementation to Understand Supports for Quality in Early Care and Education Centers describes the development of the implementation measures and how they summarize the level of implementation for each of the five key functions.
  • Developing Reliable and Valid Measures of Implementation to Support High Quality in Early Care and Education Centers discusses the psychometric properties of the ICHQ implementation measures, the relationship among the five implementation measure scores, and their validity based on patterns with observable center characteristics. The implementation measure scores for each of the five key functions produced reliable scales, have distributions that can detect differences across centers, and are related to each other, but each also provides distinct information.
  • Measuring Costs to Support Quality in Early Care and Education Centers describes the ICHQ cost measures that estimate the amount and allocation of resources used to support the ECE services a center provides. The ICHQ cost measures delve deeper than total costs to provide estimates of costs across distinct resource categories and the five key functions of center operations. The centers in the ICHQ sample directed 33 percent of costs to the Instruction and Caregiving function, on average, and 22 percent of costs to the Instructional Planning, Coordination, and Child Assessment function.
  • Measures of Implementation and Cost that Work Together to Support Quality in Early Care and Education Centers presents findings about the relationship between the ICHQ implementation and cost measures and implications for how they can be used together to better understand how to support quality in ECE centers. The implementation measure scores and cost per child care hour for each key function are positively related, meaning the higher the implementation score, the higher the cost of the function.
  • Exploring Staff Time Use in 30 Early Care and Education Centers examines the different ways that teaching and administrative staff in ECE centers use their time. Center staff reported spending their time in expected ways. Directors reported spending more of their time on Center Administration and Planning than other staff. Lead teachers, head teachers, or co-teachers reported spending more of their time on Instructional Planning, Coordination, and Child Assessment relative to assistant teachers, who reported spending more of their time on Instruction and Caregiving.

Methods

The multi-case study with 30 ECE centers included (1) conducting semistructured interviews about the intentionality, structure, and consistency of implementation of key functions of a center that can support quality; (2) collecting cost data through Excel workbooks to assess center-level costs by key function and how resources are used across functions; and (3) administering surveys about staff time use—essential information for allocating costs to key functions, since labor is a large driver of costs in ECE centers.

Implications for Next Steps

The findings from the multi-case study provide preliminary evidence that the measures capture variations in implementation and costs among centers and are working together in ways that have the potential to inform pathways to quality. The measures are not yet fully validated, meaning they have not been tested in a large, representative sample of centers to look at associations with center quality or children’s outcomes. A field test in 2021 is further testing the measures with a purposive sample of 80 centers in 4 states.

The ICHQ implementation and cost measures have the potential to support the efforts of researchers, practitioners, and policymakers to better understand pathways to achieving high quality. They can also inform decisions about the level of resources needed and how to best use resources across key functions at the center level to deliver high quality early care and education

Citation

G. Kirby, S. Monahan, and J. Lyskawa. “Measuring Implementation to Understand Supports for Quality in Early Care and Education Centers.” OPRE Brief #2022-22. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, US. Department of Health and Human Services, 2022.

S. Monahan, G. Kirby, and K. Gonzalez. “Developing Reliable and Valid Measures of Implementation to Support High Quality in Early Care and Education Centers.” OPRE Brief #2022-21. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, US. Department of Health and Human Services, 2022.

T. Schulte Neelan and P. Caronongan. “Measuring Costs to Support Quality in Early Care and Education Centers.” OPRE Brief #2022-20. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2022.

P. Caronongan, G. Kirby, K. and Gonzalez. “Measures of Implementation and Cost that Work Together to Support Quality in Early Care and Education Centers.” OPRE Brief #2022-19. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, US. Department of Health and Human Services, 2022.

K. Gonzalez and P. Caronongan. “Exploring Staff Time Use in 30 Early Care and Education Centers.” OPRE Brief #2022-17. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, US. Department of Health and Human Services, 2022.

Glossary

ECE:
Early care and education
Center:
Refers to a specific physical location where ECE classroom-based services are provided to children birth to 5 years (not yet in kindergarten).
Key Functions:
Five areas of center operations that contribute to high quality care. Each of five key functions are defined by a specific set of activities and practices. All ECE centers carry out the key functions to varying degrees to provide services to young children and their families.
Implementation measures:
Scores that summarize what a center does to support quality, including the combination of structural features (for example, teacher–child ratios, group size, and staff qualifications) and adopted practices, as well as how features and practices are supported.
Cost measures:
Estimate the amount and allocation of resources needed to support the ECE services a center provides, including how staff use their time.