Challenges and Solutions to Conducting Intensive Studies in Early Care and Education Settings

Publication Date: September 1, 2020
Current as of:
The first page of the brief, entitled "Challenges and Solutions to Conducting Intensive Studies in Early Care and Education Settings"

Download Report

Download PDF (308.54 KB)
  • File Size: 308.54 KB
  • Pages: 8
  • Published: 2022

Introduction

Research Questions

  1. What lessons can the field learn from ECE-ICHQ about conducting intensive studies in center-based settings?
  2. What challenges arise when recruiting centers to participate in research studies and what are possible solutions to those challenges?
  3. What are the challenges to collecting different types of data from centers (implementation data, cost data, and time-use data) and what are possible solutions to those challenges?

Collecting data from center-based early care and education (ECE) settings poses unique challenges. Center directors and teaching staff have limited ability to participate in data collection activities because of time pressures and the immediacy of issues that arise in providing care to young children. Centers also vary widely in their size, funding, staffing and organizational structures, and quality, so instruments and methods for collecting data must be flexible enough to capture variation and be appropriate for a variety of settings.

The Assessing the Implementation and Cost of High Quality Early Care and Education (ECE-ICHQ) project, funded by the Office of Planning, Research, and Evaluation (OPRE) in the Administration for Children and Families, recruited and collected data from ECE centers as part of a five-year measures development project. This brief presents lessons learned from the ECE-ICHQ project that could prove useful in addressing challenges when conducting different types of data collection with staff in center-based settings.

Purpose

The purpose of this brief is to present lessons learned from the ECE-ICHQ project that could prove useful to the field. The brief discusses the challenges that exist when recruiting centers and conducting qualitative research, cost analysis, and self-reported data collection with staff in center-based settings and offers potential solutions to those challenges.

Key Findings and Highlights

Key challenges from recruiting centers to participate in the ECE-ICHQ study and possible solutions include:

  • There is a lack of detailed information to efficiently create lists of centers to recruit.
    • Build large lists of centers from multiple publicly available data sources and focus on a few selection criteria.
  • Reaching a decision maker within centers and obtaining a decision on study participation takes significant time and effort.
    • Begin with a large pool of centers and set parameters on the number and type of contacts to move through the sample and achieve recruitment targets quickly.
  • ECE centers often function in time- and resource-constrained environments and find it difficult to participate in studies.
    • Offer centers a service or benefit that aligns with the level of effort they need to expend to participate.

Key challenges from collecting data in centers for the ECE-ICHQ study include:

  • ECE staff have limited time for data collection activities.
    • Adapt data collection methods to address the constraints on time and availability of information that respondents in ECE centers face.
  • Ensuring data collection and analysis approaches are appropriate for a range of ECE centers is essential. 
    • Develop data collection methods and tools that are comprehensive, yet flexible to account for variation among centers.
  • Collecting data from multiple respondents over a long time can cause dropoff in participation before data collection is complete.
    • Establish clear lines of communication within the data collection team and with the center staff for multipronged or longer-term data collection.

Methods

The goal of the ECE-ICHQ study is to create an instrument to measure the implementation and costs of providing quality services at the center level for ECE programs serving children from birth to age 5. To inform measures development, ECE-ICHQ carried out a multicase study that included (1) a qualitative study of the implementation of five key functions of center-based ECE providers, and (2) a quantitative analysis of center costs and time use of staff. Data collection in each center was time intensive and spanned multiple respondents. The multicase study was conducted in phases, which offered the opportunity to refine recruitment and data collection processes and produce lessons that might prove useful for other studies. The phases of the multicase study are summarized below:

  • The Pilot phase included a convenience sample of 3 centers and on-site data collection
  • Phase 1 included a purposive sample of 15 centers in 3 states and on-site data collection
  • Phase 2 included a purposive sample of 30 centers in 3 states and remote data collection

Citation

Lyskawa, J., Kirby, G., Caronongan, P., Kelly, A., & Burwick, A. (2020). Challenges and solutions to conducting intensive studies in early care and education settings. OPRE Brief #2020-96. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, US. Department of Health and Human Services.

Glossary

ECE:
Early care and education
QRIS:
Quality rating and improvement system