Improving Part C Results and Compliance: A Six-Step Inquiry Cycle

To support states in improving educational results and functional outcomes for children with disabilities while ensuring compliance with Individuals with Disabilities Education Act (IDEA) through state monitoring activities, a six-step inquiry process has been developed.

The six steps describe a chronological process to assist states in selecting a focus area, collecting and using data to monitor this focus area, completing root cause analyses of the current performance, planning for and evaluating progress to resolve issues effectively, and implementing continuous and sustainable improvement. Since improving results and ensuring compliance is an ongoing process the six steps of inquiry should be used as an iterative cycle as infrastructure and practice improvements continue to be made.

Background

In April 2010, a National Think Tank on Streamlining and Integrating Part C General Supervision Activities: Monitoring and Program Improvement was convened to develop resources and materials to help states identify where and how monitoring and program improvement activities could be streamlined and better integrated to meet both state needs and national reporting requirements. This think tank of state coordinators and TA center representatives was convened to address a growing need for streamlining and integrating the various general supervision activities with the newly required Annual Performance Report (APR), including using states' data systems for monitoring.

The think tank also refined a six-step framework, originally conceptualized by Western Regional Resource Center (WRRC) in July 2009. An online interactive guide based on the proceedings of the think tank, Streamlining and Integrating Part C General Supervision Activities: Monitoring and Program Improvement was published in 2010 and updated in 2012. The six steps in this guide are heavily focused on the basics related to identifying and correcting noncompliance.

In the decade since this guide was first published, state Part C lead agencies have made significant progress in streamlining general supervision activities and in developing general supervision methods focused on improving outcomes for infants and toddlers with disabilities and their families. Increasingly, state Part C general supervision systems continue to ensure that local early intervention programs are compliant with IDEA requirements but now also focus on improving outcomes for children and families. As a result, local programs are taking on more responsibility for both ensuring compliance and improving outcomes.

The Early Childhood Technical Assistance (ECTA) Center, in collaboration with the National Center for Systemic Improvement (NCSI) and the Center for IDEA Early Childhood Data Systems (DaSy), completed this guide in February 2021 to reflect the changing general supervision context in states and OSEP's focus on Results Driven Accountability (RDA). The guide continues to be organized according to a similar six-step process and addresses the increasing focus on improving results while ensuring compliance.

The reader should note this resource is not intended to guide the development of a state's complete accountability and quality improvement system. The ECTA System Framework with its accompanying Self-Assessment can be used as a resource to support states to assess their current accountability system and plan for revisions and enhancements. In addition, states must ensure implementation of all the required steps related to identifying and correcting noncompliance as outlined in A State Guide on Identifying, Correcting, and Reporting Noncompliance with IDEA Requirements. This resource reflects the OSEP 09-02 Memo and the FAQ on Identifying and Correcting Noncompliance. The six steps for improving results and compliance are outlined below.

This on-line guide presents each step separately, providing a description of the step, activities to complete the step, the associated potential pitfalls, and relevant resources as available.

Step 1: Select and Refine a Focus for Improvement or Correction

The purpose of early intervention is to improve results for infants and toddlers and their families. High-quality early intervention relies on several components, including:

• Efficient and effective state and local infrastructure that supports the provision of evidence-based practices and compliance with federal requirements;
• Implementation of evidence-based practices that are provided:
  • In compliance with federal and state requirements.
  • Through meaningful parent or family engagement in developing and implementing high quality IFSP s.
  • Within the family's routines and activities.

  The first and most important step in improving early intervention results and compliance is to make data-informed decisions to select and refine an area of focus. Anytime throughout this process, the state may review examples of how other states have planned for and achieved improvement within their systems.

  Activities for Step 1

  1. The state, with stakeholder input, selects a focus for improving results for infants and toddlers and their families such as:
     1. Child and/or family outcomes;
     2. Evidence-based practices; and
     3. State and local compliance with IDEA.
     1. Quantitative and qualitative data from components of the system including but not limited to data on results and compliance indicators, on-site monitoring, state databases, self-assessment, family survey data, dispute resolution, fiscal, personnel, etc.;
     2. Potential new qualitative and quantitative data that can provide additional insights or knowledge about results and/or compliance;
     3. Current state data on practitioners' implementation of evidence-based practices and the outcomes achieved as a result of implementation;
     4. Outcomes achieved through the State Systemic Improvement Plan (SSIP) or other improvement initiatives;
     5. Priorities recommended or established by federal requirements, state leadership, ICC, legislature, and other stakeholders;
     6. Any additional data as needed (e.g., that identifies the extent of any noncompliance or performance issue); and
     7. The relationship of any noncompliance to child and family outcomes.
     1. Improving social-emotional outcomes for infants and toddlers with disabilities;
     2. Improving timely early intervention services; and
     3. Improving practitioner's implementation fidelity of coaching practices or interest-based learning practices.
     1. Designed to result in work that is important for the quality of the state's system;
     2. Integrated across general supervision components;
     3. Built on but does not duplicate other efforts;
     4. Based on the capacity of the state to carry out the activities recognizing that systemic change takes time to be implemented effectively;
     5. Informed by what has been learned from the SSIP and other improvement initiatives;
     6. Responsive to what is needed to meet federal and state requirements as well as to support high-quality services and improvement of results for children and families; and
     7. Cognizant of the amount of data collected and analyzed and/or needed to address the focus area.

     Avoiding Potential Pitfalls for Step 1

     • Consider different areas of focus such as achievement of results for children and families, SSIP outcomes or fiscal accountability. Avoid always selecting SPP/APR indicators or other compliance requirements.
     • Explore and be open to using all available quantitative and qualitative data and not just data from the SPP/APR indicators.
     • Be mindful of over-extending resources to collect more data than are needed for your specific priority focus.
     • Consider whether there are monitoring activities that are redundant or unnecessary that could be discontinued to create resources for selected focus areas.
     • Consider the capacity of local programs to engage in meaningful improvement given their continuing responsibilities to meet their IDEA requirements.
     • If possible, invest in state data systems that are real time and capture data on all indicators and priority areas resulting in less need for more costly monitoring activities such as onsite visits.

     Step 2: Develop an Approach for Inquiry

     Once a focus area is identified and refined, the next step is to begin to explore available data more deeply, repeatedly asking the question why, and developing hypotheses about the possible root causes of the current level of performance. An approach for inquiry organizes what is known and what needs to be learned to develop a plan for improvement or correction. Investing time and resources in thinking through an approach to inquiry will lead to a streamlined and efficient data collection and analysis process to determine the root causes of the current performance. This will lead to the development of improvement activities that are measurable and most closely related to the area of focus.

     Activities for Step 2

     1. First, the state, with stakeholder input including local early intervention programs, explores available data more deeply, repeatedly asking the question WHY, to develop potential hypotheses for the current performance. Possible approaches to inquiry include:
        1. Child and/or family outcomes;
        2. Evidence-based practices; and
        3. State and local compliance with IDEA.
        1. Asking who, what, when, where and why questions until consensus is built around a few workable hypotheses;
        2. Deciding how widespread the need for improvement or correction (e.g., one program, region or statewide) is. This includes:
           • Determining where or at what level has the most need for improvement or correction.
           • Deciding who can address the needed changes.
        3. Determining if there are infrastructure components that contribute to the current performance;
        4. Identifying what practice issues are contributing to the current performance (e.g., not implementing the practice with fidelity, inconsistently using a practice, limited understanding of the practice, practice drift); and
        5. Determining what aspects of compliance are directly or indirectly related to the focus area and the extent of correction needed as outlined in A State Guide on Identifying, Correcting, and Reporting Noncompliance with IDEA Requirements.

        Avoiding Potential Pitfalls for Step 2

        • Ensure staff and stakeholders have adequate resources and are available to explore the "whys", develop hypotheses, and plan the approach to inquiry.
        • Provide necessary professional development to support staff and stakeholders with knowledge about and experience with analyzing data to develop hypotheses related to current performance.
        • Consider inviting an outside facilitator (e.g., TA provider) to assist in exploring the data and digging deeper into why, helping the group question current assumptions and getting to workable hypotheses.

        Step 3: Collect Data and Conduct Root Cause Analyses

        Data collection and root cause analysis should be based on the hypotheses and the approach for inquiry developed in Step 2.

        "Root cause analysis is a type of in-depth analysis that is conducted to identify contributing factors that help answer the question of why low performance [related to results and/or noncompliance] is occurring. The objective is to determine what is happening [and] why it is happening to help identify what can be done to reduce the likelihood that it will continue. "

        ECTA Center, Dasy Center. (2015). Planning, Conducting, and Documenting Data Analysis for Program Improvement

        The root cause analysis needs to be systematic and sufficiently in-depth to clearly identify infrastructure and practice factors that contribute to performance in the targeted focus area. Understanding these contributing factors leads to the development of a plan that includes strategies for improvement or correction that are relevant and effective (e.g., change policies, procedures, practices; improve or provide personnel development; change or provide administrative support at state or local level.). One or more factors may be contributing to the current performance and these factors may be interrelated. All contributing factors should be something that the state and/or local program can influence or control (i.e., they are actionable).

        Activities for Step 3

        1. First, the state commits resources to or notifies the local early intervention program of its responsibilities for carrying out the root cause analysis of the hypotheses, the plan for inquiry, and the associated timelines.
        2. Next, the state and/or local program:
           1. Identify the methods (e.g., desk audits, interviews, observations, review of records) for collecting the data necessary for the root cause analysis;
           2. Determine and develops the data collection instruments to be used for conducting the root cause analysis to verify whether the hypotheses are accurate. In some situations, data collection instruments, such as for new monitoring indicators that focus on results or quality practices, may need to be developed;
           3. Collect relevant information/data to determine if the hypotheses of the root causes are accurate. The information/data include:
              • Determining where or at what level has the most need for improvement or correction; and
              • Deciding who can address the needed changes.
           4. Analyze the data and identify contributing factors in sufficient detail to identify improvement or correction activities that are meaningful and doable;
           5. Verify whether or not the hypotheses are confirmed based on data analyses. Specifically, the root cause analysis should identify:
              • What specific infrastructure challenges (e.g., policies and procedures, data collection and use, personnel, professional development, funding, supervision and accountability) and/or evidence-based practices are contributing to the low performance in the focus area;
              • Where and with whom the improvement is needed (e.g., one or more service coordinators/providers or programs, regions or statewide) based on practitioner levels of practice implementation and fidelity or data related to the child outcomes or family outcomes performance;
              • If the region, program or provider has long-standing low performance or noncompliance based on historical and trend data; and
              • If there are contextual factors that need to be considered when making improvements (e.g., programs' demonstrated ability to correct prior noncompliance, providers' readiness to change practice).
           1. Percentages (e.g., ≥95%, 85-94%, 76-84%, ≤75%);
           2. Number of instances in proportion to the "N" (e.g., 1 out of 5, 1 out of 50); and
           3. Other identified factors (e.g., specific providers, service coordinators).

           Avoiding Potential Pitfalls for Step 3

           • Avoid making assumptions about factors contributing to low performance related to results or compliance without an actual investigation (e.g., identifying the root cause(s) versus attributing the cause to previous experience with a provider or geographic location.) Frequently, personnel shortages or lack of funding are automatically assumed to be the sole contributing factor and other root causes are not identified.
           • Identify realistic expectations of state/local capacity for conducting root cause analysis and provide support, including training local programs/providers in how to determine the root cause.
           • Ensure that those individuals conducting the root cause analysis have the necessary infrastructure supports, including:
             • Adequate resources (e.g., funding, personnel, time);
             • High quality and accurate qualitative and quantitative data; and
             • Skills and training to analyze, drill down, and use data to develop effective improvement activities or corrective actions.

             Step 4: Develop a Plan for Improvement or Correction

             Following collecting data and conducting a root cause analyses, a planning team, which includes the state and/or local program and appropriate stakeholders, develops a written statewide or local program plan that includes strategies for improvement and/or correction. The plan is based on the data analyses and other activities that identified the root causes and the factors that contributed to current performance.

             A written plan is essential for defining infrastructure- and practice-related improvement strategies that address the contributing factors (e.g., providing training to practitioners to address knowledge gaps) identified through the root cause analysis. A high-quality plan is often developed collaboratively between the state and local program(s), sometimes including state-directed local activities. This agreement identifies the benchmarks for improvement or correction, persons responsible for the actions, timelines, and interim and long-term evaluation criteria. The plan tracks progress and keeps everyone on the same page.

             Activities for Step 4

             1. The planning team, which includes appropriate stakeholders:
                1. Reviews written expectations related to improvement or correction (e.g., improvement plan, corrective action plan, directive, finding letter);
                2. Makes data-informed decisions based on the level/extent of the root causes (see: Steps 2 and 3); and
                3. Uses principles of improvement (e.g., improvement science and implementation science).
                1. Level of the system where improvement or resolution of noncompliance needs to occur (e.g., provider level, program level, state level);
                2. Outcomes (i.e., short-term, intermediate and long-term) or benchmarks that will be used to measure progress toward improvement in performance/results and/or achieving resolution of the noncompliance;
                3. Actions or strategies that are doable, meaningful and necessary to address the root causes (e.g., infrastructure and/or practices) to ensure improvement and/or correction and the steps to meet those actions;
                4. People responsible for the actions;
                5. Timelines necessary to carry out the actions. For noncompliance, the timeline for correction is as soon as possible but no later than one year from the date of written notification in accordance with A State Guide on Identifying, Correcting, and Reporting Noncompliance with IDEA Requirements;
                6. Data that will be used to evaluate improvements or progress toward results and/or verify correction;
                7. Tools and methods to collect and analyze the data;
                8. Process that for analyzing data, including when and with whom and how improvement or correction will be verified; and
                9. Resources, including materials, technical assistance, incentives, staff capacity, and other infrastructure supports, that are needed to successfully implement the actions or strategies, and collect and analyze data.

                Avoiding Potential Pitfalls for Step 4

                • Avoid making assumptions about factors contributing to low performance related to results or compliance without an actual investigation (e.g., identifying the root cause(s) versus attributing the cause to previous experience with a provider or geographic location.) Frequently, personnel shortages or lack of funding are automatically assumed to be the sole contributing factor and other root causes are not identified.
                • Identify realistic expectations of state/local capacity for conducting root cause analysis and provide support, including training local programs/providers in how to determine the root cause.
                • Ensure that those individuals conducting the root cause analysis have the necessary infrastructure supports, including:
                  • Adequate resources (e.g., funding, personnel, time);
                  • High quality and accurate qualitative and quantitative data; and
                  • Skills and training to analyze, drill down, and use data to develop effective improvement activities or corrective actions.

                  Step 5: Implement the Plan, Measure Progress and Refine as Needed

                  Some strategies to improve practice and change infrastructure can be implemented quickly and others may require a longer time period. State and/or local programs use evaluation tools and/or methods identified in the improvement plan, to collect, track and summarize data that measure progress over time. These data are periodically reviewed with stakeholders to determine progress in meeting the short-term, intermediate, and long-term outcomes and the effectiveness of the improvement strategies. Adjustments are made to the plan as necessary to continue to improve and sustain progress.

                  Activities for Step 5

                  1. The state and/or local program implement(s) the plan as intended (i.e., implementation fidelity).
                  2. The state and/or local program develop(s) a mechanism for implementers to provide feedback on implementation and progress.
                  3. As necessary, the state provides technical assistance to local programs and providers as they implement their improvement or correction plan.
                  4. According to the timelines in the improvement or corrective action plan, the state and/or local program collect(s) and uses evaluation data to assess the effectiveness of strategies and progress toward outcomes and makes necessary adjustments to infrastructure and practice.
                  5. The state and/or local program adjust(s) or add(s) strategies based on review of progress and ensure(s) the revisions address the root causes.
                  6. As progress occurs, state and/or local programs put in place mechanisms to ensure improvements to practice and infrastructure are sustained.
                  7. For noncompliance, the state verifies correction of noncompliance as summarized in A State Guide on Identifying, Correcting, and Reporting Noncompliance with IDEA Requirements.
                  8. Each state must use the annual local public reporting and determination processes as leverage to ensure compliance and data quality.
                  9. States are also strongly encouraged to use the annual local public reporting and determinations process to incentivize improvement in results and provide feedback to stakeholders on progress made by local programs.
                  10. The state should use evaluation data to inform the selection of appropriate incentives or sanctions to ensure improvement and/or correction.

                  Avoiding Potential Pitfalls for Step 5

                  • Involve stakeholders throughout implementation and evaluation of the plan.
                  • Develop strategies when there are changes to state infrastructure, priorities, or administration (e.g., change in leadership, new lead agency programmatic focus, natural disasters) that capitalize on opportunities and minimize potential negative impacts of these changes.
                  • Ensure the revisions address the root causes and contributing factors when adjusting or adding strategies based on review of progress.
                  • Ensure there are frequent check-in opportunities with implementers to determine if strategies are implemented as planned (i.e., implementation fidelity).
                  • Provide guidance to implementers if the strategies are unable to be implemented as planned or if progress is not being achieved.

                  Step 6: Sustain Progress Towards Improvement or Compliance

                  Improvement activities should be directed at sustaining progress that has been made in reaching desired outcomes and results. Sustainability means maintaining progress or correction at a certain rate or level. It requires supporting a system or program over time with sufficient financial and human resources to achieve child and family outcomes and maintain compliance. Effective planning includes consideration of sustainability given changes that naturally occur in state systems (e.g., staff turnover, leadership changes, new state priorities, fluctuating financial resources).

                  Activities for Step 6

                  1. With stakeholder input, the state and/or local program determine a plan for sustainability and uses available strategies, including technical assistance, local determination factors, public reporting, incentives and sanctions to ensure progress or correction is sustained.
                  2. The state applies what is being learned through the improvement process to inform policies and procedures and other parts of the system such as accountability, fiscal, governance and/or professional development.
                  3. The state continues to provide, as needed, differentiated technical assistance to address knowledge, skills and attitudes that support practice improvement and compliance with regulatory guidelines.
                  4. The state and/or local program continue to pay careful attention to contributing factors to sustain improvement and compliance and diagnose any lack of progress.
                  5. The state uses results from the ongoing accountability and quality improvement system to guide state strategic plans and priorities, ensuring necessary resources are allocated to sustain progress and correction.

                  Avoiding Potential Pitfalls for Step 6

                  • Maintain a constant cycle of communication with involved stakeholders to address ongoing knowledge and information needs, sustain motivation and enthusiasm for the work, and quickly become aware of any changes that might put improvement or correction in jeopardy.
                  • Imbed procedural changes within the system to ensure that compliance is maintained when program priorities shift to other areas.
                  • Create incentives and celebrate success in program improvement and correction of noncompliance.
                  • Support Quality Assurance staff to expand their skills to include a focus on improving results and identifying and correcting noncompliance.
                  • Use data and information to maintain leadership support to sustain infrastructure improvements and practice changes.
                  • Consider results of all general supervision activities when conducting state strategic planning.

                  The contents of this product were developed under grants from the U.S. Department of Education, #H326P170001 (ECTA Center), #H373Z190002 (DaSy), and #H326R190001 (NCSI). However, those contents do not necessarily represent the policy of the U.S. Department of Education, and you should not assume endorsement by the Federal Government. Project Officers: Julia Martin Eile, Meredith Miceli, Amy Bae, and Perry Williams.

                  

                  • The ECTA Center is a program of the FPG Child Development Institute of the University of North Carolina at Chapel Hill, funded through cooperative agreement number H326P220002 from the Office of Special Education Programs, U.S. Department of Education. Opinions expressed herein do not necessarily represent the Department of Education's position or policy. Project Officer: Julia Martin Eile
                  • © 2024 ECTA Center
                  • Accessibility
                  • Privacy Statement
                  • Subscribe to Email Newsletter

                  The ECTA Center is a program of the FPG Child Development Institute of the University of North Carolina at Chapel Hill, funded through cooperative agreement numbers H326P170001 and H326P220002 from the Office of Special Education Programs, U.S. Department of Education. Opinions expressed herein do not necessarily represent the Department of Education's position or policy. Project Officer: Julia Martin Eile

                  © 2012-2024 ECTA Center