Using Continuous Improvement to Solve For Learning Loss
COVID-19 has fundamentally changed what is expected of schools and how districts can organize themselves to support teachers, students and families. The COVID Recovery Cadre was created to support a successful return for students into classrooms in SY 20-21 with a specific focus on supporting vulnerable students. The cadre together selected the outcome they hoped to achieve: supporting the eighth to ninth grade transition year and student success in pre-Algebra and Algebra 1.
Supported by Impact Florida and guided by experts from Proving Ground at Harvard University, district teams developed a specific, actionable problem statement that highlighted a core challenge for their students in Pre-Algebra or Algebra. During a virtual convening in August 2020, teams identified root causes and explored potential interventions to address those causes. The cadre was surprised at just how similar their problems-of-practice and root causes were, given the differences in the size, location, make-up of the districts.
The districts participating in the this Districts for Impact Learning Cadre included:
Explore This Cadre's Work
Math Interventions Tested
The districts prioritized a wide variety of possible interventions including improving teacher coaching, strengthening supports for students, and providing directed tutoring support to struggling students.
- Collier County Public Schools provided content-specific coaching to algebra teachers through the MQI Coaching project. Learn more about this intervention from teacher Kayla Fulgham in her Insights post, Virtual Coaching: How Taking Risks Helped My Students Succeed.
- The School Board of Highlands County opted to provide direct student tutoring services to 8th-grade pre-Algebra and 9th-grade Algebra students using Spring Math.
- The School District of Palm Beach County focused on strengthening academic success in Algebra through delivering a research-based PERTS two-part web-based module designed to help students increase motivation.
- Pasco County Schools followed a similar path to Palm Beach, utilizing PERTS modules and also adding teacher supports centered around academic goal setting and self-efficacy.
The interventions were implemented through SY 2020-2021.
Summary of Results
Collier County - MQI Coaching for Teachers
In Collier County, for teachers was a transformative experience for the 12 math teachers who participated in a pilot of the video coaching program. Teachers appreciated that the coach provided a safe, non-evaluative space to reflect on their practice and improve. Over the course of the video coaching, teachers developed a growth mindset: instead of focusing on their weaknesses, they learned to recognize and build upon their strengths. Most importantly, teachers shifted their practice in the classroom, allowing more wait time, asking students to explain their thinking more often, and allowing students more time to discuss math. As a result, coaches observed that 80 percent of teachers improved the cognitive demand of their instruction in the classroom – putting more of the thinking work on students instead of the teacher. Finally, more than 80 percent of the teachers who participated said they would do the coaching again.
Highlands County - Spring Math Tutoring
In Highlands County, supporting students with peer tutoring through Spring Math resulted in an improvement of about ⅓ a year of additional learning for students who completed the tutoring as compared to those who did not. The randomized controlled trial showed an improvement of .135 standard deviation on end-of-year exams (the Florida Standards Assessment and Algebra 1 end-of-course exams). Increases were also apparent on the district’s formative assessment tool, iReady, particularly within their learning of numbers of operations and algebra strands.
Palm Beach and Pasco Counties - PERTS Modules
Palm Beach and Pasco counties decided to implement PERTS Online Modules for 9th Graders, which consisted of two free, half-hour online modules to increase student motivation, completed at home and in school. National studies showed that even this lighter-lift intervention could have modest, positive results for students. The results in Palm Beach and Pasco showed a tiny, less than .01 gain in GPA during Quarter 3 of SY 2020-2021, and the improvements didn’t last in the fourth quarter of the school year. These results weren’t surprising given significant implementation challenges. In Palm Beach, only 33 percent of targeted students actually completed both modules and there was only 50 percent of completion in Pasco. More worrying, students of color, low-income students, and low-performing students were less likely than their peers to have taken the modules – perhaps due to student absences, implementation issues in certain schools, or the disproportionate burden of COVID on those families.
Key Takeaways
Impact Florida shared key takeaways from this Learning Cadre on the "What We're Learning" blog of the Bill & Melinda Gates Foundation, which we also published on our blog, INSIGHTS. Key takeaways were:
- The interventions indeed showed significant promise to improve student learning in math at this critical juncture in their K-12 education, even in the midst of a pandemic.
- The impact on students could be significantly better if districts worked through implementation problems.
- A continuous improvement approach to implementation is powerful and essential.
Free Resource: Workouts for Continuous Improvement
Impact Florida developed a free resource to help education teams strengthen their improvement practices.