Strengthening the quality of Hawaii’s summer learning programs

Project Snapshot

In the summer of 2019, the Hawaii Community Foundation and the Harold K.L. Castle Foundation engaged staff from SCRI to identify opportunities to elevate quality and outcomes across summer learning programs in Hawaii.

Using BellXcel’s Quality Improvement Framework, staff from SCRI identified opportunities to both strengthen individual program practices and elevate the overall quality of summer programs statewide.

The Backstory

In fall 2018, Kamehameha Schools, Hawaii Community Foundation, and BellXcel, SCRI’s parent organization, began exploring the feasibility of scaling summer learning across the Hawaiian Islands. At the onset of the initiative, the partners met with and engaged the Superintendent’s Office and the Community Engagement Office of the Hawaii Department of Education, as well as the Hawaii Afterschool Alliance, to form a Steering Committee to guide the development of a strategy.

Although there is no singular source for information about the current supply and demand for summer learning in Hawaii, there is evidence suggesting that demand for summer programming exceeds the supply. According to the Afterschool Alliance’s America After 3 p.m. report, 29% of families in Hawaii reported having at least one child enrolled in a summer learning program, and 60% say they want their children enrolled in a summer learning program.

Steering Committee members weren’t sure how many summer opportunities were available to families across the islands, but they suspected an overall shortage and multiple geographic gaps in availability. They also recognized the lack of an island-wide focus or framework for quality and learning like the state had for afterschool. They shared a desire to standardize summer program quality and build a unified vision for addressing summer learning loss.

As a result of these conversations, SCRI was hired to assess the current landscape of Hawaii’s summer learning providers; facilitate a strategic planning process; conduct a quality assessment of a subset of summer programs; and support ongoing conversations among local partners to implement recommendations. This case study focuses on our approach to quality assessment and improvement, specifically.

Our Approach

SCRI conducted a statewide provider survey to learn more about the locations and characteristics of summer programs. Based on the findings, Steering Committee members identified and invited five programs to participate in the Quality Improvement Project. Programs were intentionally varied in their goals, structure, and numbers of students served, but they all served middle school youth. All programs demonstrated a strong commitment to quality and were well-positioned to adopt research-based practices that could be scaled within their current structure and program goals.

SCRI implemented BellXcel’s Quality Improvement Framework at each site, including:

Initial call to learn about the program
Review of documents, like lesson plans, schedules, training materials, and handbooks
Site visits to observe arrival, activities, transitions, and dismissal
On-site interviews with program leaders, staff, and youth
Findings shared via Quality Improvement Dashboard for each site
Final report offering actionable recommendations across six critical domains at the program- and systems-level: Program Administration, Culture & Climate, Curriculum & Instruction, Family Engagement, Assessment & Evaluation, and Recruitment & Attendance.

Summer learning programs were able to use the findings from the quality improvement report to gain insight into the program’s key strengths and areas for improvement. It provided a rare moment for program staff to intentionally reflect on their work; a moment I wish was commonplace for every program.
Justina Acevedo-Cross, Hawaii Community Foundation

Key Findings – Program Level

Following site visits, SCRI provided feedback directly to programs on areas of strength and opportunities for programs to better align to research-based practices within the summer space.

Strengths	Opportunities for Growth
Across all sites, SCRI found excellent and consistent examples of highly dedicated and passionate staff, committed to providing an excellent summer experience for all youth. The safety and engagement of youth were at the forefront of activities. Program staff consistently implemented best practices in enrichment instruction, regardless of their experience level, including valuing youth voice and choice, prompting higher-order questioning, and facilitating collaboration. Programs also represented a holistic approach to implementation, incorporating physical activities and social-emotional learning as key components while also infusing Hawaiian culture-based education.	Across sites, SCRI found an opportunity to extend the best practices found in enrichment programming to strengthen academic offerings. By applying enrichment practices like collaboration and youth voice and choice, programs could shift focus from teacher-directed or self-directed learning more common to their academic programming to more collaborative approaches. SCRI also found that providing instructors with access to youth academic data would make it easier for them to deliberately target the academic needs of youth. Finally, we found a clear opportunity to tie academic goals to evaluation metrics and methods at both the student and program levels to better track progress. Such efforts would lead to improved summer program evaluation, observation, and reporting.

Strengths

Opportunities for Growth

Across all sites, SCRI found excellent and consistent examples of highly dedicated and passionate staff, committed to providing an excellent summer experience for all youth.
The safety and engagement of youth were at the forefront of activities.
Program staff consistently implemented best practices in enrichment instruction, regardless of their experience level, including valuing youth voice and choice, prompting higher-order questioning, and facilitating collaboration.
Programs also represented a holistic approach to implementation, incorporating physical activities and social-emotional learning as key components while also infusing Hawaiian culture-based education.

Across sites, SCRI found an opportunity to extend the best practices found in enrichment programming to strengthen academic offerings.
By applying enrichment practices like collaboration and youth voice and choice, programs could shift focus from teacher-directed or self-directed learning more common to their academic programming to more collaborative approaches.
SCRI also found that providing instructors with access to youth academic data would make it easier for them to deliberately target the academic needs of youth.
Finally, we found a clear opportunity to tie academic goals to evaluation metrics and methods at both the student and program levels to better track progress. Such efforts would lead to improved summer program evaluation, observation, and reporting.

Recommendations and Next Steps: Statewide

At the completion of the Quality Improvement Project, SCRI made recommendations to the Steering Committee to enhance the quality of summer learning while supporting scale and sustainability across programs in the Hawaiian Islands.

1. Mirror the successful systems for afterschool programming in the summer learning space.

Many of the programs observed were a continuation of existing after-school programs running within the school or community. However, less structure and uniformity existed in summer. Program leader interviews revealed that the clear goals, activities, and coordinated systems implemented at a programmatic level during afterschool fell to individual site leaders in summer programming. Moreover, best practices didn’t always transfer down to the site level due to the quick turnaround between afterschool and summer. SCRI recommended a continuation of successful afterschool systems for staff professional development, program activities, family engagement strategies, and data tracking.

2. Develop evaluation metrics, tools, and outcomes reporting for summer learning expectations.

The summer space offers a unique opportunity for student learning, career exploration, and social emotional development and therefore warrants its own specific guidelines, tools, and metrics for program quality and impact. The Hawaiian After School Guidelines offer a robust foundation from which summer-specific guidelines can be created. SCRI recommended the Steering Committee investigate which of these guidelines should be universal to all out-of-school-time programming and which guidelines and metrics should be created specifically for summer learning. We also recommended determining a process for assessing and evaluating quality to support individual programs in developing goals and aligning goals to desired outcomes.

3. Create and align summer-specific professional development opportunities for program staff.

SCRI recommended making available explicit summer professional development to support quality and sustainability across summer programs. Even in cases where staff are retained from an afterschool program, professional development should clarify the conditions and expectations that are unique to summer. While afterschool programs complement the school day, summer programs often stand alone in offering academics, enrichment, physical activity, and nutrition, requiring enhanced skills and competencies among staff.

4. Determine goals related to academic instruction in the summer and align these goals to curriculum, planning, and desired outcomes

Many programs expressed a desire to strengthen their academic component either in terms of instruction, curriculum, and/or evaluation of skill growth and development. However, deep and explicit academic instruction may not be appropriate for all programs. SCRI recommended the Steering Committee first identify how a program’s expertise and community of learners should inform its academic goals, curriculum, and evaluation. In the varied summer space, a spectrum of academic goals, ranging from lighter-touch to explicit instruction, is appropriate. Ideally, summer learning curriculum is distinctive from the school-day and provides students with opportunities to collaborate and problem-solve with peers through engaging and scaffolded activities.

5. Consider opportunities for programs to combine/partner

Two of the programs observed–one with a heavier emphasis on culture, and one with a greater emphasis on academics—recognized that youth were floating between their programs. In partnering to combine their programs, they were able to serve more youth consistently and in a more holistic way.

SCRI recommended investigating opportunities for other programs to take a similar approach, especially in instances of half-day programs and programs with specific areas of expertise. SCRI recommended exploring opportunities to combine programs offering recovery credits from the school year with other summer programs to create a more comprehensive learning experience. Such partnerships could better serve young people by addressing academic skills as well as social emotional skills, career exposure, and cultural exploration.

The steering committee continues to meet and explore ways to expand summer opportunities, strengthen academic learning, and ensure quality experiences.

Resources