As an initial step for describing further evaluation goals, Outlier evaluators worked with BSI to clearly delineate the explicit and implicit “critical components” of SSY and the relationship between these components and program goals. This process resulted in a rich and specific description of the SSY model and the production of a program logic model that informed the collection and interpretation of SSY implementation data.

See Models to learn more.

Outlier evaluators administered questionnaires to all BPS science teachers and other teachers who participated in SSY trainings. Many BPS schools use a science specialist model in which science for most or all grades is taught by a single science teacher. Other schools use a more traditional model, where classroom teachers teach all subjects, including science. The teacher questionnaire focused on understanding enactment of science instruction behaviors. Evaluators analyzed data to describe the proportions of teachers that not only engage in outdoor instruction, but also use various SSY strategies and activities. Additionally, on-site qualitative data collection (interviews, focus groups, and observations) occurred in the spring of 2013 to further understand the details and depth to which teachers are implementing SSY strategies.

See Teaching Outdoors and Science Instruction to learn more.

In addition to measuring teachers’ enactment of science activities and behaviors, Outlier evaluators measured external factors that may influence implementation. These included teacher experience, participation in non-SSY science professional development, and supports and barriers that may exist at schools or within classrooms (e.g. perceptions of time availability, student behavior, principal support for outdoor instruction, etc.). Outlier evaluators examined these factors to determine the extent to which they may be associated with teacher use of outdoor instruction and the implementation of science activities.

See Challenges and Supports to learn more.

A key part of understanding the mechanisms of SSY was to examine relationships between outdoor instruction and science implementation, between teacher perceptions of their science knowledge and their science instructional behaviors, and other teacher attitudes about science teaching and learning. Outlier evaluators collected data on attitudinal outcomes in the questionnaire administered to all teachers. Evaluators also administered a questionnaire to school leaders to understand how SSY and BSI may be related to other school outcomes. On-site qualitative data collection in spring 2013 supported this evaluation question as well. Specifically, in interviews and focus groups, evaluators asked teachers to describe their perceptions of the ways that SSY has had an impact on their attitudes, knowledge, and instruction.

See SSY Training and Teacher Outcomes to learn more.

The evaluation also sought to understand how outdoor instruction may be related to student outcomes, such as interest in and engagement with science. The teacher questionnaire asked teachers to describe their perceptions of student enactment of science behaviors, science talk, and attitudes about themselves and science as well as science learning. Teacher perceptions about student outcomes were also measured through qualitative data collection, including focus groups and interviews. Outlier evaluators gathered perceptions from students in grades 3−5 through a questionnaire and student focus groups, and observed classrooms for examples of student engagement. Finally, evaluators conducted a brief school-level analysis of student achievement data.

See Outdoor Science Education and Student Outcomes to learn more.

A final piece of the evaluation sought to understand what has contributed to the present status of SSY, as well as what the desired lasting impacts and sustainability of SSY should look like for the program moving forward. With a greater understanding of how SSY implementation is related to student and teacher outcomes and associated factors, Outlier evaluators have further described which components and factors will be necessary for BPS to have both the capacity and will that is needed for the program to endure.

See For Districts to learn more.

The findings presented here focus on those respondents that identified as teachers of science. Very few self-identified non-science teachers participated in the questionnaire. After data cleaning, only nine non-science teachers remained in the dataset, thus, they were excluded from most remaining analyses. Of the 97 respondents in the science teacher sample, 47% (n=46) indicated that they were science specialists. Fifty-three percent (n=51) indicated that they were their students’ primary science teachers, but were not specialists. These were classroom teachers or ELL/special education specialists.

Of the science teacher sample, 59% (n=57) indicated that they had taken at least one Science in the Schoolyard (SSY) course. Forty-one percent (n=40) indicated that they had not taken any SSY courses. Only four (36%) of the non-science teacher sample indicated that they had taken any SSY courses. Thirty-six teachers (33%) had an outdoor classroom at their school (verified by BSI), and 72 (67%) did not have an outdoor classroom.

A total of 966 third through fifth grade students at 7 schools participated in the student questionnaire. Of these, 770 had usable data throughout. Three schools had high response rates, suggesting that the majority of students across grades completed the questionnaire, whereas the remaining four schools had lower response rates. About half (n=406, 52%) of the student respondents were at schools with an outdoor classroom and a science specialist trained by SSY, and about half of the students (n=363, 47%) were at schools without an OC and without a science specialist that was trained by SSY.

Students were asked to report how often they go outdoors for science, which was used as a predictor for several analyses. A total of 26% (n=201) of student respondents reported that they never go outdoors for science. Sixty-seven percent (n=514) indicated that they sometimes go outdoors for science, and 7% (n=56) indicated that they often go outdoors for science. Due to the small number of students who indicated that they go outdoors for science often, and because these students were evenly spread across schools, the “sometimes” and “often" categories were combined for analysis.

Outlier evaluators visited eight schools in spring of 2013. Of these, five had a science specialist who was SSY-trained. Two of the remaining schools had a specialist who was not SSY-trained, and the other school had no science specialist. Other teachers in two of these schools had been SSY-trained; only one school had no SSY-trained teachers. Six schools had outdoor classrooms and two had a BSI-renovated schoolyard.

Evaluators conducted in-depth interviews with 11 teachers, including eight science specialists, one classroom teacher who teaches science to her students, one non-science teacher, and one former science specialist. In addition, five focus groups were conducted with other teachers at the schools. A total of 15 teachers participated in focus groups. Student focus groups were conducted at eight schools, with students ranging from grades 3 through 5. A total of 44 students participated in student focus groups. Six principals also participated in interviews.

Evaluators observed 15 science lessons across the eight schools. Of these, seven lessons included an outdoor component.