Code.org is a non-profit dedicated to expanding access to computer science education, and increasing participation by women and underrepresented students of color. Code.org’s work is built on the belief that computer science should be part of the core curriculum in education, alongside other science, technology, engineering, and mathematics (STEM) courses. As part of its strategy, Code.org is piloting a four-year program that is focused on bringing computer science to K-12 schools nationwide. Code.org hired Outlier Research & Evaluation at the University of Chicago to evaluate their efforts.

As external evaluators, Outlier worked closely with Code.org staff to determine their evaluation priorities, given the developmental stage of Code.org’s offerings. Code.org staff prioritized understanding professional development (PD) implementation; teacher satisfaction with PD; program (course) implementation; and teacher attitudinal outcomes, as a first step in a larger evaluation effort. The evaluation thus focused on supporting program improvements; documenting the extent to which Code.org was reaching its desired short-term outcomes; and data-informed recommendations for realizing short and long-term goals. Specifically, the goals of the evaluation were to:

  1. Inform Code.org program leaders about program implementation and teacher satisfaction, both for professional development and the courses themselves;
  2. Inform Code.org leaders about the extent to which program outcomes were met; and
  3. Provide evidence-informed recommendations that would help Code.org improve its programs and professional development.

Outlier evaluators collected a wide range of data from Code.org partner sites. Code.org partner sites are districts or schools that have entered into an agreement with Code.org to work together to provide CS training for teachers and administrators, and to offer any range of CS programs to students (K5, middle school, or high school). Elementary and high school teachers; professional development facilitators; high school counselors; school leaders; and district leaders were asked to participate in the evaluation. Outlier employed a combination of qualitative and quantitative data collection methods, including interviews, surveys, focus groups, and observations.

Note that each study drew from a different sample in the population; for example, not every teacher responded to every survey and/or interview. The purpose and focus of each study was also different; see the introduction portion of each study for more information about the respondents; purpose; and focus.

Consistent with Code.org’s commitment to transparency, we are sharing this summary of findings from the 2014-15 evaluation. It is Outlier’s intention that these findings be used as a resource for the CS education field.