Studying Implementation

The Outlier approach to measuring status of implementation recognizes that no two teachers’ curriculum enactments will look exactly the same because teachers make adaptations based in their local needs, contexts, and conditions (factors). In the BASICS study, we are primarily interested in describing the range of ways that ECS is used during classroom instruction, and identifying the key supports or barriers that influence use and endurance. The study is not an evaluation of ECS; rather our research seeks to systematically document and describe implementation status and a deeper understanding of the supports for and barriers to implementation. Instead of asking the question, “Is the program being enacted as intended?” data analysis for BASICS focuses on “What is being enacted, and what factors affect that enactment?”

Why it Matters

Implementing and expanding opportunities for CS education is an issue of educational equity, particularly for students traditionally underrepresented in CS. Those already working to make K12 CS happen know that bringing change to school systems is complex. As states and school districts explore the role of CS in K12 education, understanding how ECS came to be implemented in a range of sites can have great impact on the implementation and endurance of introductory CS courses today and in the future.

Our Philosophy

The fundamental goal of all Outlier projects is to provide schools, districts, and policy makers with useful, practical information to continually improve education efforts. In the BASICS Study, we collaborate with NSF CS10K project leaders, CS education advisors, and school and district practitioners to produce findings and share relevant and practical information to help grow and sustain CS education. Our work contributes to the growing portfolio of CS education efforts. We encourage feedback on our project throughout the research study.

To see more of Outlier’s computer science education work, visit our Projects page.

Study Goals

  • Inform NSF CS10K leaders and other CS educators about the supports for and barriers to wide-scale high school Cs education and provide strategies for addressing them;
  • Provide tools for measuring introductory high school CS program implementation and instruments for measuring the factors that affect implementation; and
  • Create resources from research findings and recommendations that will be useful to the NSF CS10K community and the broader CS education field.
Research Questions
Who is Involved?

BASICS Study participants work in schools using ECS in districts across the US. Current and past participants include those in Chicago, the District of Columbia, Los Angeles, Broward County, and New York City.

New York

Broward County


The BASICS Study is a mixed-methods research study. We use a range of both qualitative and quantitative strategies to answer our primary research questions and accomplish our project goals. Our research design collects data via teacher and student questionnaires; ECS developer, school district personnel, school leader, and teacher interviews; student focus groups; informal observations; and document reviews.

This project builds on Outlier's previous NSF-funded work rigorously studying implementation of educational innovations (new practices and programs) and the factors (contexts and conditions) that affect innovation implementation and sustainability. Earlier projects yielded a suite of instruments for measuring implementation of science and mathematics programs and a conceptual framework and accompanying instruments for identifying and measuring the factors that support and inhibit implementation of school-based innovations. The BASICS Study began with these instruments and customized them for use with a CS-specific innovation (the ECS program). The project also stems from a previous CS-focused study (OS4CS) that established a more comprehensive understanding of our nation’s current high school CS teaching and learning environment.

Although implementation measures can be used to examine program impacts, for the purposes of this research, we are interested in looking at the relationships between supports and barriers to implementation and implementation itself, not the quality, efficacy or impact of ECS on student outcomes in CS courses or pursuance of CS careers.

Research Team
Project Advisors
  • Owen Astrachan, Professor of the Practice of Computer Science, Duke University, and PI of CS Principles
  • Lissa Clayborn, Deputy Executive Director/Chief Operating Officer, Computer Science Teachers Association
  • Baker Franke, former HS Computer Science Teacher, now K-12 Curriculum Development Manager,
  • Anu Gokhale, Professor of Computer SystemsTechnology, Illinois State University, and PI of Teacher Education in CS (TECS)
  • Mark Guzdial, Professor in the College of Computing at the Georgia Institute of Technology, and PI of Georgia Computes!
  • Helen Hu, Professor of Computer Science, Westminster College, and PI of the Utah ECS initiative
  • Chris Stephenson, Former Executive Director of the Computer Science Teachers Association, now Computer Science Education Program Lead, Google
  • Brenda Wilkerson, Senior Manager, Computer Science & Information Technology Ed.




What are supports and barriers to teaching introductory CS?

Computer Science teachers in Chicago and Washington, DC completed a questionnaire in Spring 2014 that, among other things, asked them to identify the three biggest supports for and barriers to their computer science classes. All of the teachers were using Exploring Computer Science (ECS) instructional materials.


What factors affect student engagement in CS?

ECS students in Chicago and Washington, DC completed a questionnaire in Spring 2014 focused on student engagement, teacher instruction, and student attitudes and characteristics.





How do teachers describe inquiry and equity in their computer science classroom?

In Spring 2014, 24 Computer Science teachers in Chicago and Washington, DC completed a questionnaire that, among other things, asked them to describe inquiry and equity in their classroom. All of the teachers were using Exploring Computer Science (ECS) instructional materials.