Research Projects

AP CSP

Bringing AP CSP to Students with Learning Differences

National Science Foundation
2015 – 2018

Bringing AP CSP to Students with Learning Differences National Science Foundation
2015 – 2018

This 2-year study, conducted in collaboration with the Wolcott School (an independent college preparatory high school in Chicago for students with learning differences), aims to expand participation in computer science by ensuring that students with“learning differences” (students with learning disabilities and/or an Attention Deficit/Hyperactivity Disorder(ADHD)) can successfully participate in the AP Computer Science Principles (CSP) course.

In support of this goal, the interdisciplinary project team will identify the challenges students with learning differences face as they engage with CS instruction and content; identify possible solutions to address those challenges, and; test specific adaptations and accommodations in lessons from two AP CSP curricula: Beauty and Joy of Computing for New York City (BJC4NYC); and Code.org’s CS Principles.

Project Resources

Visit the Research section to learn more.

Leading CS Growth

Leading Computer Science Growth:
A Toolkit for K-12 District and School Leaders and Other Stakeholders

National Science Foundation
2014 - 2015

Leading CS Growth

(LeadCS.org)
Leading Computer Science Growth: A Toolkit for K-12 District and School Leaders Other Stakeholders2014 - 2015

This project addresses the need of K-12 education leaders for resources to support implementation of computer science education initiatives. Our team has created research based-tools and recommendations housed at LeadCS.org, a website for K-12 education leaders in schools and districts and their partners looking to begin or improve a computer science education initiative.

Some LeadCS.org tools provide summaries and syntheses of current information, data, and projects. Other tools are “voices of experience” with recommendations coming from leaders who have already navigated the change process. These tools provide key advice from leader reflections on the most critical supports, barriers, and lessons learned in the process of implementing a computer science program. Still others provide guidance and frameworks for leaders ready to take actionable steps toward bringing computer science to their students.

Project Resources

Visit the Research section to learn more.

BASICS

Barriers and Supports to Implementing Computer Science (BASICS)

National Science Foundation
2013-2016

BASICS

Barriers and Supports to Implementing Computer Science (BASICS) National Science Foundation
2013-2016

The Barriers and Supports to Implementing Computer Science (BASICS) Study examines how an introductory computer science program, Exploring Computer Science (ECS) is implemented in school districts with a focus on identifying the key supports for and barriers to that implementation and endurance.

The goals of the study are to:

  • Inform NSF CS10K leaders and other CS educators about the supports for and barriers to wide-scale high school Cs education and provide strategies for addressing them;
  • Provide tools for measuring introductory high school CS program implementation and instruments for measuring the factors that affect implementation; and
  • Create resources from research findings and recommendations that will be useful to the NSF CS10K community and the broader education field.

Project Resources

Visit the Research section to learn more.

ACM

Building an Operating System for Computer Science Education (OS4CS)

ACM, Google
2012-2013

ACM

Building an Operating System for Computer Science Education (OS4CS) ACM, Google
2012-2013

This study was designed as a collaborative research and communication effort to establish a more comprehensive understanding of our nation’s current high school computer science (CS) teaching population, the support they have, and contexts in which they teach. Outlier, as the research and evaluation arm of the University of Chicago’s Center for Elementary Mathematics and Science Education (CEMSE), and the University’s Urban Education Institute (UEI) worked with a partnership established by the Association for Computing Machinery (ACM) that included the National Science Foundation (NSF), Google, the Computer Science Teachers Association (CSTA), Microsoft, and the National Center for Women and Information Technology (NCWIT) to provide a wide range of information and guidance to inform and shape CS education efforts.

Project Resources

Visit the Research section to learn more.

Factors that affect implementation

Factors that Affect Implementation, Spread, and Sustainability:
An Implementation Study of Everyday Mathematics

National Science Foundation
2011 – 2015

Factors that Affect Implementation, Spread, and Sustainability: An Implementation Study of Everyday Mathematics National Science Foundation
2011 – 2015

This study is investigating the implementation of the Everyday Mathematics elementary mathematics program and the factors that affect its implementation, spread and sustainability in educational settings. In this study, we use the instruments developed in earlier work to collect data on the specific EM components that are present in the classrooms of several school districts, use that data to understand the status of EM implementation at the classroom and school levels, and investigate if and how specific components of EM vary according to maturity of implementation in each of the sites and as a result of the influence of the implementation factors. Using a components approach to measuring implementation will uncover variations in patterns of implementation that we refer to as implementation “types” and enable us to examine variations in use within and across sites as well as relationships between types of implementation and student outcomes.

Project Resources

Visit the Research section to learn more.

Rigourous Measures of Implementation

Rigorous measures of implementation:
A Suite of Tools for Evaluating STEM Instructional Materials Use

National Science Foundation
2011-2014

Rigorous Measures of Implementation: A Suite of Tools for Evaluating STEM Instructional Materials Use National Science Foundation
2011-2014

This study is funded through NSF’s Promoting Research and Innovation in Methodologies for Evaluation (PRIME) program and as such, is focused on using implementation measures in the context of program evaluation. The study’s primary aim is to explore the use of our implementation frameworks and adaptations of our instruments for measuring use of elementary science and mathematics instructional materials to help school and district leaders understand more about their program and its relationships to student outcomes. Specifically, we are exploring the ways implementation data, innovative methodologies for data analysis and visualization, and communication methods in schools and districts are most helpful. This project emphasizes the development of an implementation evaluation process that includes regular engagement with school personnel and using implementation data to inform adoption, implementation, and professional development decisions.

Project Resources

Visit the Research section to learn more.

Validation of instruments

Validation of Instruments Measuring Implementation of Reform-Based Science and Mathematics Instructional Materials

Department of Education
2011-2014

Validation of Instruments Measuring Implementation of Reform-Based Science and Mathematics Instructional Materials Department of Education
2011-2014

Outlier Research & Evaluation received support from the Institute of Education Sciences to validate three teacher-level instruments for measuring innovation implementation (Teacher Questionnaire, Teacher Log, Classroom Observation Protocol) and to develop and validate a student-level questionnaire focused on student-reported engagement in mathematics and science instruction. Data were collected from over 400 K-5 teachers and 5,000 students in four school districts across three states (Colorado, Connecticut, and Illinois). Analysis will include tests for construct validity and reliability, tests of measurement invariance between content domains, grades, and districts, and a test of inter-rater reliability for the observation protocol. Convergent validity will be examined for instruments administered concurrently, comparing Classroom Observation to Teacher Log, and comparing Teacher Questionnaire to Student Questionnaire. Predictive validity will be tested by examining relationships between patterns of curriculum use and student outcomes.

Project Resources

Visit the Research section to learn more.

FOI

Applied Research on Science Materials Implementation:
Bringing Measurement of Fidelity of Implementation (FOI) to Scale

National Science Foundation
2007-2010

FOI

Applied Research on Science Materials Implementation: Bringing Measurement of Fidelity of Implementation (FOI) to Scale National Science Foundation
2007-2010

This project created the foundation for all of Outlier's implementation measurement work: the implementation conceptual framework. This framework grew from this project that was focused on developing instruments to measure the use of science and mathematics instructional materials in schools and classrooms. Its three phases were (1) the development of a conceptual framework to study FOI, (2) the development of a suite of instruments for measuring FOI, and (3) the exploration of types of materials use that are associated with improved student outcomes. The first phase of this project resulted in Outlier’s conceptual framework for measuring innovation use. The second phase led to the development of instruments that measure use of science and mathematics instructional materials. Finally, the third phase was a starting point for the study of Everyday Mathematics currently underway.

Project Resources

Visit the Research section to learn more.

Accumulating Knowledge

Accumulating Knowledge on Scaling and Sustaining Reform:
A Foundation for Future Research

National Science Foundation
2007-2009

Accumulating Knowledge on Scaling and Sustaining Reform: A Foundation for Future Research National Science Foundation
2007-2009

This study sought to provide a conceptually sound foundation for accumulating knowledge about scaling and sustainability of innovations in education. The first strand of project work entailed a review of literature in education, health, marketing, business and economics. The initial search led to an analysis of 30,828 abstracts and 572 reports. This analysis resulted in a conceptual organizer of the factors that contribute to and inhibit implementation, spread and sustainability of reform in education. An interdisciplinary conference was held to discuss and review his work in September 2009.

Project Resources

Visit the Research section to learn more.

Leading CS Growth

STEM Indicators:
Identifying and Measuring STEM Schools and Programs

National Science Foundation
2014-2015

STEM Indicators: Identifying and Measuring STEM Schools and ProgramsNational Science Foundation
2014-2015

This project addresses the first of the 14 indicators identified in the National Research Council’s “Monitoring Progress Towards Successful K-12 Education: A Nation Advancing”: “Number of, and enrollment in, different types of STEM schools in each district.” The Outlier project team, in partnership with other leading STEM school researchers, is developing a “taxonomy” that identifies and describes types of STEM schools and programs. The tool will provide the field with much-needed clarity and shared understanding of the landscape of STEM education, and will lay the groundwork for the development of measures to count and describe the status of the nation’s STEM schools and programs.

Project Resources

Visit the Research section to learn more.

CPS Logo

Chicago Public Schools STEM Standards for Success (SSS)

2014-2015

CPS Logo

Chicago Public Schools STEM Standards for Success (SSS) 2014-2015

Outlier has developed a close collaborative partnership with the Office of College and Career Success at Chicago Public Schools. The SSS project seeks to utilize CPS’ mission and vision for STEM education to develop concrete, measurable benchmarks for STEM schools. These benchmarks will be incorporated into a user-friendly rubric for schools across the district. The SSS project will also develop measures of implementation for the identified benchmarks. These measures will facilitate self-improvement and targeted development for CPS STEM schools.

Project Resources

Visit the Research section to learn more.

S3

The STEM School Study (S3)

National Science Foundation
2012-2016

S3

The STEM School Study (S3) National Science Foundation
2012-2016

This study comprehensively describes and measures models of 20 inclusive STEM high schools in seven states (California, New York, Ohio, Texas, Washington, Tennessee, and North Carolina), measures the factors that affect their implementation, and examines the relationships between model components and a range of student outcomes. The study contributes to the field and the growing attention to STEM schools by a) describing the elements of inclusive STEM high school models and the ways those elements are operationalized individually and in combination with others; and b) identifying and describing elements of the schools that appear to be related to desired student outcomes. In addition to study findings, this project will develop a clear framework for describing STEM school models and instruments for measuring enactment of those models, identify the factors that affect implementation, and create rich descriptions of STEM school practices.

Project Resources

Visit the Research section to learn more.

Ohio STEM Learning Network

The Ohio STEM Learning Network:
A Study of Factors Affecting Implementation, Spread and Sustainability

National Science Foundation
2010-2014

Ohio STEM Learning Network

The Ohio STEM Learning Network: A Study of Factors Affecting Implementation, Spread and Sustainability National Science Foundation
2010-2014

The Ohio STEM Learning Network (OSLN) is a statewide network-oriented collaborative of partners from preK-12 education, higher education and business and industry in the state of Ohio. This study focused on one portion of OSLN's effort – the Platform Schools Initiative, which was supported by local and state resources and philanthropy including the Battelle Memorial Institute and the Bill & Melinda Gates Foundation. This initiative's goal was to help launch and connect five STEM secondary schools strategically placed in key economic and cultural regions of the state. This study sought to understand the status of implementation at each of the platform schools, the factors that contributed to or inhibited the implementation and spread of these models, and the factors that appeared to influence the sustainability of the schools.

Project Resources

Visit the Research section to learn more.

Evaluation Projects

Tech Corps

TECH CORPS

2013-2014

Tech Corps

TECH CORPS 2013-2014

TECH CORPS is a national non-profit organization dedicated to improving K−12 education at the grassroots level by helping educators effectively use technology in their schools. The evaluation of TECH CORPS examines the status of implementation for two of their programs: Techie Club and Techie Camp. The evaluation examines the TECH CORPS mission and strategies as a whole, as well as specific and unique strategies and experiences between Clubs and Camps.

Project Resources

Visit the Evaluation section to learn more.

The source

The Source

2013

The Source 2013

The Source is a STEM-based alternate reality game initiated by Ci3 and Game Changer Chicago Design Lab. Outlier’s portion of the larger evaluation effort had two strands. The first strand focused on youth attitudes toward STEM (science, technology, engineering and mathematics) and STEM careers. The second strand focused on the youth’s experience with The Source itself and their perspectives on the parts of The Source that impacted them the most.

Project Resources

Visit the Evaluation section to learn more.

Code.org

Code.org

2014-2016

Code.org 2014-2016

Code.org is a non-profit organization dedicated to expanding participation in computer science, especially by women and under represented students of color. Code.org believes that every K-12 student should have the opportunity to learn computer science. Outlier's evaluation of Code.org examines all of their course offerings, K-12, and measures student and teacher attitudes about computer science, as well as program implementation. The second year of evaluation (2015-16) will also explore potential relationships between program implementation, student and teacher attitudes, and student academic outcomes.The evaluation also examines Code.org’s progress towards their overall mission and goals and strategies used to achieve their vision of broadening participation in computer science.

Project Resources

See our online report from Evaluation Year 1 2014-15 here.

Visit the Evaluation section to learn more.

Google RISE Awards

Evaluation of the Google RISE Awards

Google
2015-2016

Evaluation of the Google RISE AwardsGoogle
2015-2016

Google RISE helps nonprofit organizations around the world to promote computer science education and run initiatives that reach K-12/pre-university girls, underrepresented minorities, and students facing socioeconomic barriers. The RISE Awards program is committed to helping these organizations to inspire the next generation of computer scientists by raising interest and exposure to the field of CS. In 2014, RISE awards were granted to 42 organizations spanning 19 countries across the world. These organizations collectively reached over 87,000 youth with programs and experiences in computer science. The evaluation of the RISE Awards program being conducted by Outlier aims to collaboratively work with Google RISE leadership to articulate and clearly define their program components and overall strategy, create a typology of past and current RISE Award recipients, and richly describe how and to what extent recipients are using RISE Awards to support, scale, and sustain their programs.

Project Resources

Visit the Evaluation section to learn more.

Google CAPE

Google Computing and Programming Experience (CAPE)

2012

Google CAPE

Google Computing and Programming Experience (CAPE) 2012

Outlier conducted an evaluation of Google CAPE, a multi-week summer program located in Google’s Mountain View and New York City campuses. This study focused on understanding the Google CAPE intended model, and measuring implementation of core model components, as well as student and faculty outcomes. This study was structured under a highly collaborative model, to provide timely feedback to CAPE program staff and faculty.

Project Resources

Visit the Evaluation section to learn more.

Urban Education Teacher Program (UTEP)

Urban Education Teacher Program (UTEP)

2009-2014

Urban Education Teacher Program (UTEP)

Urban Education Teacher Program (UTEP) 2009-2014

The primary mission of the UTEP is to prepare exceptional urban public school teachers who are equipped with the knowledge and skills to instruct middle school and secondary students in Chicago Public Schools in the areas of biology and mathematics. The evaluation of this program includes a range of data collection activities focused on program improvement and achieving outcomes in the areas of recruitment, retention, preparation, and effective teaching.

Project Resources

Visit the Evaluation section to learn more.

IGERT

IGERT

2009-2014

IGERT

IGERT 2009-2014

The Motor Control and Movement IGERT program, funded by the National Science Foundation, is a collaboration between the University of Chicago, Northwestern University, and the Field Museum that brings together graduate students with backgrounds in biology, engineering, and mathematics to develop an integrative understanding of movement. The evaluation focuses on the implementation of the program and the experiences of the graduate student trainees, with the goal of providing information that will guide improvements to the program as it progresses.

Project Resources

Visit the Evaluation section to learn more.

University of Illinois at Chicago Gross Anatomy Lab

University of Illinois at Chicago Gross Anatomy Lab

2013

University of Illinois at Chicago Gross Anatomy Lab

University of Illinois at Chicago Gross Anatomy Lab 2013

The purpose of this project was to implement a different approach to anatomical sciences teaching and learning through a more student-centered learning environment. This approach shifted from the traditional model of using instructor produced video to an approach where anatomical information was captured and described by the students themselves. The evaluation focused on student participation in the lab as well as their attitudes toward the innovative lab structure.

Project Resources

Visit the Evaluation section to learn more.

Boston Schoolyard Initiative

Boston Schoolyard Initiative’s (BSI) Science in the Schoolyard Program

2012- 2013

Boston Schoolyard Initiative

Boston Schoolyard Initiative’s (BSI) Science in the Schoolyard Program 2012- 2013

Outlier conducted a “legacy” evaluation (an evaluation designed to describe the project and participants’ experience at the project’s end in order to benefit others’ learning) of BSI’s Science in the Schoolyard professional development program. The Science in the Schoolyard (SSY) program helps teachers use the urban natural environment in their schoolyard (ranging from asphalt lots to BSI-designed outdoor classrooms) to teach the Boston Public School district science curriculum. Evaluators examined the core components of the BSI SSY model, the implementation of outdoor science instruction across Boston Public Schools, and student and teacher outcomes.

Visit Project Site

Project Resources

Visit the Evaluation section to learn more.

Purple Asparagus

Purple Asparagus Delicious Nutritious Adventures

2012-2013

Purple Asparagus

Purple Asparagus Delicious Nutritious Adventures 2012-2013

The Delicious Nutritious Adventures program is designed to help students and their families learn better eating habits and to make simple, healthy snacks and meals. The focus of the evaluation was to understand student and family participation in the Delicious Nutritious Adventures program activities, and the knowledge, behavioral, and attitudinal outcomes for participating youth and their families.

Project Resources

Visit the Evaluation section to learn more.

Thinking Skills

Developing 21st Century Thinking Skills

2010-2013

Developing 21st Century Thinking Skills 2010-2013

This project was a partnership between the School of Education at Saint Xavier University and Chicago Public Schools (CPS), and the International Renewal Institute, Inc. (iRi). Funded by the Illinois Board of Higher Education (IBHE), the project trained selected K – 2 teachers and instructional leaders in Reuven Feuerstein’s Theory of Structural Cognitive Modifiability and to teach the Feuerstein Instrumental Enrichment, Basic course. Outlier determined the degree to which the project met its objectives for professional development and how the program was used in schools.

Project Resources

Visit the Evaluation section to learn more.

Baxter Science Education Initiative

Baxter Science Education Initiative

2008-2011

Baxter Science Education Initiative 2008-2011

The Biotechnology Center for Excellence located at Lindblom Math and Science Academy hosted professional development sessions for teachers in the biotechnology cohort throughout Chicago Public Schools and housed the biotechnology resource center, an in-house and online collection of tools and curricula for teachers integrating biotechnology into their classrooms. This evaluation examined changes in teacher knowledge, confidence and competency with key biotechnology concepts and tools as a result of participation in professional development. Additionally, Outlier examined student outcomes in relation to enrollment in biotechnology courses, such as confidence using biotechnology tools and enrollment in post-secondary education

Project Resources

Visit the Evaluation section to learn more.

MC2 STEM Hub

MC2 STEM Hub

2010

MC2 STEM Hub 2010

Part of the Ohio STEM Learning Network, the MC2 STEM Hub is a network-based education innovation infrastructure based in Ohio with the goal of amplifying the reach and impact of Science, Technology, Engineering, and Mathematics (STEM) education. Outlier evaluated the extent to which the project activities contributed to MC2’s infrastructure to support spread of innovation and its ability to generate regional impact and provide support to Cleveland’s K-8 STEM School development.

Project Resources

Visit the Evaluation section to learn more.

YRC

Young Researchers’ Collaborative (YRC) at the Lincoln Park Zoo

2009-2014

YRC

Young Researchers’ Collaborative (YRC) at the Lincoln Park Zoo 2009-2014

The YRC is a program designed to assist Chicago Public School educators in making inquiry-based science a key component of their classroom instruction and to teach students research skills consistent with the National Science Education Standards. This evaluation is built around a set of evaluation questions derived from the project goals and includes elements of both process evaluation and progress evaluation in addition to summative evaluation.

Project Resources

Visit the Evaluation section to learn more.

Zoo Explorers

Zoo Explorers

2012-2013

Zoo Explorers

Zoo Explorers 2012-2013

The Zoo Explorers program at the Lincoln Park Zoo offers teachers, students, and chaperones a field trip that aligns with the research of scientists at the zoo. Outlier’s mixed methods evaluation of the program was built around a set of evaluation questions derived from the project goals and includes elements of both process evaluation and progress evaluation in addition to the summative evaluation. Data collection included interviews, questionnaires, observations, and review of student work.

Project Resources

Visit the Evaluation section to learn more.

The Early Elementary Science Partnership

The Early Elementary Science Partnership (E2SP)

2010-2013

The Early Elementary Science Partnership

The Early Elementary Science Partnership (E2SP) 2010-2013

The E2SP was a three-year science education partnership between The Field Museum, the Peggy Notebaert Nature Museum, the Chicago Children’s Museum, the Lincoln Park Zoo, Chicago Public Schools’ Office of Science, and Northwestern University. Outlier evaluated the E2SP programming and collaboration using multiple data sources. Outlier also conducted case studies at two schools to provide a holistic view of a school’s experience in E2SP.

Project Resources

Visit the Evaluation section to learn more.

SETSEP

Science, Engineering and Technology for Students, Educators, and Parents (SETSEP) at the Museum of Science and Industry

2009-2013

SETSEP

Science, Engineering and Technology for Students, Educators, and Parents (SETSEP) at the Museum of Science and Industry 2009-2013

The SETSEP project, funded by the National Science Foundation, provided highly engaging, age appropriate, hands-on science and engineering activities for CPS students in grades K-3 and their parents. The evaluation determined changes in students and parents’ understanding of technology and engineering.

Project Resources

Visit the Evaluation section to learn more.

Civic Leadership Academy

Civic Leadership Academy

2015-2017

Civic Leadership Academy2015-2017

The Civic Leadership Academy (CLA) is an interdisciplinary professional development and training program for leaders in Nonprofit Organizations and Government Agencies in the City of Chicago. CLA’s objective is to equip participants, or Fellows, with key knowledge and skills for effective leadership through coursework taught by faculty from UChicago’s five professional schools, while also providing opportunities for Fellows to apply their learning in authentic settings. The program was developed by the University’s Office of Civic Engagement in partnership with LISC Chicago and the Civic Consulting Alliance, with funding from the Searle Funds at The Chicago Community Trust.

Project Resources

Visit the Evaluation section to learn more.