Our Work

We do research, evaluation, strategic planning,
and other change-focused work.

UI ASC

Evaluation of the University of Illinois Area Studies Centers

Department of Education

UI ASC

Evaluation of the University of Chicago Language Centers Department of Education
2015 – 2022

Outlier is collaborating with the Area Studies Centers Associate Directors at the University of Chicago to provide them with formative and summative feedback on their programs. We are focusing on two aspects of their work: 1) their partnership with the City Colleges of Chicago (CCC); and 2) the FLAS academic-year Fellowship program. Additionally, we are working work closely with the Area Studies Associate Directors to create a process by which to track and gather information from alumni graduating from their programming. Creating procedures for collecting data from alumni will be important to gauge the effects of their programming nationally and internationally, across different sectors of the workforce.

University of Chicago Area Studies Centers

Evaluation of the University of Chicago Language Centers

Department of Education

University of Chicago Area Studies Centers

Outlier Research & Evaluation at UChicago STEM Education: Department of Education
2018 – 2022

University of Chicago continues to collaborate with the Area Studies Centers Associate Directors at the University of Chicago to evaluate: 1) their partnership with the City Colleges of Chicago (CCC) and 2) the FLAS academic-year Fellowship program at The University of Chicago. These evaluations primarily aim to inform program improvement so that these programs can continue in the future. The secondary purpose of the evaluation is to provide findings that can demonstrate success and support future proposals for funding.

The evaluation of the partnership with the CCC and the FLAS Fellowship Program will continue In the current evaluation cycle (2018-2022). The partnership with the CCC will involve examining outreach efforts through quantitative questionnaire administration and qualitative interview data collection methods. Information collected during this phase of the evaluation will be used to improve outreach efforts with the CCC. A pre-(beginning of the academic year)/post-research design will be used to collect data from academic-year FLAS Fellows about their language skills, confidence in their language abilities, identity as a FLAS Fellow, and future career goals.

In addition, the evaluation will also involve working closely with the Area Studies Associate Directors to create a process by which to track and gather information from alumni graduating from their programming. Creating procedures for collecting data from alumni will be important to gauge the impact of UChicago Area Studies programming nationally and internationally, across different sectors of the workforce, and students focusing on different languages and content concentrations.

Network for College Success

Network for College Success

Network for College Success

Network for College Success 2019 - 2021

Outlier is working closely with the Network for College Success (NCS) in an iterative process to understand, evaluate, and improve their work. NCS is a University of Chicago-based organization that “envisions Chicago neighborhood high schools that continuously cultivate collaboration, powerful learning, and a culture of high achievement to prepare all students for college and career success.” The NCS model aims to improve equitable student outcomes at traditionally underserved Chicago neighborhood schools by fostering a deep connection and ongoing learning opportunities with each school’s staff. Outlier has developed a vibrant, intimate relationship with NCS’s Research & Development team to deeply understand how this model is enacted at the organization and school level. Our work focuses on Continuous Improvement (CI) strategies that facilitate ongoing and responsive strategies to advance all students.

Study of Innovation Spread

Study of Innovation Spread: The role of innovativeness, resourcefulness and readiness to change

Study of Innovation Spread

Study of Innovation Spread: The role of innovativeness, resourcefulness and readiness to change2018-2021

This is a Research-Practice Partnership (RPP) with Broward County Public Schools (BCPS) funded by the Hewlett Foundation. This RPP is part of a larger RPP community that, while focusing on varied areas, are all sharing the goal of contributing to knowledge about spread of innovations in schools. Our RPP is “DIG Deeper.” DIG Deeper aims to scale quality implementation of three teaching practices that support students’ critical thinking: “Debate,” (facilitation of argumentation), “Illustrate,” (supporting statements with evidence), and “Generate”( facilitating communication with clearly articulated thoughts and ideas). In addition to building knowledge, skills and expertise (capacity) among teachers, our strategy intentionally targets building teachers’ will for change. Our research will yield findings about spread strategies that focus on developing teacher will and organizational and individual readiness for change.

Project Resources

Big Shoulders Fund: All Are Welcome Program Evaluation

Big Shoulders Fund: All Are Welcome Program Evaluation

Big Shoulders Fund

Big Shoulders Fund: All Are Welcome Program Evaluation

Big Shoulders Fund: All Are Welcome Program Evaluation Big Shoulders Fund
2019-2020

The All Are Welcome (AAW) program aims to help educators in Catholic schools serve students who learn in diverse ways. These schools reside in the Chicago and Joliet Archdioceses .Educators in these schools face a number of challenges including: lack of knowledge about how to structure learning opportunities for diverse students, insufficient resources to screen students, an underdeveloped relationship with the public school system that results in few resources reaching Catholic schools, a lack of professional development for teachers, and limited connections to outside resources that provide direct services to students. The AAW program includes a certificate program for principals, professional development for teachers, collaborations with service organizations to screen students, assistance applying for available funds, and provision of direct services. The evaluation assesses the implementation and impact of the program including the extent to which teacher and school leader participation is associated with a more inclusive mindset about diverse learners, an ability to work more effectively in classrooms with diverse learners, a school culture that supported inclusion of diverse learners, and more effective use of outside resources.

i-Ready Implementation Study

i-Ready Implementation Study

Curriculum Associates

i-Ready Implementation Study

Curriculum Associates: i-Ready Implementation Study Curriculum Associates
2019-2020

Outlier is working with Curriculum Associates to learn about implementation of the “Top Teacher Actions” of their i-Ready program. We are applying our implementation and factor frameworks to create instruments for data collection and collaboratively creating a data collection design. The findings will enable Curriculum Associates to best support the practitioners across the country who are using i-Ready.

CLX

Evaluation of Chicago Learning Exchange Programming

CLX

Evaluation of Chicago Learning Exchange Programming2018 – 2020

Outlier is evaluating two programs that the Chicago Learning Exchange (CLX) launched in 2018. Both programs focus on promoting collaboration, digital media tools and technology use, and connected learning pedagogical approaches for youth. The first is “The Youth Digital Media Activism Grant” (YDMA) program that pairs youth-serving organizations with different focus areas to work together. One of the organizations in each pair emphasizes use of digital media tools and technologies while the other organization focuses on civics. With support from CLX, the organizations work together to empower youth to participate in media activism and to become more civically engaged. The second is the “Illuminating Energy Pathways” (IEP) program that focuses on creating a pathway to energy careers for Chicago’s youth. The program uses the LRNG platform to deliver a curriculum curated by learning designers and informed by overhead electrical line workers to help youth earn digital badges that “unlock” real-world experiences with industry partners. The mixed-method evaluation will inform program improvement as well as CLX’s grant making decisions.

Project Resources

Partnerships for Innovation: Creating Dibble

Partnerships for Innovation: Creating Dibble

National Science Foundation

Partnerships for Innovation: Creating Dibble

Strategic Planning for IEEE’s Pre-University Education Program National Science Foundation
2017 – 2020

Supported by the National Science Foundation’s Partnerships for Innovation program, Outlier is developing a prototype of an app called “Dibble.” In the tradition of Citizen Science projects, rather than participate in others’ scientific research, Dibble enables anybody to create a project for collecting and analyzing data. Dibble is particularly useful for educators in both, formal and out of school settings, who wish to create multiple project opportunities for learners while easily being able to monitor progress of the entire group. Dibble can also be used by mission driven organizations and by individuals. Prototype is expected in June 2020.

PRISMS

PRISMS
Plans to Realize Implementation of Standards in Mathematics and Science

Department of Education

PRISMS

PRISMS
Plans to Realize Implementation of Standards in Mathematics and ScienceDepartment of Education
2016-2018

This study is a research-practice partnership involving Outlier, the Consortium on School Research (our colleagues at the University of Chicago), and the Chicago Public Schools (CPS). The study is examining the implementation of district and school-level plans to realize the goals of the Common Core State Standards (CCSS) and Next Generation Science Standards (NGSS) in CPS. Across the country, districts and schools are expending considerable resources to enact strategies with the intention of meeting these standards. However, there is a risk that effective strategies are not implemented equitably across schools. This study is examining the varied implementation plans and the relationships between plan implementation, changes in instructional practices and student achievement. We are also looking at how these relationships vary for different subgroups of students. We will create both, scholarly papers as well as tools and resources for our CPS practitioner collaborators.

IEEE

Strategic Planning for IEEE’s Pre-University Education Program

IEEE

IEEE

Strategic Planning for IEEE’s Pre-University Education Program IEEE
2018 – 2019

IEEE is the world’s largest technical professional organization for the advancement of technology. Building on IEEE’s substantial history of providing educational opportunities for youth, IEEE and Outlier engaged in a planning and development process to determine the best strategy for moving forward. The work included a “landscape” review, interviews with stakeholders and identification of IEEE’s key assets. Then, we worked with IEEE to help them clearly articulate their goals for future K12 education work and collaborated to create a theory of action and implementation design for their pre-university education work. We finished with an evaluation framework with measurable benchmarks for success.

Brookfield Institute for Innovation + Entrepreneurship: Digital Literacy + Coding Pilot

Brookfield Institute for Innovation + Entrepreneurship: Digital Literacy + Coding Pilot

Brookfield Institute

Brookfield Institute for Innovation + Entrepreneurship: Digital Literacy + Coding Pilot Brookfield Institute
2017-2019

Outlier’s early collaboration with the Brookfield Institute for Innovation + Entrepreneurship (BIIE) entailed serving as a thought partner to develop an evaluation plan for their pilot. The pilot provided curriculum and support for an after-school program in youth-serving organizations in Ontario, Canada. Later, Then, BIIE engaged Outlier to evaluate the pilot project in six organizations, including YMCAs and Boys and Girls Clubs. The “Digital Literacy + Coding” name suggest the two key problems the pilot is intended to address. The first is a need for young people to be “digitally literate,” including being safe on the internet and critically assessing the information quality. The second is a gap in coding skills for young people, particularly girls and learners in underrepresented groups. The evaluation entails participant surveys, interviews with instructors and leadership staff at the hosting organizations, program observations, and focus groups with participants. Outlier’s final report will provide feedback on the potential for this pilot to be sustained in current sites and expanded to others.

Evaluation of the University of Florida’s Engaging Struggling Learners Study

Evaluation of the University of Florida’s Engaging Struggling Learners Study

National Science Foundatione

Evaluation of the University of Florida’s Engaging Struggling Learners Study

Evaluation of the University of Illinois at Urbana-Champaign Engaging Struggling Learners StudyNational Science Foundation
2017-2019

Dr. Maya Israel at the University of Florida is directing a project designed to 1) investigate the challenges that students with learning disabilities face during CS instruction, and 2) develop interventions to address those challenges based on research-based practices from other content areas. Outlier’s evaluation of this project consists of three components to support the work of the research team: a) process evaluation, to provide ongoing feedback about project work and to identify areas for improvement, b) ongoing quality review, to provide feedback on the quality of work with a specific focus on instruments, data analysis, and findings, and c) regular project leader meetings to serve as a research thought partner and help problem-solve immediate and emergent study issues.

Engaging Struggling Learners Study

Evaluation of the University of Illinois at Urbana-Champaign Engaging Struggling Learners Study

Engaging Struggling Learners Study

Evaluation of the University of Illinois at Urbana-Champaign Engaging Struggling Learners Study2017-2019

The CS for All: Engaging Struggling Learners in Computer Science Instruction study, funded by the National Science Foundation, is a project designed to (1) investigate the challenges that students with learning disabilities face during CS instruction, and (2) develop interventions to address those challenges based on research-based practices from other content areas. Outlier’s evaluation of this UIUC Department of Special Education project consists of three components to support the work of the research team: a) process evaluation, to provide ongoing feedback about project work and to identify areas for improvement, b) ongoing quality review, to provide feedback on the quality of work with a specific focus on instruments, data analysis, and findings, and c) regular project leader meetings to serve as a research thought partner and help problem-solve immediate and emergent study issues.

Time for CS

Finding Time for CS: Integrating Computer Science into the Elementary School Day

National Science Foundation

Time for CS

Finding Time for CS: Integrating Computer Science into the Elementary School Day National Science Foundation
2016 – 2019

Computer science is quickly becoming recognized as an essential part of K-12 education. Concurrently, there is increasing interest in computer science for our youngest learners in elementary schools. However, much of the elementary school day is devoted to English Language Arts and Mathematics. This research-practice partnership between Outlier and Broward County Public Schools (BCPS) sought to address this challenge. BCPS RPP team members developed and field-tested six multi-disciplinary modules (2 for each 3rd, 4th and 5th grade) that integrated computer science, science, social studies and literacy by building them around a problem-based storyline. Then, informed by evaluation data from the pilot, the team revised two additional modules as prototypes for further revisions. The associated implementation study examined not only if the modules successfully created time in the day for CS, but also looked at which parts pf the modules had associations with student outcomes on standardized ELA, mathematics and science tests.

Center on Halsted Needs Assessment

Center on Halsted Needs Assessment

Center on Halsted

Center on Halsted Needs Assessment

Center on Halsted Needs Assessment Center on Halsted
2018

Center on Halsted is a community center dedicated to advancing community and securing the health and well-being of the Lesbian, Gay, Bisexual, Transgender and Queer (LGBTQ) people of Chicago. Outlier conducted a needs assessment for Center on Halsted in order to inform the organization’s ability to realize several strategic priorities pertaining to better serving a wider range of audience populations.

Digital Media Tools and Technology Landscape Study

Digital Media Tools and Technology Landscape Study

Chicago Community Trust

Digital Media Tools and Technology Landscape Study

Digital Media Tools and Technology Landscape StudyChicago Community Trust
2017-2018

This study, funded by the Chicago Community Trust, captured the "landscape" of Digital Media Tools and Technologies (DMTT) in out-of-school programs for youth in Chicago. DMTT have shown promise as a means for empowering students and developing student investment in learning. This study sought to uncover where these opportunities are in Chicago and what they look like. The study included administration of questionnaires to all organizations in Chicago providing out-of-school programming for youth ages 14-24. Organizations provided information about the programs they offer, whether or not their programs use DMTT, and if so, the nature and purpose of that use. Data collection also included interviews with 12 of the responding organizations as well as site "stories" of five organizations. The final report shares findings from all data sources. CCT has widely distributed the report within the Chicago community and to those outside of Chicago who are interested in DMTT programming.

The USE Alliance

The USE Alliance – a Partnership to Support STEM Schools

Department of Education

The USE Alliance

The USE Alliance – a Partnership to Support STEM SchoolsDepartment of Education
2016-2018

The USE Alliance was a research-practice partnership that united Outlier researchers and Chicago Public Schools’ STEM policy-makers and practitioners. The Alliance targeted 12 elementary STEM Initiative (SI) schools located in the typically under-resourced South and West sides of Chicago. All of the K-8 SI schools faced numerous challenges as they were identified as “welcoming schools” (which meant that they were designated to receive students from schools that had just closed in a controversial decision) and simultaneously, they were anointed “STEM schools.”

The purpose of the USE Alliance was to combine data collection and analysis with capacity building in the CPS Office of College and Career Success (OCCS). Outlier collected, analyzed, and interpreted data on implementation of the STEM school benchmarks and their relationships to student outcomes using questionnaire data, secondary student achievement data, focus group data, school leader self-assessment data, and observational data. Outlier researchers also worked with OCCS to develop basic data structures for internal OCCS use and to build their capacity for conducting descriptive analyses and interpreting data.

YRC

Evaluation of the Lincoln Park Zoo Young Researchers Collaborative (YRC) and Partners in Fieldwork (PIF)

Lincoln Park Zoo

YRC

Evaluation of Young Researchers Collaborative (YRC) and Partners in Fieldwork (PIF) Programs at the Lincoln Park Zoo Lincoln Park Zoo
2009-2018

The Lincoln Park Zoo’s YRC and the PIF programs partnered with public school teachers to provide inquiry science experiences to Chicago-area middle and high school students. The Zoo provided teachers with training and the resources they needed to facilitate their students’ science skills through inquiry-based research activities. Through these programs, students and teachers worked together to carry out scientific investigations to solve conservation challenges facing their school or neighborhood communities. Outlier’s mixed-method evaluation was focused on questions of interest to the LPZ education department and included elements of process, progress and summative evaluation.

AP CSP

Computer Science & Students with Learning Differences study

National Science Foundation

Computer Science & Students with Learning Differences study National Science Foundation
2015 – 2017

This research-practice partnership was a collaboration between Outlier and Wolcott School (an independent college preparatory high school in Chicago for students with learning differences). Our work together targeted an often over-looked underrepresented group in computer science — students with learning differences (students with specific learning disabilities and attention deficit disorders). We worked with administrators, teachers and students at the school to look at the AP Computer Science Principles (CSP) and identify the challenges students with learning differences face as they engage with CS instruction and content. We identified possible solutions to address those challenges and tested specific adaptations and accommodations in lessons from two AP CSP curricula: Beauty and Joy of Computing; and Code.org’s CS Principles.

Project Resources

BASICS

Barriers and Supports to Implementing Computer Science (BASICS)

National Science Foundation

BASICS

Barriers and Supports to Implementing Computer Science (BASICS) National Science Foundation
2015-2017

The BASICS study examined how an introductory computer science program, Exploring Computer Science (ECS) was implemented in school districts with a focus on identifying the key supports for, and barriers to ECS’s and endurance. The study had both, practitioner and researcher audiences. We aimed to inform CS educators about ways to address challenges to bring CS education to high schools on a large scale and identify key strategies to support their efforts. We also sought to provide researchers with tools for measuring introductory high school CS program implementation and instruments for measuring the factors that affect implementation. We also created practitioner resources from the research findings.

Project Resources

Civic Leadership Academy

Evaluation of the University of Chicago Civic Leadership Academy

Civic Leadership Academy Supporters

Civic Leadership Academy

Civic Leadership Academy2015-2017

The Civic Leadership Academy (CLA) is an interdisciplinary professional development and training program for leaders in Nonprofit Organizations and Government Agencies in the City of Chicago. CLA’s objective is to equip participants, or “Fellows,” with key knowledge and skills for effective leadership through coursework taught by faculty from UChicago’s five professional schools, while also providing opportunities for Fellows to apply their learning in authentic settings. The evaluation used mixed methods to document fellows’ experiences and changes in their skills and network including focus groups, questionnaires and case studies.

Strategic Planning to Identify Future Directions for Hive Chicago

Strategic Planning to Identify Future Directions for Hive Chicago

Susan Crown Exchange

Strategic Planning to Identify Future Directions for Hive Chicago

Strategic Planning to Identify Future Directions for Hive Chicago Susan Crown Exchange
2016

Outlier engaged with Hive Chicago in a collaborative research and development process focused on identifying promising progress, lessons learned and new directions for the organization to enable them to best serve their member needs. This process included working with Hive leadership to articulate a problem statement, a theory of action and priority outcomes. Additionally, Outlier worked with Hive leadership to generate strategies that will move them closer to their outcomes.

Project Resources

iCorps – L: Developing Dibble

iCorps – L: Developing Dibble

National Science Foundation

iCorps – L: Developing Dibble

iCorps – L: Developing Dibble National Science Foundation
2016

The National Science Foundation’s iCorps-L program convenes teams of entrepreneurs who seek to commercialize ideas generated in NSF funded work. The iCorps-L program entails the team creating a Business Model Canvas and conducting 100 customer discovery interviews to inform revisions of the canvas over the iCorps-L program. A team from Outlier attended iCorps-L to develop an idea focused on creating a tool to enable educators, organizational leaders and individuals to crowd-source data and collaborate to answer questions of importance to them.

Strategic Planning for Amplify at the University of Chicago

Strategic Planning for Amplify at the University of Chicago

Lloyd A. Fry Foundation

Strategic Planning for Amplify at the University of Chicago

Strategic Planning for Amplify at the University of Chicago Lloyd A. Fry Foundation
2015 - 2016

Amplify is a partnership of five University of Chicago arts and cultural organizations—Arts + Public Life, the Court Theatre, the Oriental Institute Museum, the Reva and David Logan Center for the Arts, and the Smart Museum of Art—that have come together to develop a collaborative model of K12 arts learning at the University of Chicago. Amplify’s mission is to both broaden and deepen arts learning experiences for educators and students, particularly those living on the South Side of Chicago, through diverse program offerings within and across the five organizations. Outlier worked with Amplify leadership to conduct a collaborative research and development process to identify existing strengths in the current programs of each organization; articulate shared goals for collaborative work; and develop a strategic plan to realize those goals.

Google RISE Awards

Evaluation of the Google RISE Awards

Google

Google RISE Awards

Evaluation of the Google RISE AwardsGoogle
2015-2016

The Google RISE program helped nonprofit organizations around the world promote computer science education and run initiatives to reach K-12/pre-university girls, underrepresented minorities, and students facing socioeconomic barriers. In 2014, Google gave RISE awards to 42 organizations spanning 19 countries across the world. Outlier’s evaluation of the RISE program included working with leadership to articulate and define their program components, creating a typology of RISE Award recipients and richly describing how and to what extent the recipients used their RISE Awards to support, scale, and sustain their programs.

Code.org

Evaluation of Code.org Programs

Code.org

Code.org Code.org
2014-2016

Code.org is a non-profit organization dedicated to expanding participation in computer science, especially by reaching women and underrepresented students of color. In order to inform Code.org about their progress toward their overall mission and goals, Outlier's evaluation examined their course offerings, K-12, and measured student and teacher attitudes about computer science, as well as program implementation. The evaluation also explored potential relationships between program implementation, student and teacher attitudes, and student academic outcomes.

Project Resources

S3

The STEM School Study (S3)

National Science Foundation

S3

The STEM School Study (S3) National Science Foundation
2012-2016

This study aimed to clarify what inclusive STEM Schools are and how they might influence student outcomes. In order to do so, we partnered with 20 inclusive STEM high schools in seven states (California, New York, Ohio, Texas, Washington, Tennessee, and North Carolina). We worked with each school to articulate the elements of their inclusive STEM high school models and document the ways those model components are operationalized individually and in combination with others. We also identified the supports and barriers to model implementation and identified model elements that were most associated with desired student outcomes. In addition to study findings, we framework of STEM school model components and instruments for measuring enactment of those models, and create rich descriptions of STEM school practices.

Project Resources

Leading CS Growth

Leading Computer Science Growth:
A Toolkit for K-12 District and School Leaders Other Stakeholders

National Science Foundation

Leading CS Growth

(LeadCS.org)
Leading Computer Science Growth: A Toolkit for K-12 District and School Leaders Other Stakeholders National Science Foundation
2014 - 2015

This project addressed what was, at the time, an urgent need to provide K-12 education leaders with resources to support implementation of computer science education initiatives. Outlier and collaborators at UChicago STEM Education created research based-tools and recommendations which are housed at a website we named LeadCS.org. The site was designed for K-12 education leaders in schools and districts, teachers, and their partners looking to begin or improve a computer science education initiative. Some LeadCS.org tools provided summaries and syntheses of information, data, and projects that were current at the time. Other resources provided guidance and frameworks for leaders who, at the time, were ready to take actionable steps toward bringing computer science to their students. There were also “voices of experience” components that include recommendations from individuals who had already navigated the start of their computer science education process. The outcome of this work was intended to be a starting point on which to build a more comprehensive, robust and up-to-date toolbox.

Project Resources

STEM Indicators

Identifying and Measuring STEM Schools and Programs: A STEM School and Program Taxonomy

National Science Foundation

STEM Indicators

Identifying and Measuring STEM Schools and Programs: A STEM School and Program Taxonomy National Science Foundation
2014-2015

This project addresses one of the great challenges in education: precision in our language. Specifically, this project focused on bringing clarity to STEM schools and programs STEM schools and programs. The charge was to create a tool for addressing the first of the 14 indicators identified in the National Research Council Report, “Monitoring Progress Towards Successful K-12 Education: A Nation Advancing.” Indicator One has to do with the “Number of, and enrollment in, different types of STEM schools in each district.” We recognized that every state and district can have its own definition of a STEM school and that what is a STEM school in one place may not be in another. Accordingly, in partnership with other leading STEM school researchers, and with input from professional organization leaders and STEM school administrators across the country, we developed a “taxonomy” for STEM schools and programs. The taxonomy sets a clear base for STEM schools and programs and provides descriptions of additional possible STEM school and program components that assure the best fit for the school and program contexts. The taxonomy also addresses the project’s original charge by creating a foundation for the development of measures to count and describe the status of the nation’s STEM schools and programs.

Project Resources

CPS Logo

Chicago Public Schools STEM Standards for Success (SSS)

Chicago Public Schools

CPS Logo

Chicago Public Schools STEM Standards for Success (SSS) Chicago Public Schools
2014-2015

Building on Outliers close partnership with the Office of College and Career Success at Chicago Public Schools, we worked together to create tools that OCCS could facilitate self-improvement and targeted development in CPS’ STEM schools. The SSS project used CPS’ mission and vision for STEM education to develop concrete, measurable benchmarks for the STEM schools and then incorporated these benchmarks into a user-friendly rubric for schools across the district. Outlier also developed implementation measures for the identified benchmarks.

Factors that affect implementation

An Implementation Study of Everyday Mathematics: Factors that Affect Implementation, Spread, and Sustainability

National Science Foundation

Factors that affect implementation

An Implementation Study of Everyday Mathematics: Factors that Affect Implementation, Spread, and Sustainability National Science Foundation
2011 – 2015

This study investigated the implementation of the Everyday Mathematics elementary mathematics program and the factors that affect its implementation, spread and sustainability in educational settings. In this study we used our implementation and factor frameworks developed in earlier work to frame our data collection of specific EM components in the classrooms of several school districts. We analyzed the data to understand the status of EM implementation at the classroom and school levels, and investigated if and how specific components of EM vary according to maturity of implementation in each of the sites, and as a result of other implementation factors. Using a component approach to measure implementation uncovered variations in patterns of implementation that we referred to as implementation “types” and enabled us to examine variations in use within and across sites as well as relationships between types of implementation and student outcomes.

Project Resources

Tech Corps

Evaluation of TECH CORPS Techie Club and Techie Camp

Tech Corps

Tech Corps

Evaluation of TECH CORPS Techie Club and Techie Camp Tech Corps
2013-2014

TECH CORPS is a national non-profit organization dedicated to improving K−12 education at the grassroots level by bringing technology resources to K12 students and teachers through national, state and local programs. Outlier’s evaluation of TECH CORPS focused on the status of implementation for two of their programs: Techie Club and Techie Camp. The evaluation examined TECH CORPS mission and strategies as a whole, as well as specific and unique strategies and experiences between Clubs and Camps.

Validation of instruments

Validation of Instruments Measuring Implementation of Reform-Based Science and Mathematics Instructional Materials

Department of Education

Validation of instruments

Validation of Instruments Measuring Implementation of Reform-Based Science and Mathematics Instructional Materials Department of Education
2011-2014

Outlier received support from the Institute of Education Sciences to validate three teacher-level instruments for measuring innovation implementation (Teacher Questionnaire, Teacher Log, Classroom Observation Protocol) and to develop and validate a student-level questionnaire focused on student-reported engagement in mathematics and science instruction. Data were collected from over 400 K-5 teachers and 5,000 students in four school districts across three states (Colorado, Connecticut, and Illinois). We conducted tests of instrument reliability, measurement invariance between content domains, grades, and districts, and inter-rater reliability for the observation protocol. These instruments can and have been adapted for a variety of education innovations.

Ohio STEM Learning Network

The Ohio STEM Learning Network:
A Study of Factors Affecting Implementation, Spread and Sustainability

National Science Foundation

Ohio STEM Learning Network

The Ohio STEM Learning Network: A Study of Factors Affecting Implementation, Spread and Sustainability National Science Foundation
2010-2014

This study was Outlier’s first application of our implementation and factor framework to an innovation that was not instructional materials. We focused on Ohio STEM Learning Network (OSLN) a statewide network-oriented collaborative of partners from preK-12 education, higher education and business and industry in the state of Ohio. This study focused one portion of OSLN's effort at the time – the Platform Schools Initiative, which was supported by local and state resources and philanthropy including the Battelle Memorial Institute and the Bill & Melinda Gates Foundation. This initiative's goal was to help launch and connect five STEM secondary schools strategically placed in key economic and cultural regions of the state. We examined the status of implementation at each of the platform schools, the factors that contributed to or inhibited the implementation and spread of these models, and the factors that appeared to influence the sustainability of the schools.

Project Resources

Urban Education Teacher Program (UTEP)

Evaluation of the University of Chicago Urban Education Teacher Program (UTEP)

Supporter of UTEP program

Urban Education Teacher Program (UTEP)

Urban Education Teacher Program (UTEP) Supporter of UTEP program
2009-2014

The primary mission of the UTEP program is to prepare exceptional teachers for careers in urban public schools. The evaluation of this program included a range of data collection activities focused on program improvement and achieving outcomes in the areas of recruitment, retention, preparation, and effective teaching.

Project Resources

IGERT

Evaluation of the University of Chicago IGERT Program

National Science Foundation

IGERT

Evaluation of the University of Chicago IGERT Program National Science Foundation
2009-2014

The Motor Control and Movement IGERT program, funded by the National Science Foundation, was a collaboration between the University of Chicago, Northwestern University, and the Field Museum that brought together graduate students with backgrounds in biology, engineering, and mathematics to develop an integrative understanding of movement. The evaluation focused on the implementation of the program and the experiences of the graduate student trainees, with the goal of providing information to guide improvements to the program as it progressed.

University of Illinois at Chicago Gross Anatomy Lab

Evaluation of Innovation at the University of Illinois at Chicago Gross Anatomy Lab

University of Illinois at Chicago Gross Anatomy Lab

Evaluation of Innovation at the University of Illinois at Chicago Gross Anatomy Lab 2013

The University of Illinois at Chicago was implementing a different approach to anatomical sciences teaching and learning through a more student-centered learning environment. This approach shifted from the traditional model of using instructor produced video to an approach where anatomical information was captured and described by the students themselves. The evaluation focused on student participation in the lab as well as their attitudes toward the innovative lab structure.

The source

Evaluation of “The Source,” a program of the Ci3 Center at the University of Chicago

National Science Foundation

The Source

Evaluation of “The Source,” a program of the Ci3 Center at the University of Chicago National Science Foundation
2013

The Source was a STEM-based alternate reality game initiated by Ci3 and Game Changer Chicago Design Lab. Outlier’s portion of the larger evaluation effort had two strands. The first strand focused on youth attitudes toward STEM (science, technology, engineering and mathematics) and STEM careers. The second strand focused on the youth’s experience with The Source itself and their perspectives on the parts of The Source that impacted them the most.

Project Resources

ACM

Building an Operating System for Computer Science Education (OS4CS)

Association for Computing Machinery and Google

ACM

Building an Operating System for Computer Science Education (OS4CS) Association for Computing Machinery and Google
2012-2013

This research set out to establish a comprehensive understanding of the status of high school computer science in our nation’s schools. The goal was to provide a wide range of information and guidance that would inform and shape computer science education efforts. We designed this work as a collaborative research and communication effort that would collect data on our nation’s current high school computer science (CS) teaching population, the support they have, and contexts in which they teach. Outlier worked with others at the University in a partnership established by the Association for Computing Machinery (ACM) that included the National Science Foundation (NSF), Google, the Computer Science Teachers Association (CSTA), Microsoft, and the National Center for Women and Information Technology (NCWIT). We worked in collaboration with members of these organizations to execute data collection and get feedback on our work.

Project Resources

Boston Schoolyard Initiative

Evaluation of the Boston Schoolyard Initiative’s (BSI) Science in the Schoolyard Program

Boston Public Schools

Boston Schoolyard Initiative

Evaluation of the Boston Schoolyard Initiative’s (BSI) Science in the Schoolyard Program Boston Public Schools
2012- 2013

Outlier conducted a “legacy” evaluation of the BSI Science in the Schoolyard Program. A legacy evaluation is designed to describe the project and participants’ experiences at the project’s end in order to benefit others. The Science in the Schoolyard (SSY) program aimed to help teachers use the urban natural environment in their schoolyard (ranging from asphalt lots to BSI-designed outdoor classrooms) to teach the Boston Public School district science curriculum. Outlier examined the core components of the BSI SSY model, the implementation of outdoor science instruction across Boston Public Schools, and student and teacher outcomes.

Project Resources

Purple Asparagus

Evaluation of Purple Asparagus’ Delicious Nutritious Adventures

Purple Asparagus

Purple Asparagus

Evaluation of Purple Asparagus’ Delicious Nutritious Adventures Purple Asparagus
2012-2013

The Delicious Nutritious Adventures program was designed to help students and their families learn better eating habits and to make simple, healthy snacks and meals. The focus of the evaluation was to understand student and family participation in the Delicious Nutritious Adventures program activities, and the knowledge, behavioral, and attitudinal outcomes for participating youth and their families.

Zoo Explorers

Evaluation of the Lincoln Park Zoo - Zoo Explorers

Lincoln Park Zoo

Zoo Explorers

Zoo Explorers Lincoln Park Zoo
2012-2013

The Zoo Explorers program at the Lincoln Park Zoo offered teachers, students, and chaperones a field trip aligned with the research taking place at the zoo. Outlier’s mixed methods evaluation of the program was built around a set of evaluation questions derived from the project goals and included elements of both process evaluation and progress evaluation in addition to the summative evaluation.

Thinking Skills

Evaluation of the “Developing 21st Century Thinking Skills” Program at St. Xavier

Illinois Board of Higher Education

Thinking Skills

Evaluation of the “Developing 21st Century Thinking Skills” Program at St. Xavier Illinois Board of Higher Education
2010-2013

The Developing 21st Century Thinking Skills program was a partnership between the School of Education at Saint Xavier University and Chicago Public Schools (CPS), and the International Renewal Institute, Inc. (iRi). The program trained selected K – 2 teachers and instructional leaders in Reuven Feuerstein’s Theory of Structural Cognitive Modifiability and to teach the Feuerstein Instrumental Enrichment, Basic course. Outlier determined the degree to which the project met its objectives for professional development and how the program was used in schools.

The Early Elementary Science Partnership

Evaluation of Informal Institution Partnerships: The Early Elementary Science Partnership (E2SP)

The Early Elementary Science Partnership

Evaluation of Informal Institution Partnerships: The Early Elementary Science Partnership (E2SP) 2010-2013

The E2SP was a three-year science education partnership between The Field Museum, the Peggy Notebaert Nature Museum, the Chicago Children’s Museum, the Lincoln Park Zoo, Chicago Public Schools’ Office of Science, and Northwestern University. Outlier evaluated the E2SP programming and collaboration using mixed methods including case studies at two schools to provide a holistic view of a school’s experience in E2SP.

SETSEP

Evaluation of the Science, Engineering and Technology for Students, Educators, and Parents (SETSEP) program at the Museum of Science and Industry

SETSEP

Evaluation of the Science, Engineering and Technology for Students, Educators, and Parents (SETSEP) program at the Museum of Science and Industry 2009-2013

The SETSEP project, funded by the National Science Foundation, provided engaging hands-on science and engineering activities for CPS students in grades K-3 and their parents. The evaluation determined changes in students and parents’ understanding of technology and engineering.

Google CAPE

Evaluation of the Google Computing and Programming Experience (CAPE)

Google CAPE

Evaluation of the Google Computing and Programming Experience (CAPE) 2012

Google CAPE was a multi-week summer program located in Google’s Mountain View and New York City campuses. Outlier’s evaluation focused on understanding the Google CAPE intended model, and measuring implementation of core model components, as well as student and faculty outcomes. This study was structured under a highly collaborative model, to provide timely feedback to CAPE program staff and faculty.

Project Resources

Baxter Science Education Initiative

Evaluation of the Baxter Science Education Initiative

Baxter Science Education Initiative

Evaluation of the Baxter Science Education Initiative 2008-2011

The Biotechnology Center for Excellence located at Lindblom Math and Science Academy hosted professional development sessions for teachers in the biotechnology cohort throughout Chicago Public Schools and housed the biotechnology resource center, an in-house and online collection of tools and curricula for teachers integrating biotechnology into their classrooms. This evaluation examined changes in teacher knowledge, confidence and competency with key biotechnology concepts and tools as a result of participation in professional development. Additionally, Outlier examined student outcomes in relation to enrollment in biotechnology courses, such as confidence using biotechnology tools and enrollment in post-secondary education

MC2 STEM Hub

Evaluation of the MC2 STEM Hub

MC2 STEM Hub

Evaluation of the MC2 STEM Hub 2010

The MC2 STEM Hub was part of the Ohio STEM Learning Network, a statewide network-oriented collaborative of partners from preK-12 education, higher education and business and industry in the state of Ohio. The MC2 Hub was a network-based infrastructure entity that had the goal of amplifying the reach and impact of Science, Technology, Engineering, and Mathematics (STEM) education. Outlier evaluated the extent to which the project activities contributed to MC2’s infrastructure to support spread of innovation and its ability to generate regional impact and provide support to Cleveland’s K-8 STEM School development.

FOI

Applied Research on Science Materials Implementation:
Bringing Measurement of Fidelity of Implementation (FOI) to Scale

National Science Foundation

FOI

Applied Research on Science Materials Implementation: Bringing Measurement of Fidelity of Implementation (FOI) to Scale National Science Foundation
2007-2010

This project created the foundation for all of Outlier's implementation measurement work: the implementation framework. This project was focused on developing instruments to measure the use of science and mathematics instructional materials in schools and classrooms in Chicago. Its three phases were (1) the development of a conceptual framework to study FOI, (2) the development of a suite of instruments for measuring FOI, and (3) the exploration of types of materials use that are associated with improved student outcomes.

Accumulating Knowledge

Accumulating Knowledge on Scaling and Sustaining Reform:
A Foundation for Future Research

National Science Foundation

Accumulating Knowledge

Accumulating Knowledge on Scaling and Sustaining Reform: A Foundation for Future Research National Science Foundation
2007-2009

This study sought to provide a conceptually sound foundation for accumulating knowledge about scaling and sustainability of innovations in education. The first strand of project work entailed a review of literature in education, health, marketing, business and economics. The initial search led to an analysis of 30,828 abstracts and 572 reports. This analysis resulted in a conceptual organizer of the factors that contribute to and inhibit implementation, spread and sustainability of reform in education. An interdisciplinary conference was held to discuss and review this work in September 2009.