Our strengths enable us to do things
differently and develop innovative,
robust approaches to our projects.
Engaging Youth in a Connected World:
A Landscape Study of Digital Media Tools and Technologies in Chicago’s Out-of-School Time Programs>
2017 – 2018
Engaging Youth in a Connected World: A Landscape Study of Digital Media Tools and Technologies in Chicago’s Out-of-School Time Programs.2017 – 2018
Outlier Research & Evaluation collaborated with the Chicago Learning Exchange, formerly the Hive Chicago Fund for Connected Learning at The Chicago Community Trust, to investigate digital media tools and technology (DMTT) use in Chicago out-of-school time (OST) youth programs.
This study is the first of its kind; therefore, Outlier designed it to ensure that it 1) used clear language to make the findings easy to understand and interpret; 2) was replicable by others interested in learning more about the DMTT landscape in their city’s or community’s programming; and 3) was representative of the variety of OST programs currently being offered to youth participants in the city of Chicago. Using questionnaire, interview, and site visits Outlier documented the types of DMTT used, the goals program leaders held for participating youth, and the learning strategies carried out to accomplish those goals.
Findings indicate that OST programs provided youth with opportunities to engage with many different types of DMTT. DMTT was often used to help participants reach program goals, including cultivating “youth development and leadership,” “21st century skill learning,” and “civic engagement.” They did so by using learning strategies such as “keeping youth actively engaged,” “supporting youth collaboration,” and “engaging youth in creating and making.” Other findings highlight thoughtful advice from OST program practitioners as well as youth perspectives on DMTT use in the programs where they participate. This study illustrates the importance of DMTT use in OST programs, and also provides the foundation for future work striving to examine the impact of DMTT in Chicago, and other major cities across the United States.
Report Press Release:
Other Press Coverage:
Strategic Planning and Development for IEEE
2018 - 2019
Strategic Planning and Development for IEEE2018 – 2019
IEEE is the world’s largest technical professional organization for the advancement of technology. Building on its substantial history of providing educational opportunities for youth, IEEE and Outlier will be engaging in a strategic planning and development process. This process aims to clearly articulate goals for future programming, outline implementation strategies, and identify measurable benchmarks for success. It will set the stage for a multi-year implementation and evaluation plan.
Study of Innovation Spread: The role of innovativeness, resourcefulness and readiness to change
Study of Innovation Spread: The role of innovativeness, resourcefulness and readiness to change2018-2020
This project, funded by the Hewlett Foundation, is a Research-Practice Partnership (RPP) with Broward County Public Schools (BCPS). This project will scale quality implementation of deeper learning practices targeting student critical thinking to at least 1,600 grades 4-5 teachers (80% of the total) in BCPS. The research will 1) yield findings about spread strategies that focus on the development of teacher will or intention to implement new practices by building both organizational and individual readiness for change, and 2) examine relationships among these change strategies, and relationships between the strategies and teacher implementation of instructional practices supporting students’ critical thinking.
Digital Media Tools and Technology Landscape Study
Digital Media Tools and Technology Landscape Study2017-2018
This study, funded by the Chicago Community Trust, aimed to capture the "landscape" of Digital Media Tools and Technologies (DMTT) in out-of-school programs for youth in Chicago. The study included administration of questionnaires to all organizations providing programming, out of school, for youth ages 14-24. Organizations provided information about the programs they offer, whether or not their programs use DMTT, and if so, the nature and purpose of that use. Data collection also included interviews with 12 of the responding organizations as well as site "stories" of five organizations. The final report shares findings from all data sources. CCT is widely distributing the report within the Chicago community and to those outside Chicago who are interested in programming with DMTT.
The USE Alliance
The USE Alliance2016-2018
The USE Alliance is a unique research partnership that unites Outlier STEM education researchers Chicago Public Schools’ STEM policy-makers and practitioners.
The USE Alliance targets 12 elementary STEM Initiative (SI) schools located on the typically under-resourced South and West sides of Chicago. All of the K-8 SI schools are “welcoming schools,” which, the name notwithstanding, were developed amid much controversy. Welcoming schools are those that received students when other local neighborhood schools were closed as part of a highly contested and divisive district decision to close 54 of the city’s schools in 2013. This context is critical for understanding the experiences of school leaders, teachers, and students in these schools because at the same time that they were designated “welcoming schools,” the 12 SI schools were also suddenly anointed “STEM schools,” adding a drastic shift in educational model to the already existing pressures they faced.
The goals of the USE Alliance are to collect, analyze, and interpret implementation and student outcome data on the eleven K-8 STEM Initiative schools. The USE Alliance will employ a mixed-methods study. The USE Alliance will measure implementation of the STEM school benchmarks and its relation to student outcomes, as measured by teacher questionnaire data, secondary student achievement data archive, focus group data, school leader self-assessment data, and observational data. Researchers at the University of Chicago will work with OCCS to develop basic data structures for internal OCCS use, as well as work with OCCS staff to build their capacity for conducting descriptive analyses and interpreting data.
The USE Alliance will use the data and findings from the study to develop timely, targeted, and effective support strategies for the 12 participating STEM Initiative schools, and develop a long-term collaborative research plan and action steps for funding of future research on STEM Initiative schools. The USE Alliance will disseminate products and publications to inform K-8 STEM education policymakers and practitioners locally and nationally.
Plans to Realize Implementation of Standards in Mathematics and Science
Plans to Realize Implementation of Standards in Mathematics and Science2016-2018
This study will explore the implementation of district and school-level plans to realize the goals of the Common Core State Standards (CCSS) and Next Generation Science Standards (NGSS) in mathematics and science in Chicago Public Schools. Specifically, it is examining the relationships between different implementation plan, strategies and changes in instructional practices and student achievement, and whether these relationships vary for different subgroups of students. Across the country, districts and schools are expending considerable resources to enact instructional materials, provide professional development and other strategies with the intention of meeting these standards. However, there is a risk that effective strategies are not implemented equitably across schools. Because these strategies are key influences on whether standards will ultimately lead to improvements in student learning, we are examining whether student achievement and equity in student achievement have changed with the adoption of the CCSS and NGSS and how those changes are related to variation in strategy implementation in schools and classrooms. Building from the findings of this study, school and district leaders can make research-informed decisions about their strategies and supports for the meeting the goals of CCSS and NGSS.
Time for CS in Elementary School Project (“Time for CS”)
National Science Foundation
Jan 2016 – Dec 2017
Time for CS
Integrating Computer Science into Elementary Literacy
Jan 2016 – Dec 2017
Time for CS is a National Science Foundation (NSF) funded 2-year exploratory study that aims to bring computer science to the elementary school day. Outlier Research & Evaluation at UChicago STEM Education at the University of Chicago, in partnership with Broward County (FL) Public Schools (BCPS), seeks to understand the implementation and outcomes of interdisciplinary problem-based modules that are taught as part of a 180 minute literacy block. The modules and integrate computer science, science, social studies and literacy by building them around a problem-based storyline. There are two modules for each 3rd, 4th and 5th grade and all address BCPS standards for each discipline at the respective grade level.
The overarching goal of “Time for CS” is to create an evidence-based model for bringing computer science into the already full elementary school day. It seeks to do this by capitalizing on the promising practice of problem-based learning by creating multi-disciplinary modules to be used during the already established literacy block. More specifically, this study will:
- Create six multi-disciplinary (science, social studies, computer science, literacy) problem-based modules that integrate by focusing problem-centered storyline.
- Develop a model for the infrastructure (including professional development and stakeholder advocacy) that supports the use of these modules in elementary schools;
- Rigorously examine the merits of the CS integration model by looking at relationships to attitudinal and academic student outcomes; and
- Increase elementary students' exposure to STEM, specifically CS, as part of the regular school day.
Visit the Research section to learn more.
Computer Science & Students with Learning Differences study
National Science Foundation
2015 – 2017
Computer Science & Students with Learning Differences study National Science Foundation
2015 – 2017
This 2-year exploratory study, conducted in collaboration with the Wolcott School (an independent college preparatory high school in Chicago for students with learning differences), aims to make the AP Computer Science Principles (CSP) course more accessible for students with learning differences (students with specific learning disabilities and attention deficit disorders).
In support of this goal, our researcher - practitioner team is identifying the challenges students with learning differences face as they engage with CS instruction and content; identifying possible solutions to address those challenges, and; testing specific adaptations and accommodations in lessons from two AP CSP curricula: Beauty and Joy of Computing; and Code.org’s CS Principles.
The STEM School Study (S3)
National Science Foundation
The STEM School Study (S3) National Science Foundation
This study comprehensively describes and measures models of 20 inclusive STEM high schools in seven states (California, New York, Ohio, Texas, Washington, Tennessee, and North Carolina), measures the factors that affect their implementation, and examines the relationships between model components and a range of student outcomes. The study contributes to the field and the growing attention to STEM schools by a) describing the elements of inclusive STEM high school models and the ways those elements are operationalized individually and in combination with others; and b) identifying and describing elements of the schools that appear to be related to desired student outcomes. In addition to study findings, this project will develop a clear framework for describing STEM school models and instruments for measuring enactment of those models, identify the factors that affect implementation, and create rich descriptions of STEM school practices.
Leading Computer Science Growth:
A Toolkit for K-12 District and School Leaders and Other Stakeholders
National Science Foundation
2014 - 2015
Leading Computer Science Growth: A Toolkit for K-12 District and School Leaders Other Stakeholders2014 - 2015
This project addresses the need of K-12 education leaders for resources to support implementation of computer science education initiatives. Our team has created research based-tools and recommendations housed at LeadCS.org, a website for K-12 education leaders in schools and districts and their partners looking to begin or improve a computer science education initiative.
Some LeadCS.org tools provide summaries and syntheses of current information, data, and projects. Other tools are “voices of experience” with recommendations coming from leaders who have already navigated the change process. These tools provide key advice from leader reflections on the most critical supports, barriers, and lessons learned in the process of implementing a computer science program. Still others provide guidance and frameworks for leaders ready to take actionable steps toward bringing computer science to their students.
Barriers and Supports to Implementing Computer Science (BASICS)
National Science Foundation
Barriers and Supports to Implementing Computer Science (BASICS) National Science Foundation
The Barriers and Supports to Implementing Computer Science (BASICS) Study examines how an introductory computer science program, Exploring Computer Science (ECS) is implemented in school districts with a focus on identifying the key supports for and barriers to that implementation and endurance.
The goals of the study are to:
- Inform NSF CS10K leaders and other CS educators about the supports for and barriers to wide-scale high school Cs education and provide strategies for addressing them;
- Provide tools for measuring introductory high school CS program implementation and instruments for measuring the factors that affect implementation; and
- Create resources from research findings and recommendations that will be useful to the NSF CS10K community and the broader education field.
Building an Operating System for Computer Science Education (OS4CS)
Building an Operating System for Computer Science Education (OS4CS) ACM, Google
This study was designed as a collaborative research and communication effort to establish a more comprehensive understanding of our nation’s current high school computer science (CS) teaching population, the support they have, and contexts in which they teach. Outlier, as the research and evaluation arm of the University of Chicago’s Center for Elementary Mathematics and Science Education (CEMSE), and the University’s Urban Education Institute (UEI) worked with a partnership established by the Association for Computing Machinery (ACM) that included the National Science Foundation (NSF), Google, the Computer Science Teachers Association (CSTA), Microsoft, and the National Center for Women and Information Technology (NCWIT) to provide a wide range of information and guidance to inform and shape CS education efforts.
Factors that Affect Implementation, Spread, and Sustainability:
An Implementation Study of Everyday Mathematics
National Science Foundation
2011 – 2015
Factors that Affect Implementation, Spread, and Sustainability: An Implementation Study of Everyday Mathematics National Science Foundation
2011 – 2015
This study is investigating the implementation of the Everyday Mathematics elementary mathematics program and the factors that affect its implementation, spread and sustainability in educational settings. In this study, we use the instruments developed in earlier work to collect data on the specific EM components that are present in the classrooms of several school districts, use that data to understand the status of EM implementation at the classroom and school levels, and investigate if and how specific components of EM vary according to maturity of implementation in each of the sites and as a result of the influence of the implementation factors. Using a components approach to measuring implementation will uncover variations in patterns of implementation that we refer to as implementation “types” and enable us to examine variations in use within and across sites as well as relationships between types of implementation and student outcomes.
Rigorous measures of implementation:
A Suite of Tools for Evaluating STEM Instructional Materials Use
National Science Foundation
Rigorous Measures of Implementation: A Suite of Tools for Evaluating STEM Instructional Materials Use National Science Foundation
This study is funded through NSF’s Promoting Research and Innovation in Methodologies for Evaluation (PRIME) program and as such, is focused on using implementation measures in the context of program evaluation. The study’s primary aim is to explore the use of our implementation frameworks and adaptations of our instruments for measuring use of elementary science and mathematics instructional materials to help school and district leaders understand more about their program and its relationships to student outcomes. Specifically, we are exploring the ways implementation data, innovative methodologies for data analysis and visualization, and communication methods in schools and districts are most helpful. This project emphasizes the development of an implementation evaluation process that includes regular engagement with school personnel and using implementation data to inform adoption, implementation, and professional development decisions.
Validation of Instruments Measuring Implementation of Reform-Based Science and Mathematics Instructional Materials
Department of Education
Validation of Instruments Measuring Implementation of Reform-Based Science and Mathematics Instructional Materials Department of Education
Outlier Research & Evaluation received support from the Institute of Education Sciences to validate three teacher-level instruments for measuring innovation implementation (Teacher Questionnaire, Teacher Log, Classroom Observation Protocol) and to develop and validate a student-level questionnaire focused on student-reported engagement in mathematics and science instruction. Data were collected from over 400 K-5 teachers and 5,000 students in four school districts across three states (Colorado, Connecticut, and Illinois). Analysis will include tests for construct validity and reliability, tests of measurement invariance between content domains, grades, and districts, and a test of inter-rater reliability for the observation protocol. Convergent validity will be examined for instruments administered concurrently, comparing Classroom Observation to Teacher Log, and comparing Teacher Questionnaire to Student Questionnaire. Predictive validity will be tested by examining relationships between patterns of curriculum use and student outcomes.
Applied Research on Science Materials Implementation:
Bringing Measurement of Fidelity of Implementation (FOI) to Scale
National Science Foundation
Applied Research on Science Materials Implementation: Bringing Measurement of Fidelity of Implementation (FOI) to Scale National Science Foundation
This project created the foundation for all of Outlier's implementation measurement work: the implementation conceptual framework. This framework grew from this project that was focused on developing instruments to measure the use of science and mathematics instructional materials in schools and classrooms. Its three phases were (1) the development of a conceptual framework to study FOI, (2) the development of a suite of instruments for measuring FOI, and (3) the exploration of types of materials use that are associated with improved student outcomes. The first phase of this project resulted in Outlier’s conceptual framework for measuring innovation use. The second phase led to the development of instruments that measure use of science and mathematics instructional materials. Finally, the third phase was a starting point for the study of Everyday Mathematics currently underway.
Accumulating Knowledge on Scaling and Sustaining Reform:
A Foundation for Future Research
National Science Foundation
Accumulating Knowledge on Scaling and Sustaining Reform: A Foundation for Future Research National Science Foundation
This study sought to provide a conceptually sound foundation for accumulating knowledge about scaling and sustainability of innovations in education. The first strand of project work entailed a review of literature in education, health, marketing, business and economics. The initial search led to an analysis of 30,828 abstracts and 572 reports. This analysis resulted in a conceptual organizer of the factors that contribute to and inhibit implementation, spread and sustainability of reform in education. An interdisciplinary conference was held to discuss and review his work in September 2009.
Identifying and Measuring STEM Schools and Programs
National Science Foundation
STEM Indicators: Identifying and Measuring STEM Schools and ProgramsNational Science Foundation
This project addresses the first of the 14 indicators identified in the National Research Council’s “Monitoring Progress Towards Successful K-12 Education: A Nation Advancing”: “Number of, and enrollment in, different types of STEM schools in each district.” The Outlier project team, in partnership with other leading STEM school researchers, is developing a “taxonomy” that identifies and describes types of STEM schools and programs. The tool will provide the field with much-needed clarity and shared understanding of the landscape of STEM education, and will lay the groundwork for the development of measures to count and describe the status of the nation’s STEM schools and programs.
Chicago Public Schools STEM Standards for Success (SSS)
Chicago Public Schools STEM Standards for Success (SSS) 2014-2015
Outlier has developed a close collaborative partnership with the Office of College and Career Success at Chicago Public Schools. The SSS project seeks to utilize CPS’ mission and vision for STEM education to develop concrete, measurable benchmarks for STEM schools. These benchmarks will be incorporated into a user-friendly rubric for schools across the district. The SSS project will also develop measures of implementation for the identified benchmarks. These measures will facilitate self-improvement and targeted development for CPS STEM schools.
The Ohio STEM Learning Network:
A Study of Factors Affecting Implementation, Spread and Sustainability
National Science Foundation
The Ohio STEM Learning Network: A Study of Factors Affecting Implementation, Spread and Sustainability National Science Foundation
The Ohio STEM Learning Network (OSLN) is a statewide network-oriented collaborative of partners from preK-12 education, higher education and business and industry in the state of Ohio. This study focused on one portion of OSLN's effort – the Platform Schools Initiative, which was supported by local and state resources and philanthropy including the Battelle Memorial Institute and the Bill & Melinda Gates Foundation. This initiative's goal was to help launch and connect five STEM secondary schools strategically placed in key economic and cultural regions of the state. This study sought to understand the status of implementation at each of the platform schools, the factors that contributed to or inhibited the implementation and spread of these models, and the factors that appeared to influence the sustainability of the schools.
University of Chicago Area Studies Centers
2015 - 2022
Outlier Research & Evaluation at UChicago STEM Education: 2018 – 2022
University of Chicago continues to collaborate with the Area Studies Centers Associate Directors at the University of Chicago to evaluate: 1) their partnership with the City Colleges of Chicago (CCC) and 2) the FLAS academic-year Fellowship program at The University of Chicago. These evaluations primarily aim to inform program improvement so that these programs can continue in the future. The secondary purpose of the evaluation is to provide findings that can demonstrate success and support future proposals for funding.
The evaluation of the partnership with the CCC and the FLAS Fellowship Program will continue In the current evaluation cycle (2018-2022). The partnership with the CCC will involve examining outreach efforts through quantitative questionnaire administration and qualitative interview data collection methods. Information collected during this phase of the evaluation will be used to improve outreach efforts with the CCC. A pre-(beginning of the academic year)/post-research design will be used to collect data from academic-year FLAS Fellows about their language skills, confidence in their language abilities, identity as a FLAS Fellow, and future career goals.
In addition, the evaluation will also involve working closely with the Area Studies Associate Directors to create a process by which to track and gather information from alumni graduating from their programming. Creating procedures for collecting data from alumni will be important to gauge the impact of UChicago Area Studies programming nationally and internationally, across different sectors of the workforce, and students focusing on different languages and content concentrations.
University of Illinois Area Studies Centers
2018 - 2022
UOutlier Research & Evaluation at UChicago STEM Education: 2018 – 2022
University of Chicago has begun to collaborate with Area Studies Centers Associate Directors at the University of Illinois to evaluate: 1) the FLAS summer Fellowship program at the University of Illinois and 2) the Spurlock Big History Artifact program with middle school students in the Champaign-Urbana community. These evaluations primarily aim to inform program improvement so that these programs can continue in the future. The secondary purpose of the evaluation is to provide findings that can demonstrate success and support future proposals for funding. Mixed-methods data collection approaches, including administering quantitative questionnaires and conducting qualitative interviews, will be used to evaluate both programs.
Evaluation of Chicago Learning Exchange Programming
2018 - 2020
Evaluation of Chicago Learning Exchange Programming2018 – 2020
Chicago Learning Exchange (CLX) has engaged Outlier Research & Evaluation to evaluate two programs launched in 2018 — The Youth Digital Media Activism Grant Program (YDMA) and the Illuminating Energy Pathways (IEP) Program. Both programs focus on promoting collaboration, digital media tools and technology use, and connected learning pedagogical approaches to help youth participants reach desired program goals.
The YDMA grant pairs youth-serving programs to work together. One program has an emphasis on digital media creation while the other program is focused on civics. They work together to help youth participate in media activism and to become more civically engaged. For instance, youth from the Community TV Network (media creators) were paired with youth participating in Vote 16 Illinois (civic organizers) to spark conversations about lowering the voting age in Illinois. Collaboration as part of this program aims to help organizations, and their participants, redefine civic engagement through the powers of civic media.
The IEP program focuses on creating a pathway to energy careers for Chicago’s youth. The workforce in the energy sector, particularly individuals employed as overhead electrical line workers, is aging. Accordingly, in the coming years it will be critical to educate youth about potential career options in the energy field in order to fill the influx of available jobs. The program uses the LRNG platform to deliver a curriculum curated by learning designers and informed by overhead electrical line workers to help youth earn digital badges that “unlock” real-world experiences with industry partners. Engaging with this curriculum will enable youth to carry out hands-on learning with their peers, make decisions about their learning with the support of adult mentors, and take risks as they learn about careers in energy fields.
In both cases, the evaluation of these programs involves conducting interviews with individuals who were actively involved in the ideas behind and the creation of the programs (i.e., program conceptual developers) and key program stakeholders, including learning designers, organizational leaders, and youth participants. Knowledge gained from these interviews will inform future CLX grant-making decisions as well as the types of out-of-school time programs they support for Chicago’s diverse youth. Interviews are currently underway, and an interim report will be shared with CLX in March 2019.
Evaluation of the Foreign Language and Area Centers, University of Chicago
Evaluation of the Foreign Language and Area Centers, University of Chicago2015-2020
Outlier Research & Evaluation at UChicago STEM Ed | University of Chicago is working in collaboration with the Foreign Language and Area Centers leadership to evaluate: 1) their work with the City Colleges of Chicago (CCC) and; 2) the FLAS Fellowship program at The University of Chicago. The primary purpose of the evaluation is to inform program improvement. The secondary purpose of the evaluation is to provide findings that can demonstrate success and support future proposals for funding.
The majority of this year’s evaluation plan is targeting the University of Chicago National Resource Center’s (NRC) joint outreach project with City Colleges of Chicago. In particular, the NRCs are holding a variety of educator outreach events (conferences, educator institutes, customized programming and workshops) specifically for CCC faculty. The NRCs seek to capture detailed quantitative and qualitative data from CCC faculty about their perceptions of the NRC offerings, as well as information about the specific needs of CCC faculty. The NRCs will use this information to improve their outreach efforts, and potentially to generate new programming that will best meet the specific needs of the CCC faculty.
In addition to the CCC project, Outlier will continue the evaluation of the FLAS program. Specifically, Outlier will collect survey data from FLAS fellows about their self-efficacy with regard to their foreign language skills; their career plans; their satisfaction with the support received from the NRCs; and the extent to which they have an identity as a FLAS fellow.
Evaluation of the University of Illinois at Urbana-Champaign Engaging Struggling Learners Study
Evaluation of the University of Illinois at Urbana-Champaign Engaging Struggling Learners Study2017-2019
The CS for All: Engaging Struggling Learners in Computer Science Instruction study, funded by the National Science Foundation, is a project designed to (1) investigate the challenges that students with learning disabilities face during CS instruction, and (2) develop interventions to address those challenges based on research-based practices from other content areas. Outlier’s evaluation of this UIUC Department of Special Education project consists of three components to support the work of the research team: a) process evaluation, to provide ongoing feedback about project work and to identify areas for improvement, b) ongoing quality review, to provide feedback on the quality of work with a specific focus on instruments, data analysis, and findings, and c) regular project leader meetings to serve as a research thought partner and help problem-solve immediate and emergent study issues.
Evaluation of Young Researchers Collaborative (YRC) and Partners in Fieldwork (PIF) Programs at the Lincoln Park Zoo
Evaluation of Young Researchers Collaborative (YRC) and Partners in Fieldwork (PIF) Programs at the Lincoln Park Zoo 2009-2018
The YRC and the PIF programs are designed to help Chicago-area middle and high school students take a “hands-on” learning approach to inquiry-based science education by partnering with public school teachers. Through these programs, students build their research skills and further develop their interests in science. Teachers help facilitate students’ science skills through inquiry-based research activities, and they are provided with training sessions and resources to do so. For example, through these two programs, students and teachers work together to carry out scientific investigations to solve conservation challenges facing their school or neighborhood communities. The conservation challenges identified by students are varied, and include solving problems related to pollution, recycling, and protecting the habitats of urban plants and wildlife.
This mixed-method, short-term longitudinal evaluation is built around questions of interest to the Lincoln Park Zoo’s Education Department, and it includes elements of process, progress, and summative evaluation. To answer these evaluation questions, students and teachers complete questionnaires to assess their knowledge of key program content, their experiences in completing inquiry-based scientific research projects, and their interest in science, more broadly. Students and teachers also have the opportunity to share their first-hand perspectives about the program through focus groups and interviews. Through this evaluation, Outlier strives to provide the Lincoln Park Zoo with valuable insight on how to make inquiry-based science a key component of public school science instruction.
Civic Leadership Academy
Civic Leadership Academy2015-2017
The Civic Leadership Academy (CLA) is an interdisciplinary professional development and training program for leaders in Nonprofit Organizations and Government Agencies in the City of Chicago. CLA’s objective is to equip participants, or Fellows, with key knowledge and skills for effective leadership through coursework taught by faculty from UChicago’s five professional schools, while also providing opportunities for Fellows to apply their learning in authentic settings. The program was developed by the University’s Office of Civic Engagement in partnership with LISC Chicago and the Civic Consulting Alliance, with funding from the Searle Funds at The Chicago Community Trust.
Evaluation of the Google RISE Awards
Evaluation of the Google RISE AwardsGoogle
Google RISE helps nonprofit organizations around the world to promote computer science education and run initiatives that reach K-12/pre-university girls, underrepresented minorities, and students facing socioeconomic barriers. The RISE Awards program is committed to helping these organizations to inspire the next generation of computer scientists by raising interest and exposure to the field of CS. In 2014, RISE awards were granted to 42 organizations spanning 19 countries across the world. These organizations collectively reached over 87,000 youth with programs and experiences in computer science. The evaluation of the RISE Awards program being conducted by Outlier aims to collaboratively work with Google RISE leadership to articulate and clearly define their program components and overall strategy, create a typology of past and current RISE Award recipients, and richly describe how and to what extent recipients are using RISE Awards to support, scale, and sustain their programs.
Code.org is a non-profit organization dedicated to expanding participation in computer science, especially by women and under represented students of color. Code.org believes that every K-12 student should have the opportunity to learn computer science. Outlier's evaluation of Code.org examines all of their course offerings, K-12, and measures student and teacher attitudes about computer science, as well as program implementation. The second year of evaluation (2015-16) will also explore potential relationships between program implementation, student and teacher attitudes, and student academic outcomes.The evaluation also examines Code.org’s progress towards their overall mission and goals and strategies used to achieve their vision of broadening participation in computer science.
Urban Education Teacher Program (UTEP)
Urban Education Teacher Program (UTEP) 2009-2014
The primary mission of the UTEP is to prepare exceptional urban public school teachers who are equipped with the knowledge and skills to instruct middle school and secondary students in Chicago Public Schools in the areas of biology and mathematics. The evaluation of this program includes a range of data collection activities focused on program improvement and achieving outcomes in the areas of recruitment, retention, preparation, and effective teaching.
The Motor Control and Movement IGERT program, funded by the National Science Foundation, is a collaboration between the University of Chicago, Northwestern University, and the Field Museum that brings together graduate students with backgrounds in biology, engineering, and mathematics to develop an integrative understanding of movement. The evaluation focuses on the implementation of the program and the experiences of the graduate student trainees, with the goal of providing information that will guide improvements to the program as it progresses.
TECH CORPS 2013-2014
TECH CORPS is a national non-profit organization dedicated to improving K−12 education at the grassroots level by helping educators effectively use technology in their schools. The evaluation of TECH CORPS examines the status of implementation for two of their programs: Techie Club and Techie Camp. The evaluation examines the TECH CORPS mission and strategies as a whole, as well as specific and unique strategies and experiences between Clubs and Camps.
The Source 2013
The Source is a STEM-based alternate reality game initiated by Ci3 and Game Changer Chicago Design Lab. Outlier’s portion of the larger evaluation effort had two strands. The first strand focused on youth attitudes toward STEM (science, technology, engineering and mathematics) and STEM careers. The second strand focused on the youth’s experience with The Source itself and their perspectives on the parts of The Source that impacted them the most.
Boston Schoolyard Initiative’s (BSI) Science in the Schoolyard Program
Boston Schoolyard Initiative’s (BSI) Science in the Schoolyard Program 2012- 2013
Outlier conducted a “legacy” evaluation (an evaluation designed to describe the project and participants’ experience at the project’s end in order to benefit others’ learning) of BSI’s Science in the Schoolyard professional development program. The Science in the Schoolyard (SSY) program helps teachers use the urban natural environment in their schoolyard (ranging from asphalt lots to BSI-designed outdoor classrooms) to teach the Boston Public School district science curriculum. Evaluators examined the core components of the BSI SSY model, the implementation of outdoor science instruction across Boston Public Schools, and student and teacher outcomes.
Purple Asparagus Delicious Nutritious Adventures
Purple Asparagus Delicious Nutritious Adventures 2012-2013
The Delicious Nutritious Adventures program is designed to help students and their families learn better eating habits and to make simple, healthy snacks and meals. The focus of the evaluation was to understand student and family participation in the Delicious Nutritious Adventures program activities, and the knowledge, behavioral, and attitudinal outcomes for participating youth and their families.
University of Illinois at Chicago Gross Anatomy Lab
University of Illinois at Chicago Gross Anatomy Lab 2013
The purpose of this project was to implement a different approach to anatomical sciences teaching and learning through a more student-centered learning environment. This approach shifted from the traditional model of using instructor produced video to an approach where anatomical information was captured and described by the students themselves. The evaluation focused on student participation in the lab as well as their attitudes toward the innovative lab structure.
Zoo Explorers 2012-2013
The Zoo Explorers program at the Lincoln Park Zoo offers teachers, students, and chaperones a field trip that aligns with the research of scientists at the zoo. Outlier’s mixed methods evaluation of the program was built around a set of evaluation questions derived from the project goals and includes elements of both process evaluation and progress evaluation in addition to the summative evaluation. Data collection included interviews, questionnaires, observations, and review of student work.
Developing 21st Century Thinking Skills
Developing 21st Century Thinking Skills 2010-2013
This project was a partnership between the School of Education at Saint Xavier University and Chicago Public Schools (CPS), and the International Renewal Institute, Inc. (iRi). Funded by the Illinois Board of Higher Education (IBHE), the project trained selected K – 2 teachers and instructional leaders in Reuven Feuerstein’s Theory of Structural Cognitive Modifiability and to teach the Feuerstein Instrumental Enrichment, Basic course. Outlier determined the degree to which the project met its objectives for professional development and how the program was used in schools.
Google Computing and Programming Experience (CAPE)
Google Computing and Programming Experience (CAPE) 2012
Outlier conducted an evaluation of Google CAPE, a multi-week summer program located in Google’s Mountain View and New York City campuses. This study focused on understanding the Google CAPE intended model, and measuring implementation of core model components, as well as student and faculty outcomes. This study was structured under a highly collaborative model, to provide timely feedback to CAPE program staff and faculty.
The Early Elementary Science Partnership (E2SP)
The Early Elementary Science Partnership (E2SP) 2010-2013
The E2SP was a three-year science education partnership between The Field Museum, the Peggy Notebaert Nature Museum, the Chicago Children’s Museum, the Lincoln Park Zoo, Chicago Public Schools’ Office of Science, and Northwestern University. Outlier evaluated the E2SP programming and collaboration using multiple data sources. Outlier also conducted case studies at two schools to provide a holistic view of a school’s experience in E2SP.
Baxter Science Education Initiative
Baxter Science Education Initiative 2008-2011
The Biotechnology Center for Excellence located at Lindblom Math and Science Academy hosted professional development sessions for teachers in the biotechnology cohort throughout Chicago Public Schools and housed the biotechnology resource center, an in-house and online collection of tools and curricula for teachers integrating biotechnology into their classrooms. This evaluation examined changes in teacher knowledge, confidence and competency with key biotechnology concepts and tools as a result of participation in professional development. Additionally, Outlier examined student outcomes in relation to enrollment in biotechnology courses, such as confidence using biotechnology tools and enrollment in post-secondary education
Science, Engineering and Technology for Students, Educators, and Parents (SETSEP) at the Museum of Science and Industry
Science, Engineering and Technology for Students, Educators, and Parents (SETSEP) at the Museum of Science and Industry 2009-2013
The SETSEP project, funded by the National Science Foundation, provided highly engaging, age appropriate, hands-on science and engineering activities for CPS students in grades K-3 and their parents. The evaluation determined changes in students and parents’ understanding of technology and engineering.
MC2 STEM Hub
MC2 STEM Hub 2010
Part of the Ohio STEM Learning Network, the MC2 STEM Hub is a network-based education innovation infrastructure based in Ohio with the goal of amplifying the reach and impact of Science, Technology, Engineering, and Mathematics (STEM) education. Outlier evaluated the extent to which the project activities contributed to MC2’s infrastructure to support spread of innovation and its ability to generate regional impact and provide support to Cleveland’s K-8 STEM School development.