High School – Exploring Computer Science - Questionnaires

Respondents’ CS teaching experience varied widely, though many were experienced teachers overall.

Teachers reported varying levels of teaching experience. The largest percentage of teachers reported experience teaching for 6-10 years (25%), while the smallest percentage reported being in their first year of teaching (2%). More than one-fifth of teachers (21%) reported experience teaching for twenty-one years or more.

Teachers also reported varying years of experience teaching CS-related courses. Most teachers reported they were in their first year of teaching ECS (70%), with far fewer teachers reporting having taught the course for two (12%) and three (12%) years. The majority of teachers (84%) reported having never taught AP Computer Science, while large percentages reported having many years of experience teaching applications courses, like Keyboarding, Microsoft Office Suite, or Other Software Applications (44%), and “Other Computing Courses” such as Networking, Databases, Web Design, etc. (37%). The table below presents the six or more years of teaching these courses in a collapsed category labeled “6+ Years”, however 30% of the overall total reported for Keyboarding, Microsoft Office Suite, or Other Software Applications and 20% of the overall total reported for “Other Computing Courses” have taught the course for ten years or more.

On average, teachers like teaching with ECS and are committed to what ECS is trying to do.

Teachers were asked to report their feelings about the Exploring Computer Science (ECS) curriculum including their commitment to the ECS philosophy, as well as their attitude toward use of the instructional materials. In the questionnaire, on average, participants reported having positive feelings about the curriculum (4.047 out of 5). When asked whether they felt “committed to what ECS is trying to do,” respondents agreed “quite a bit” on average, giving that item a rating of 4.392 out of 5. Teachers reported that they “liked teaching with ECS materials” a “fair amount,” with an average rating of 3.782 out of 5. They also reported that “teaching with ECS materials is personally rewarding,” on average, agreeing “quite a bit” with the statement (3.965 out of 5).

Teachers like teaching with complete introductory CS curricula, like ECS.

Teachers were also asked more generally how they felt about teaching with a complete introductory compute science curriculum, like ECS. These items were asked with a six point scale. On average teachers agreed or strongly agreed that they liked having “a more complete introductory computer science curriculum, like ECS” (mean of 5.093 out of 6). They also reported wishing there were “more complete curricula for teaching computer science, like ECS,” giving that item an average rating of 4.5 out of 6.

Most teachers indicated completion of at least the first three units of ECS at the time of questionnaire administration.

Teachers were asked to report on the units they had completed at the time of the spring questionnaire. The timeframe in which a teacher could have responded to the questionnaire varied from late-March to early-June, so reports of completed units depends both on individual teachers’ progress, as well as when they took the questionnaire. Only a few teachers (2%) reported that they had not completed any units. Nearly all of the teachers (91%) reported completing Unit 1: Human Computer Interaction with nearly the same amount (90%) reporting that they had completed Unit 2: Problem-Solving. A majority (79%) completed Unit 3: Web Design, and 63% completed Unit 4: Introduction to Programming.

In all, nearly half of the teachers reported completing four units total (47%), with 23% reporting that they had completed three units. Few teachers (2%) had completed all six units at the time of questionnaire administration.

Fewer than a quarter of respondents completed Units 1 – 4 in the same amount of time specified in the ECS curriculum.

The amount of time teachers spent on each of the first 4 units of ECS varied widely; between 19 – 24% reported completing Units 1-4 in the amount of time specified in the ECS curriculum. For the rest of the teachers in the sample, most (23-65%) reported spending one or more weeks of additional time on Units 1-3 than specified in the materials. However, only 31% of teachers reported spending more than the time specified on Unit 4.

Most teachers use ECS unit/final projects and presentations to assess student learning (86%).

Teachers were asked to report on the ways they assess student learning, selecting all specified in the questionnaire that applied to their classroom. The most popular approach across all ECS units was unit/final projects and presentations (86%). The second most utilized assessment method was evaluation of student artifacts and portfolios (70%), followed closely by simple observation of student interactions with the computer and other course tools (67%). A sizeable percentage of teachers also reported using “other types of assessments” (39%) to assess student learning. These are described below.

A majority of teachers (79%) reported using multiple types of student assessment strategies.

Most teachers (92%) reported using at least two types of assessments to grade students in their introductory computer science class with over a third (34%) reporting that they used four different types of assessments, and 17% reporting that they used all of the assessment approaches, including an “other” type.

Some teachers create their own student assessments.

Nearly all of the teachers who reported using “other” assessments (i.e., those not specified in ECS, and not included in the list of questionnaire options) reported that they had used “something [they] created [themselves]” (97%; n = 68), while about one-third (32%) reported using “something another teacher created. Nearly half of these teachers (46%) reported using an “assessment from another source” (46%) including “online lessons or quizzes” and “Internet resources,” (e.g. Khan Academy, W3schools, Scratch.mit.edu forums, sam.course.com, typing.com, Google First, Code.org).

Roughly one quarter of teachers were “totally confident” in their ability to assess their students’ learning.

In addition to asking all teachers what they were using to assess their students’ learning, we also asked them to rate their confidence in their ability to provide a grade to students in their introductory computer science class. The majority of teachers in the sample (80%) reported high levels of confidence by responding they were “reasonably confident” (53%) or “totally confident (27%) on a 4-point response scale ranging from “Not at all confident” to “Totally confident.”

Teachers were also asked to comment on their confidence rating. Some teachers who reported low levels of confidence (“somewhat confident” or “not confident at all”) in their ability to grade their introductory CS students disclosed their challenges associated with assessment; the following quotes are a sample of responses specific to assessment challenges:

I question whether or not the assessments are good. I struggle with evaluating student work. Many times it seems subjective.

[I] haven’t been sure of what is important and should be graded and what is not.

In hindsight, I probably graded more heavily on the final product that I did on the process. I am still struggling with rubrics that adequately reflect a grading process.

There is not right or wrong answer for most of the projects. I feel there should be some solid projects that can be graded with a definite answer.

Unit tests are difficult to create.

This is my first time teaching computer science. I feel like I am learning with the students.

When there are mistakes, for instance, [when] code that doesn't work, I feel that I ought to figure out why it doesn't work. That is very time consuming. It is difficult for me, as a science teacher, to grade projects where there is no real ‘right’ answer.

It is hard to not get an A in this course if you are a member of the class and contribute to discussions, partner, and group activities. It is one of the problems I have with the work being done, I don't feel the students all deserve A's but the way the curriculum is written it is hard to separate out the A's, B's, and C's.

I don't think I have come up with good consistent methods of evaluating student work. Especially as primarily being a math teacher, I feel the grades are very subjective and arbitrary which I am not used to.

I would like more concrete means of assessment that wouldn't depend on my perception of their participation.