In this section, Dennis Freeman describes how, in the past, instructors conducted oral exams with students to assess their learning in 6.01 Introduction to Electrical Engineering and Computer Science I. He shares the advantages and limitations of this strategy, and concludes by considering how online exams might help instructors offer scalable assessments that are personalized and productive.
Reflecting on Oral Exams
We informally assess students during the four and half hours of lab each week. We get to know them and have a pretty good idea of what they do and do not understand. We also give exams in the course. I don't particularly like the exams because I think the important thing is the learning that's done in the lab. The exams don't always do a good job of assessing the things that we hope they learned in the lab.
If we could offer […] exams online, we could personalize assessments in a way that's more scalable than oral exams, and much more effective than what we currently do with paper-and-pencil tests.
—Dennis Freeman
For several years, we gave oral exams, and they were much better. This allowed us to say to students, “In this lab, we were trying to design a circuit to give the robot vision. So what was the hard part? What was easy?” We asked very open-ended questions, from which we learned a lot. We would start with an easy question, a question that we expected everybody would get. If the student got it easily, then we would ask a harder question. If the student didn't get it easily, then we'd ask another easier question. If he or she sailed through the easy question, we went straight to a difficult question. In other words, we adjusted to the student’s level, and it was really quick. In 10 minutes, and sometimes fewer, we had a good feeling for the student’s level of understanding.
The hard part of giving oral exams was that we needed many evaluators to assess the large number of students in the course. Also, it was time-consuming to train all of the evaluators. The one-on-one discussions were extremely effective, but we haven't found a way to scale them yet.
The Potential of Online Exams
If we could offer the 6.01 Introduction to Electrical Engineering and Computer Science I exams online, we could personalize assessments in a way that's more scalable than oral exams, and much more effective than what we do with paper-and-pencil tests. Unlike paper-and-pencil exams that require someone who can do A-level work to answer six C-level questions before getting to the A-level questions, computer-adaptive tests would allow students to focus on questions at their level. This seems like it would be a far better use of students' time.