In this section, Dr. Jeremy Orloff and Dr. Jonathan Bloom describe how problem set checkers were used in 18.05.
Students completed weekly problem sets, which they submitted on paper.
For each problem, students were provided with an online problem set checker run through our MITx site. The interface was similar to that of the reading questions. Students could input each answer into a box and find out if the answer was correct or incorrect. For problems for which the answers were formulas, students were asked to check their answers by inputting a numerical value into the formula. Students could use the problem set checker as many times as they wanted to, and they weren’t graded on this in any way. A version of the problem set checker is available on the Assignments page of this OCW course.
Because students could check their answers, they were able to notice mistakes and fix incorrect work before submitting the problem sets. We believe the best time for students to be aware of their mistakes is when they are actively working on problem sets; at that point, they’re most engaged in the work and invested in solving the problems correctly. In contrast, by the time graded problem sets are returned, students are usually distanced from solving those problems, occupied by new assignments, and lacking motivation because their grades have already been determined. Indeed, some students don't even bother to collect their graded problem sets.
We didn’t worry that students would find the answers by using the problem set checker. Answers had to be correct to the nearest hundredth, making it onerous for students to guess-and-check their way to the correct answer. Even if they did find the correct answer this way, it wouldn’t matter; problem set grades were based on logic and completeness of their written solutions rather than the correctness of the final numerical answer.
Students really loved the problem set checker. Based on our web analytics, 90% of the students checked the majority of their problem sets; about half the students checked all of their problem sets.