Lecture 10: Course Summary and Your Questions

• Download English-US transcript (PDF)

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

PROFESSOR: What I want to do now is basically summarize the theme of each of the sessions that we've had, partly as a way of reminding you of all the things we did and also jogging your memory or any question that you might have about that. And then it'll be your chance to ask questions. So I'm going to do it, so I'll think aloud here. I'll do it in a way that I can leave them up the whole time.

So let me do it here. I'll do it on these two guys. Yeah, I'll do it here.

So our sessions and the summaries of them-- so session number one was the introduction. But I'll start with session two, was teaching equations. Now the fundamental idea I wanted people to learn in this one is to help students learn equations in big chunks.

So doing derivations is generally antithetical to that. Because students will just see the little symbols, and they'll see a ton and ton and ton and ton of symbols. And they're chunk size is much smaller than you, so you will overload their short term memory. And they won't really be understanding, they'll be just trying to remember and write as fast as they can.

So is that visible, the orange? Yeah? Great. Oh yeah, you're a good test.

So teaching equations, you want to try and chunk the equations, because that gives meaning for the students. So, for example, what does each term mean? Why would you expect that term to show up in the equation? Those are the kinds of things that a book generally does not do very well. And that's something that you, as a teacher, actually have a lot of value to add.

Now I just learned about a book that does try to do this, which is called A Student's Guide to Maxwell's Equations. Does anybody know that book? Yeah, you know it? So, it looks fantastic. I'm number seven or something in the queue to get a copy from the MIT library, which shows how popular it is.

But the way it's organized-- well, first of all, it's sold an immense number of copies for a technical textbook, something like 7,000 or something like that. So it's a physics best seller. It's not an Agatha Christie novel, but it's really well done.

And each chapter talks about one of the equations. So there's four, roughly, Maxwell's equations. And there's four chapters, one per equation. So it's helping the students chunk just by the very structure of the books. But most books don't do that, aren't that well or that interestingly constructed, and that thoughtfully constructed, so it's something that you, as a teacher, really should do.

The next one-- misconceptions. So why did we spend a whole session on misconceptions? Well, it's fundamentally important, misconceptions, because if you don't know where your students are, you have no chance of bringing them to where you want them to go. The example that's classic, that's [? probably ?] started this whole line of research about misconceptions, was students' understanding of force and motion.

And basically, what you find is that most peoples' intuitive understanding of force and motion is f equals mv. If you have an object, you stop pushing it, it stops moving. Force and velocity are connected. If you have a heavier object, you have to push it harder. If you have a lighter object, you push it less. So force and mass are connected, so it looks like f equals mv.

Whereas, we would like to teach them f equals ma, Newton's Second Law, force equals mass times acceleration. So there's a fundamental difference there, and if you don't take account of that, you'll end up producing just rote knowledge. They'll say OK, well whenever I happen to be in physics class, I'll use f equals ma, and I'll just solve problems symbolically using that procedure you've told me. But I don't really believe it, or understand it, or use it in my own reasoning, so my intuition is no longer available for reasoning. So by taking account of students' misconceptions and helping them come to an intuitive understanding of how things really work, you're actually making them much smarter.

So this is knowing-- so that's knowing how students think. So this is pretty much opposite to behaviorism.

Because you're saying, well, I really want to peer into how people are reflecting, and how their cognition works, what's going on under the hood. And behaviorism says, well, no, all you need to do is look at the behavior. If they're solving problems correctly, that's fine.

Whereas actually you want to know, well, why did they solve it that way? Is it because they really understood it or not? Did they just say 6 times 3 is 18, because they were told, or they memorized it? Or was it because they really understood it, in which case they could make a word problem related to it?

So knowing how students think, you're actually part of the cognitive revolution. And you're actually connecting to students, directly and making them much smarter.

Homework and exam questions-- so constructing homework and exam questions, the main theme of that was Bloom's Taxonomy. So this was a hierarchy of goals that you could have. So the full name is A Taxonomy of Educational Objectives.

And the ones we're talking about, if you look at the full title, it's the cognitive domain. There's the cognitive and affective. Affective means related to feelings and emotions. This is the cognitive domain ones I'm talking about here.

So are you asking for just comprehension? Are you asking for students to analyze? Are you asking them to synthesize knowledge? What is the level of cognitive activity that you're asking for?

And generally, you want to mix levels. So you don't want to just push the students off a cliff and say-- the first day in thermodynamics class, discuss how the Second Law of Thermodynamics applies to the entropy of the universe? That's just too hard. You've pushed them way off the cliff, and you jumped way too high in Bloom's Taxonomy.

You also don't want to just give them pure comprehension problems, because they remember the rule that you're trying to teach. Rather, you want to mix them, and Bloom's Taxonomy gives you a structured way of seeing what level your questions are at and making sure you mix them, so that you prepare students comprehension questions, so that they can later do evaluation questions.

Course design-- so for course design, the big idea is big ideas. So you want to organize your course around some kind of large ideas. And ideally, those large ideas will transfer outside of the course too.

And if a whole curriculum is organized around a few large ideas, for example, waves and oscillatory motion, you could also organize a whole bunch of a physics major around ideas like that, or conservation. By giving structure to the curriculum, you turn it away from just a series of unconnected facts, like we did this topic, that topic, that topic, that topic, actually you give it form and organization. And then it's much more likely to be built into long lasting chunks in the students' minds.

So the next one was interactive teaching. The sixth session was interactive teaching. And the main theme is that questioning and reflecting lead to long lasting learning.

So I showed you three or four different time scales on which you could be interactive. A short one is when you ask a question, you wait 5 seconds. Or actually some of you suggested seven, which is probably better. It's just harder to do, but it's probably better. So you wait a certain amount of time to allow people to actually formulate questions.

So that promotes interaction on a short scale. You can ask short questions, one or two minutes, two or three minutes, with a multiple choice right in the middle of lecture. It gives you feedback. So it's related to the misconceptions one. You're learning how students think, just as they answer the questions.

Or on the same time scale, you can ask the feedback questions from this sheet at the end of the lecture. So that's also promoting a space for questions. And you can actually see how successful that is. There's so many questions, actually, we could spend two more sessions answered all the questions, which I'm glad about.

Because questioning and reflecting-- that's how people make knowledge their own. So the formal name for that is constructivism. But what you're doing is you're helping people construct knowledge.

And you can do this on a long scale. Like for example, for the whole lecture, you can use a question like the word blocks, which I showed you earlier. You can actually do that for an entire lecture.

And actually, we just did that for entire lecture in my Art of Approximation class. Because the word blocks leads to xylophones, and I brought in a xylophone. And we can talk about how the pitches of the thick and thin wood blocks relate to the pitches of shorter and longer xylophone slats.

So rich examples like that-- one doesn't have that many of them. But when you do have them, you can actually build an entire session around them. And it produces all of these good things.

Seven-- lecture planning and performing. So for the planning of the lecture, just like for the course, the planning of a course, you want to organize it around some kind of big idea, some kind of thing that gives it structure. The whole lecture, you want an objective.

For example, by the end of the lecture, I want students to be able to explain the origin of the terms in the Navier-Stokes equation. So then you would choose your activities in lecture around this goal. But that goal, actually just like the longer scale one for the whole course, gives structure to the lecture. And you know what to put in and what not to put it. And the lecture planning blank sheets, or the templates I gave you, I posted, you can use those to that end.

And that's for the planning part. And for the performing, I would say the main thing in performing that people don't normally think about when they think about teaching is timing. So if you want to create little bits of suspense and interest, little bits of tension, that are then released. And you want to do that on multiple scales-- short and long.

The interactive teaching questions from here help do that, especially if you have a demonstration. Because people want to know, is the thick wood block going to be higher pitch or is it going to be a lower pitch? And you'll find that when you do things like that, the entire room is dead silent, because people really want to know.

And the way you structure the questions can promote that. By having people vote, they've now made a public commitment, which then activates the cognitive dissonance part of people's mind. By making a public commitment, they've now made their internal state more towards the public state, which is oh, I want to know what's going on. And they really do want to know what's going on.

So timing-- you can get a lot of timing ideas from the interactive teaching methods. But timing is really important for performing. Imagine jokes where people tell you the punchline first, and then they tell you the rest of the joke. They don't go over too well, and most comics, at least most successful comics, don't do that. But we usually do that with our teaching.

Blackboards and slides for teaching-- so the main concepts to think about there are the chunk size that students have and how many spots they have in short term memory. So with the blackboard, you have many advantages for free, which is that on the blackboard, you don't have to use too many short term memory slots. You can put the entire session all on one giant set of blackboards in the front. And you can offload a lot of the work onto something that's in the visual field and present for everyone, all the time.

Because remember, their chunk size is much smaller than yours. So you might see the entire lecture as one chunk. But obviously, they don't see it that way, or they wouldn't be in the lecture. They're the learners.

And then when you're doing the slides, if you're going to use slides for teaching, to mitigate some of this problem of not having enough short term memory slots, generally not enough space, well, you could try to get four slide projectors or three slide projectors. But it's pretty hard to do. And it triples or quintuple or [? nontuples ?] the time to prepare the slides, because you have to synchronize three different slide presentations. So at least use assertions at the title and then visual evidence.

Visual evidence is much easier to remember. And the assertions help people know what to look for. So they're not spending all their cycles, which they don't have that many of, because their short term memory's been filled up. They're spending their cycles trying to understand the visual evidence.

And I posted a set of example slides with the [? text ?] source. There was a request for the [? text ?] source, so I posted that, as well, showing this method of presentation for the factorial, the logarithm of the factorial.

And then the last one was [INAUDIBLE]. So maybe the summary could be-- the comment that was on one of the sheets was is there any hope?

[LAUGHTER]

PROFESSOR: And I think, well, that question-- it's coupled to another question. So the main theme from that is that social and educational change are coupled. So if there's no hope for education, there's no hope for society.

Or alternatively, if there's hope for society, there's hope for education, and vice versa. So the main positive lesson from this is that by improving society, you improve education, and also, by improving education, you improve society. So that's one really good reason to want to be a teacher and to make teaching a part of your career, as I'm sure many of you are thinking about doing.

So those are the short summaries of the nine sessions. And what I'd like to do now is give you a chance to ask any questions you have. We'll, of course, have our-- let's see, in 10 minutes, at 10 o'clock, we'll take a 10 minute break. But before that, I want to start the question session.

So the way we'll do that is if everyone could just look at your index card that you brought with a question, or it doesn't have to be an index card, it could be a sheet of paper, and spend one minute thinking about something that puzzles you, you wonder about, you're interested in, you'd like to know more about, about anything related to teaching, and check in with a neighbor or two. And then we'll just start doing questions. And then we'll have our break at 10:00.

I'll leave all these guys up here, and I'll mark your questions.

[INTERPOSING VOICES]

PROFESSOR: Take another 10 or 20 seconds to formulate a question or formulate your thoughts, and we'll start.

AUDIENCE: So--

PROFESSOR: [INAUDIBLE] yeah?

AUDIENCE: Implicitly throughout this semester, we've been thinking that our audiences, like the college age audience-- so if you're teaching older people, [INAUDIBLE] things that you have to do differently?

PROFESSOR: Good question. So let me repeat the question, and then also, that will give me a chance to bring in one of the other comments from the question sheets. So the question was what about teaching older learners, so people who aren't college age, so middle aged people? Do you have to do things differently?

And now, before I answer that, let me say why I repeat the question. So their comment was that it's a good idea to repeat the question, because OCW asks that I repeat the question. Because the pick up, the microphone pick up, I guess, is here, and it picks up me much better than the audience.

So it's good for that, but it's also good, as the commenter pointed out. It's good, because it shows the students who are asking the question that I actually value their questions. Because I'm actually telling everybody, look, this is something we should all be thinking about. So it brings the whole class together around it, and it also gives me time to think about the question. Although, in this case, I didn't actually use any of that time to think about the question, so let me think about the question now.

So I'm not sure it actually gives you that much time. It gives you one or two seconds. But actually, if you want one or two seconds, it's best to just sit there still and think for one or two seconds. So that's what I'm going to do now.

[LAUGHTER]

PROFESSOR: So now I waited actually four seconds just to show that it's possible to do that, though difficult.

[LAUGHTER]

PROFESSOR: So there are some things that are different. And I would say one of the main differences is not just the misconceptions, but the conceptions of learning that people have. So for example, you might be teaching people who just failed out of math in school and never did math again, and now you want to help them learn math. Or you want to help them learn reading, and they always thought of themselves as terrible readers. So now they've imbibed the lesson of the blue eyes brown eyes experiment, which is that they're bad at this-- for whatever reason, it's their fault.

And we're very good at that. We're very good at passing blame. So why didn't people learn physics? Well, physics is a very hard subject. It's not that we teach it terribly, it's a very hard subject.

So basically, we're saying, blame the victim. I know that's very convenient. But after a while of that, the victim actually starts to get Stockholm Syndrome. So they start to identify with their captors. And they agree, yes, physics is a really hard subject. I never had a head for physics.

So as an experiment for that, next time you go to a party, ask people what you do-- I mean, obviously not a party of other graduate students in science and engineering, because that test wouldn't work here. But just for example, my office mate, when I was a graduate student, he used to live in Hollywood, in Hollywood, itself. So he used to go to parties, and his roommate was a script rewriter. So his roommate would actually-- scripts that were considered too bad even to make movies out of, but they really thought they should make a movie out of it, because they thought they could make a lot of money if they just rewrote the script a bit, so like D-grade scripts that needed to be made C-grade, his roommate rewrote them. So he had a lot of Hollywood connections.

So he had a lot of parties with people who weren't from Caltech, which was good. So go to a party like that, and when people say, oh what do you do? Say, oh I'm a chemistry graduate [? student, ?] I teach physics, or I'm a TA for this class, or I'm working on a Ph.D. in mathematics. And just watch how fast people run.

[LAUGHTER]

PROFESSOR: So if they're not very polite, they'll say oh, what do you do? Right? But if they're more polite, they'll actually say, oh, yeah, I always was terrible with that. That was never my thing. And this is so prevalent.

So now, what do you have to do in that case? So that's their misconception. And so number three up there, you have to really take account of that. Their way of thinking about math and physics, the way that it was taught to them completely did not work, and it just produced fear and phobia. So now what you have to do is you have to go around that.

And so one of the ways actually Brian Butterworth, who actually studies this-- he's a professor of neurophysiology, in London. And I mentioned his book a couple times, I think, it's called The Mathematical Brain or What Counts, depending on whether it's the English or American title. So math phobia a lot caused by people having a fear that there's only one way to get the right answer.

So they think they're walking across a tight rope, and if you make it one misstep, then you fall into the abyss. And so, of course, if you have that conception of mathematical reasoning and problem solving, you're going to have a lot of fear. So what you have to do is you have to show people, no, actually-- right over here. So if this was their model, that if that's the one way to get to the solution, actually you want to show them not that, and show them even dead ends.

Not every method of solution will work out. Some might just end up there, and some might end up there. And some connect to other methods and get to a solution. And even maybe, that's one solution, but maybe there's two.

So you want to show them that there's robust paths across the river, let's say. You're not trying to, for example, jump Niagara Falls, where one misstep and down you go. So that's a way of reducing the math phobia. Now that's a particular example, but that's an example of how you have to think slightly differently for the adult learner.

The reason is that they've had longer time to internalize the misconception. The high school student, for example, they haven't necessarily formed their own view of them self. And it's less true at the college level, that people are starting to form their view of self more and more, but it's still changeable, especially in America. And that's one really excellent part of the American university system. So you may not realize it if you come from America, but in almost every other country in the world, when you go to university, you pick your subject and that's what you do for the rest of the university time.

So England, for example, it's even before that. If you want to do physics at Cambridge, you have to have done, for the last two years of high school-- basically everyone who did physics at Cambridge did physics, math, chemistry, and further maths, I should say. So further maths are just more maths-- basically like DE calculus. so they did, for the last two years of school, only math and physics and chemistry courses. And then they go on to university, and they do just physics.

And Cambridge had a radical innovation on that, which was that in the first year, you didn't do just physics, you did physics, math, chemistry, and maybe a fourth subject. So that was considered really very progressive. And it was compared to the rest of the educational system. So in Europe, in general, there isn't that freedom.

So in America, you still have it. So the students in college are still going to be quite different from the people who've basically fixed their view of them self as bad at math or bad at whatever. And that's one of the things that you change when you're thinking about adult learners. I'll take one question back there, and then we'll have our break.

AUDIENCE: So towards the beginning of the class, we talked about the experiment in math teaching in New Hampshire

PROFESSOR: Benezet, the experiment?

AUDIENCE: Yeah, the Benezet experiment. I don't think we ever really got to figuring out what happened to that. [INAUDIBLE].

PROFESSOR: So what happened to the Benezet experiment in New Hampshire? So Louis Benezet was superintendent of schools in Manchester, New Hampshire from 1924 to 1938. And so what happened? If it was so great, where is it now?

So I might have mentioned that I actually went and did research on what happened to it. And I went to the Manchester, New Hampshire school board and just spent a whole day in their archives reading all the minutes of all the school board discussions about it and all the votes about Benezet and whether to reappoint him as superintendent or what to do about the curriculum. And basically, what happened was two things.

So first of all, there was extensive studies done showing that the curriculum was very successful. So one study was by-- so Etta Berman, she was a teacher in Manchester in the program. And then she did her master's in education I'm pretty sure at Boston University. Yes, I'm sure it is, because you can find her thesis in the Boston University library.

So her education thesis was an assessment of how the students did in that experimental program versus the regular program. And the students in the experimental program were just much better. It was just totally clear. So now what happened?

Well, what happened, partly, I think a lot of it is what Alfie Kohn article about "Not for my Kid: How Privileged Parents Undermine School Reform." So there was huge opposition from the privileged parents. And so I interviewed some of the people who had those privileged parents and who had taken that curriculum 50 or 60 years ago. And one of them was later mayor of Manchester, and he said his parents just hated the curriculum.

So generally, the people whose parents were foreign, for example, French Quebecois who had come to New Hampshire, had to work in the mills, the non-English speakers, the lower middle class, it was OK, and it was fine. And the ones I talked to from that background loved the curriculum. And then the few who really hated it were actually socially quite powerful, and they came from the predominately English speaking families. And they hated that he wasn't, for example, giving homework.

Because he said, if you have eight hours of school, what more homework do you need, which I think is actually true. If you can't teach people in eight hours a day, that's ridiculous. That's 40 hours a week. So why are you giving them still more homework?

So he didn't have homework, he had them actually learn in school, which is one of the benefits of interactive teaching. If your lecture is no longer dictation, you are actually learning, you don't have to do a lot of stuff out of school. So there was opposition, there.

And what happened was the votes in the school board went something like this. So this is t. So this is 1938 and 1924. And this is the votes in favor, minus against. They went roughly like this.

So the school board, I think it had 15 people on it, or 14 people. It depended, sometimes one person wasn't on the school board. So the first votes were something like 13 to 1, so in other words, plus 12. And then, over here, 1924, when he was first appointed, and then the votes went like this, and then it started going down like that.

And by 1938-- so there was votes taken every three or four years, or two years, whether to renew the appointment, because that was the term. And by here, the vote was 6 to 6 right here. Yeah, that's right-- it was 6 to 6 in 1937. And then what happened in 1938 was they didn't vote to not take him on, but what they did is they voted to use a standard textbook. So they voted to use a standard textbook in the curriculum, and then the curriculum was then declared incompatible with the standard textbook.

Of course, the standard textbook was pretty terrible. Well, I don't know the textbook, personally. But I'm sure, given the state of standard textbooks, then and now. And it certainly wasn't the progressive, understand the algorithms approach that Benezet had.

So then the textbook was instituted, and the curriculum was basically killed off, like that. So then Benezet left for a professorship of education at Dartmouth. And then, after Benezet left-- so this is Benezet-- Benezet left, the new superintendent put in a, basically, drill and kill curriculum with lots of testing and the standard now. There was No Child Left Behind 60 years before or 70 years before its time. But the curriculum was very successful, and it was basically killed off by opposition from wealthy parents and, to be fair, Benezet didn't help his case.

So a couple of the students told me that actually when the students wouldn't answer things in class, he would sometimes make fun of them.

[LAUGHTER]

PROFESSOR: Now that's terrible. I mean it's terrible just on moral grounds, but also it's terrible political strategy. Because now, the kids will tell their parents, and if the parents needed ammunition, at all, there they have it. And so Benezet was just arming the enemy too. Yeah?

AUDIENCE: If, presumably, most or all of us would be teaching college level courses, we don't have to worry about parents.

[LAUGHTER]

[INTERPOSING VOICES]

PROFESSOR: Good question.

AUDIENCE: [INAUDIBLE].

PROFESSOR: Hey.

AUDIENCE: Is it a comparable force to the parents [INAUDIBLE]?

PROFESSOR: Yes, there is. Good question. So is there a comparable force in college education to parents with privilege? Yeah, and unfortunately it's one's colleagues on the faculty. The reason is because they were the ones who did well by the old system.

And so now a change is likely to be misinterpreted or maybe rightly interpreted as well, this isn't for the benefit of the top 5%. So we can't do it. So for example, interactive teaching is often criticized as well, that's really useful for the people who aren't learning anything, but the top 5%, it will just slow them down. So we can't possibly do it. So the wars about interactive teaching turn on a lot of that question.

And that is a very similar force. Because people are saying well, it's those people who are just like me, because they're the ones who are going to become future faculty. And those ones aren't going to be benefited by the system. Now I think, empirically, that's not true. I think they are actually benefited, as well.

And Benezet found that too. It wasn't that some people did worse. They were all doing better. But it's the intensity of the response that you get to things sometimes in that vein shows that its an underlying social force and not just purely a cognitive reason.

Is that what you were thinking about?

AUDIENCE: [INAUDIBLE].

PROFESSOR: OK so 10:08, so 10:18, we'll start again with more questions.

[INTERPOSING VOICES]

PROFESSOR: So I know there are several more questions, because I already have two in the queue. Go ahead.

AUDIENCE: My question is--

PROFESSOR: And then [? Lortis. ?] Yeah?

AUDIENCE: What do you do when you're confronted with a question you don't know the answer to.

PROFESSOR: What do you-- I don't know? No, sorry.

[LAUGHTER]

PROFESSOR: No, no, no. The question was, what do you do when you're confronted with a question that you don't know the answer to? And my answer was I have no idea. No, there's good answers to that question, and hopefully, I'll give you one of them.

So I would say the number one thing is not to bullshit. That can only end-- at best, it won't end in disaster. But it can end in disaster.

The first thing-- so if you don't know the answer to it, but you have to think of what's going to get triggered in you? So if it's something, for example, really related to the material, you're more likely to get triggered and think-- [GASPS] Excuse me. Oh, I should have known the answer that, I must be a lousy teacher, how will they ever respect me again?

So if you remember the discussion we had last time in political barriers about the three levels at which conversations are carried on, so this was from difficult conversations, there's the factual level, the emotional level, and the meaning level. So the fact level is that someone just asked you a question, and you don't know the answer right away. The emotional reaction, and the emotional transmission, is oh my God, I feel worried and nervous.

And then the meaning level is oh, what does that mean for me? Oh, it means that I'm a bad teacher. They found me out. I'm really not supposed to be here.

[LAUGHTER]

PROFESSOR: So that same thing happens with students when you ask them a question, and they don't know. So you want to create structures where that bad meaning, isn't triggered. So you want to just pause, first of all. The reason is, if you don't pause, then you're more likely to act out, based on the emotion.

So I've seen this happen several times when I was a graduate student. So this is sometimes when the person didn't know, but in general, when they were somehow challenged by a question, so either way, whether it's because you don't know the answer, or because the student's tone has some kind of cheekiness to it, and you feel a bit of insolence, and then you think, I'm the teacher God dammit, how dare you talk to me like that? So that emotional reaction, or that challenge, that worry, that anxiety about your authority will produce a bad response, in general.

So if you just, right away, respond, you are likely to do bad things. And this is a thing I've seen happen a few times, is where you put the student down, you compete with the student, you say things like, well, how could you not know that? And you dismiss the question. So if you pause for just one or two seconds, even an instant after you practice it, just the pausing to realize oh, wait a minute, I just had an emotional reaction happen, I felt in my body, I got a bit sweaty, my palms sweated, whatever it may be that your particular reaction is. I feel it often in my face-- my face gets warm.

If you remember, way back when we were doing-- was it the wood blocks or the cones? No, it was the cones. I was dropping the cones. And there was a question when it was time for questions and discussion. One person said, oh, the answer is block, and I've seen you do it before, and I just remember just feeling flushed. I was like, this is not the time to answer that question.

So it's the same thing. So that's lesson number one is to pause. And then you can think and regroup and have a reflective response. The reflective response may be, oh, that's a really interesting question, I'd never thought about that, but I'm going to think about that tonight, and I'll tell you the next time. That's perfectly fine.

Another one is you can try to figure out the answer together. You say, well I'm not sure, but let's see if we can figure out the answer in the next minute or two. If not, I'll work on it in the evening or work on it tomorrow and tell you. And then you can try it together, and students have the advantage of seeing how you would reason about a question.

And it helps if you do that intentionally. Because there's a general rule of thumb which is that one's facility with equations declines with distance to the blackboard. So the closer to the blackboard, the lower the facility of the equation. So if you just leap to the blackboard and start writing, you're likely to feel too nervous and actually just mess it up just because of that.

So if you say, well, let's actually think about doing this together, think about what you might do, and think about some approaches, then go to the blackboard and maybe start working. And when you have your out, it doesn't work in two minutes, you say look, it's a good question. I clearly need to think about this some more, and I'll do that. And I'll come back to you. Does that help answer your question?

AUDIENCE: Yes, because we were just talking about just having these discussion with our students. And I feel like perhaps there would be somebody who would point out a different way of doing something. And then you aren't really prepared to talk about that with them. [INAUDIBLE].

PROFESSOR: Right. So if you're going to do interactive teaching, you're going to have lots of discussion. So people think, oh, interactive teaching means you just let the students do everything, you kick back. No, it's actually much harder. Because you've changed it back from basically playing a prerecorded tape, which is what a lot of lecturing is-- you just get out and read the book, that's what a lot of my lecturers did-- to now it's a stage performance-- a stage performance in the old days of Elizabethan theater where the audience interacted with the stage.

So that's much harder. You have to, quote, "Know your lines." Not that you're going to say the same lines no matter what-- you have to really feel the lines. You have to really understand the feel. So it is harder.

And you're going to be producing lots of discussions. And at a place like MIT, the students are very curious. And they're going to think, oh, well, what about this? Well, that's fine.

So you want to use Aikido. So if someone suggests something you haven't thought of, don't dismiss it. You say, oh, I hadn't thought of that, that's a really good suggestion, which then, usually, maybe, I'll use that next year when I teach the course, thank you. So if you just think about the students as your allies, then your reaction that you'll intrinsically produce will be much less confrontational and much less worried.

So I'll give you one example of where I was very, very nervous and what I did. So I was lecturing thermodynamics, and there was 200 students. And I was doing the Carnot cycle, so this was the last lecture. So we were doing the Carnot cycle, and I drew it up on the board. And I'd encouraged questions throughout the whole lecture, throughout the whole lecture course.

So, of course, someone raised their hand and said, oh, excuse me, I think that this box is not right. They didn't know what was wrong, but they said, oh, that box isn't right. So I looked at it, and I thought, oh my God, they're right. So I'd now just drawn rubbish on the board for 200 people, and I had a bunch of my colleagues that day who happened to decide that was the day to come visit my lecture.

[LAUGHTER]

PROFESSOR: So I think that was probably why I was a bit nervous to begin with. And that probably contributed to me writing down the wrong diagram. So I realized I needed some time to actually think, so I just said, you know, you're right. Let me just think. And I just turned my back, for a whole minute, and I sorted out what was wrong. And then everything was fine after that.

But I still remember that. I still remember the feeling in my body, just that sinking feeling of oh my God, disaster has happened. I've just been exposed. And really, I should've been one of the students in the class. So pausing is immensely valuable. [? Lordis? ?]

AUDIENCE: Follow up to the question that was said before, how do you effectively argue against [INAUDIBLE] to the top students?

PROFESSOR: Right, so often when you want to do so something more interactive in your teaching, the counter arguments are well, you're going to harm the top 5%. Well, one counter argument-- so first of all, it doesn't often help to counter argue cognitive reasons if there's an emotional underpinning. So generally, I try to step aside from those arguments, unless the people have a sense of humor, I find. If the people have a sense of humor, then you can present things in a humorous way that challenge their view, and they can take it on board. But if they don't have a sense of humor, or you don't have a good way of presenting it that way, I find it's just wiser to step aside and see if you can find some way around it.

Maybe try half of what you wanted to do, or say, well, yeah, I worry about that too, a bit. So let's leave the main course alone that way. Let's try an experiment and see how it goes. Maybe we can agree on some questions that all the students should be able to do to try to make it cooperative. So that's one thing.

But if you do want to go directly into cognitive discussions and reasons, one is, and this has scope for making humor out of, [? I've ?] said, what's a top student? The top students, the top, quote, "5%" are the top 5% by the current system of testing and homeworks. So they're are often the top 5% in regurgitating canned formulas. Or by the time they get higher and higher and more selections happened, they're good at pattern matching and solving problems they don't even understand.

I mean certainly, the final exam in Cambridge that people took in their final year-- if I just sat down, I would not have passed it in the physics department. And so there were students who did really, really well on it. But I'm sure I understood more physics than them. And it's because if you train for those kind of questions, even if you don't really understand what you're doing, you can actually answer them.

So now the top 5% at that, well is that so valuable to preserve? Maybe actually we should change the rules of the game. So now, if you just say it as directly as I've said it, then you'll trigger the meaning of-- well, because a lot of times, those 5% became the people you're talking to. So you have to maybe do it slightly differently, and say, look, what is the thing you're most concerned about, about the students not knowing, whether they're the top 5% or not?

And a lot of times, they'll just give you a big, long list. On their own, before you even say it, they'll say, oh, they can never use anything outside of the class. So people will just outright concede that, not even concede it, they'll say, yeah, it's so terrible. And people will talk about this all the time, and say, look, we all agree that. Well, let's see, is there a way that we can make people really good at that?

Because even the top 5%, I find, are not like that. So now you've opened a door. And you can have a shared discussion based on something like that. So that's how you can reach the 80% who are not sure but are willing to listen. Yeah?

AUDIENCE: I was wondering what you think of the surveys that they have at MIT by which the students evaluate the professors and teachers, and also, how to interpret the results of those?

PROFESSOR: Oh good, I'm glad you said that for several reasons. So one of the other reasons is that this course has one too.

[LAUGHTER]

PROFESSOR: So I'll put it on the course website just after class. So 595 is one of the course numbers. But it's also 6.92 or something, so all course six classes have an online evaluation system. So for everybody, we're doing it that way. So everybody who is registered for the class is listed as a possible survey filler-outer. So I encourage everyone-- I'll put the link on the website. Just click on the link, and if you have your MIT certificates installed, it'll just take you to the online survey.

So I think the surveys are very useful. Now the numbers are a ballpark-- well, were things working or not? But I think the most useful part of the surveys is the comments people make. So again, I encourage you to make comments in the comment boxes, wherever they are.

For example, there's one that says, what would you suggest for people next year? So part of that I'm going to learn by reading these. So I'll crib good ideas from how you would do this course from your homeworks. But the survey will actually help with that.

So I think the surveys are really useful, if used properly. I don't think they're very useful for saying, you get tenure, you don't get tenure. Because people just use the numbers for that. I don't think the numbers are super reliable. I mean seven is probably significantly better than four or five, but there are a lot of ways to get six and a half, and six or five.

How did that happen? Well, you want to really look into the course and see that. And the course survey doesn't really address that super well. But the comments give you space, if you really listen closely, to understand it. So I think they're quite useful.

But what I'd really like to see-- because those are surveys right after the class, and my goal, what I've been trying to stress this whole time, is how can you construct teaching for a long lasting learning, questioning and reflecting, ideally for a long lasting learning? Well, if it really lasts long, people should remember something a year later. So I think actually the policy should be changed to do feedback sheets at the end of each lecture, so you get the quick feedback you need as the lecturer. And then by the end of this course, you have a lot of feedback, and you can really improve things next time.

And then do the survey a year after the course is finished. And if people don't remember anything from the course right away, that's a pretty important piece of feedback. So I fear though that the results from that kind of survey would be quite distressing. Because you'd ask things like, well, which of the ideas from the course do you use?

And people have done surveys like that in physics majors. So they're surveys of physics majors in the job, whether their in research or in industry or finance-- well, before that ended-- what do you use from your physics degree? And very rarely do you find people saying, if you just average across all the jobs, Maxwell's equations. It's just not on the list-- is very low on the list.

It's things like general problem solving, quantitative skills, working in groups. Now it's not necessarily true that therefore we should just dump all our classes and teach people how to work in groups. That's not what I'm saying. But it's useful to know what people are doing with the things you teach them, so that you can take that into account.

So that would be my ideal-- surveys one year, five years, 10 years, later. And there are a few of those at MIT, alumni surveys. [? Meki, ?] I know, has done a few of them. Other questions? Yes?

AUDIENCE: So let's say you're a new teacher, and you're giving a course. And you design it, and you really think about how you want to do it, and the labs and the homeworks, et cetera. And you refine that over a number of years.

PROFESSOR: Yeah, two or three years.

AUDIENCE: So five or six years, you've got it down pretty well. Does it become boring?

PROFESSOR: Good question. Yeah, so the question is you make a new course, two or three years you've refined it, after five years, does it become boring? My guess is probably yeah, unless you keep changing it.

So I've taught Art of Approximation for almost nine years now. And every three years or four years, I completely figure out a new way of doing it. And now it's finally converging, I would say. I've finally realized, oh the main thing-- because I didn't know this when I first started, because I was a new teacher-- so to organize it around large ideas, main themes, transferable technique. So I've finally converged to that.

And now I'm changing the examples. And so I would say the [INAUDIBLE] is probably converging. But I could probably keep doing that for another three or four years. But if you just do it, refine it for two or three years, and then just keep doing it, I would say yeah, after another couple years, it's not necessarily that it will get boring, because when you teach this way, the teaching always stays interesting, but it's that you find yourself maybe just thinking about other courses.

And I already have a sign that that's happening, because I'm now thinking, I'd like to make a Physics of Music class. And they may be related to the other class. Because we just did the wood blocks and the xylophone in my approximation class. I'm thinking, oh, a whole class of the physics of music would be really a lot of fun to make-- organize it around demonstrations, and then you could bring in the physics, as needed, for the demonstrations, and you choose all the demonstrations so that all the main physical ideas in acoustics, and electromagnetism, and mechanics, and sound are all combined. Oh that'll be a really nice design problem.

So maybe I'm converging with the Art of Approximation one after nine years. So it depends how often you keep changing things. But yeah, and that's OK, because you'll find that that's the natural life cycle.

So for example, if you're a faculty member say in almost any university in the country, after six years, you get a sabbatical. So in the sabbatical, you'd be very well advised-- I mean, you could teach if you wanted-- but you're very well advised to not show up on any committees, not teach, and generally, you're pretty much required not to and then just leave. Because otherwise, you'll be dealing with all the normal, run of the mill things. You wouldn't be able to have a block of time to really think.

So if you've developed this course, and it's your course, after five or six years, it's automatically going to have to be given to somebody else. And that's fine. Because that's around the time that you'll be thinking about other stuff. And the sabbatical is a time to reorient yourself towards other stuff.

So looking ahead towards that, what you want to do is write up what you do. So the first year, it's pretty hard, you're just barely scrambling, you're trying to breathe. It's like having four children, or us having quadruplets. Well, maybe, I don't know, I haven't had that happen to me.

[LAUGHTER]

PROFESSOR: But the first year you do a brand new course, especially a three days a week one, it's pretty hard. So I can tell you, this is two hours a week, but it's the first time I've taught this course. So it's a lot of work. And that will be true. So the first year, yeah, you just want to survive.

The second year, you start to figure out what to do, what to write, and just start writing stuff up bit by bit-- lecture notes, problem sets that are type set, things that other people can use pretty easily. So then when the course is handed off to somebody else, they can continue that and refine it, based on what you've done. And everything you did doesn't just vanish. Yeah?

AUDIENCE: So you mentioned in classes you really like to [? do a lot of ?] teaching and [? carve ?] the main ideas and then put some of the mechanistic--

PROFESSOR: The grunge.

AUDIENCE: [INAUDIBLE]. So do you have anything illuminating to say about how to choose those textbook [? readings, ?] other than [INAUDIBLE]?

PROFESSOR: Yeah, so the question is-- so I like to use lecture time for the larger ideas, for the concepts, for the chunks. So for example, in the teaching equation one, and leave the derivations and detailed manipulations, the mechanics of stuff, the grunge let's say, for the textbook. Do I have any suggestions for how to choose a good textbook?

Well, it is important to choose a good textbook. But it becomes less important if the classroom is focusing on concepts in chunks. Because that's the differentiator, generally, between the good and the bad textbooks, is that the good textbooks do more of that. So if you're focusing on that in the classroom, then you have more freedom-- not freedom to use bad textbooks, but the bad textbooks don't do you as much harm. Because the students don't need so much out of the textbook.

They need to basically have a place where the derivations are correct. And that's an important part, and it has the derivations you need, sure. So right away, you can relax about choosing textbooks, slightly.

But how do you find good ones? Actually now, I look at the reviews online and see what students say about them. So I do a web search for books, and I see what books people are using, what people are commenting on, look at the online bookstores, the various reviews of the books. That helps a lot.

And then, nowadays, often people put the first chapter and the Table of Contents online-- the publishers do. And ideally, they would put the whole book online. Some are doing that too. So have a look at that.

From the Table of Contents, you can really tell a lot about a textbook. You can say, is this organized right? Does this have the right philosophy that I'm looking for? So look, right away, at the preface, where the author talks about why this book. And if they have no good reason for why this book, it's probably not the book for you.

Sometimes you don't have a choice about the book. People just say, well, this is the book that we always use. Or it's the first time you've taught the course. For example, you're hired say in March, you start your job in July, and in August, they come to you and say, oh yeah, by the way, Professor Blah is on sabbatical teaching X, you're up, the course book is already there. So you can't do much about that.

So sometimes you don't have freedom, but then when you do, you want to do it. So look at the preface and the online reviews. They're quite helpful. But don't worry as much if your classroom is interactive. That mitigates a lot of textbook problems. Yeah?

AUDIENCE: So you [? focused ?] on the fact that in Europe, in your undegrad, you take more specific classes towards [INAUDIBLE].

PROFESSOR: Mm-hmm.

AUDIENCE: And I wonder actually if there is some [? standard ?] or analysis of what is more effective. Because in general, I goes that if you have more variety, than you have a higher tendency to do more disciplinary work. [INAUDIBLE] that would be the advantage. But the advantage of having taken only chemistry, math, and physics during your undergrad is that then you know a lot about that.

PROFESSOR: Right.

AUDIENCE: And that also goes a little bit-- I don't know, I keep hearing, although I have never seen proof that math skills in the US are not as good as the people who come from outside. And also, [? because ?] we were talking about doing a class on music and--

PROFESSOR: Physics and music, yeah.

AUDIENCE: And then I wonder if I'll ever get an undergrad that took that class instead of taking a physics hardcore math class. And I'd be like, oh, I wish you had taken this class that would give you the tools to work, rather than some class that you would enjoy, but that I'd rather have you taking, later on, at the end of your grad school.

PROFESSOR: OK, so let me unpack those. So there were several questions in there. So first, are there any studies on the difference between the results produced by the European's, say, educational system versus the American, or the European where people just do their one subject to first order, where the American one where you take courses across many different subjects? I'm not sure about that. But also, the notion of effectiveness is not clear, so what's the goal?

So that the American one, I think the historical reasons why it's different is that education in America, early on, back in colonial times, and just post-colonial or the early 1800s, was seen as a democratic thing. The idea was that this is a democracy, and all the citizens are expected to participate in that. And so they had a narrow definition of citizens-- just white males-- but within that definition, the idea was that everybody had to participate or was going to participate.

So to that end, they needed a broad education. They couldn't just specialize in one thing. Because they would be too narrow to actually help govern society. Because society is multi-farious and multi-disciplinary, intrinsically.

Whereas in Europe, that idea of democracy came hundreds of years later, if at all, maybe at the end of World War I. So the European educational system was much more oriented towards producing people who would fit into a particular slot in society. So now if you want to measure how effective each was at its historical goals, I think that's a fair question. To measure how effective each is at producing people who are trained for a particular slot in society-- I don't think that's a very useful analysis. Because the historical origins of the two systems are different.

It's much more fruitful, I would say, to think about the goals. Which goals do we agree with? Can we get the benefits of the American goal with some of the benefits of the specialization in Europe? And I think at places like MIT, you do do that. The students who come out of MIT-- they have a broader education than the students who come out of European universities, and they actually know a lot.

Now I would like it to be much more conceptual and long lasting. But that's true of the European ones too. So I would take the question back from just studying the end result to think carefully, as a society, about their purposes.

Now about the Physics of Music, I would say, actually, I could imagine teaching an intro physics course that's Physics of Music, where students would learn all the fundamental ideas of say first semester physics, but through the physics of music. And so they wouldn't be deprived of the essential knowledge of physics. They would actually be contextualized. So it might have longer lasting value.

AUDIENCE: [INAUDIBLE].

PROFESSOR: OK, other-- yes?

AUDIENCE: So you talk about emotional barriers, and I think one of the big ones that I've seen, especially in students at MIT, through TA'ing or just through hearing people talk is I don't belong here. I must have gotten here by accident.

PROFESSOR: Yeah.

AUDIENCE: And I think so many people say that to themselves, at one time or another, or multiple times while they're here, that it can't possibly be true, because everybody is saying it.

PROFESSOR: Right.

AUDIENCE: Because if they have that barrier up, I don't think there's much to do to combat that. So how do you get through that kind of barrier? Because no matter how clear your teaching is, if they're worried that they're not going to get it, because they don't belong--

PROFESSOR: Yeah, if they're in the math phobic mode or phobic mode.

AUDIENCE: Yeah.

PROFESSOR: So this is a really interesting question. So the comment was that a lot of people at MIT, a lot of students, one of the emotional barriers they have is they feel-- I would call that a meaning barrier. Let me rephrase it in the three levels of conversation.

So their emotion is that they're very tense and anxious. And the meaning is that when something doesn't go right for them, they get something wrong on a problem set or in class, they think that that's exposing how they really don't belong here. And that creates a lot of anxiety and tension and misery.

So at Caltech, there was a saying that, oh yeah, Caltech is a terrible place for the bottom half of the 99th percentile, which captures that pretty well. All these people, all the undergrads who went to Caltech-- they were probably valedictorians in their high school and two sigmas above in doing science fair projects than everybody else around them. And now they come to Caltech, and they feel terrible. What are we doing that produces that? It's crazy.

So how can your teaching-- can you take account of that? So that, I would say, also relates partly to the misconceptions. They have this misconception or this conception about themselves. So what can you do?

Well phobias are very difficult. You have to work around them. So one thing is, you don't want to trigger them. So every time you trigger them, you reinforce them.

So any neurobiology people in here? I forget. [? PCS? ?] Well the fundamental rule of neurobiology of brain neurons-- so this is-- of course, I teach Art of Approximation, so forgive me for approximating the whole brain with one sentence-- is neurons that fire together wire together.

So wiring together, that means they activate, they increase their coupling. And so for example, you create a classroom environment or something happened in the classroom that they get something wrong. And now that is still wired, somewhat, to the bad feeling and their thoughts of oh my God, I don't belong here. So now you've reinforced that, they're going to wire together more. It reinforces that pathway.

So what you have to do is find a way to actually prevent those guys from wiring together. One thing is create an environment where whenever people get things wrong, it's actually great. So oh, that wasn't right, great, now think how much you're going to learn. Or when you get things wrong, you want to model that too. You say, oh you're right, oh thanks, I've learned something from that.

So try not to be defensive. Because if you're defensive, then that triggers that wiring in them. And that firing together builds up the wiring together.

So creating a classroom environment where feeling bad and competing against other people isn't the goal is one way to minimize it. So minimizing the importance of defined distinctions on homework-- did I get a 95? Or oh, that person's getting a 95, I only got a 90.

So that's one reason MIT was pass/fail. For a whole year, originally, was pass/fail. And that was to mitigate the pressure that students were coming in, feeling like, oh my God, now I don't belong here. [? It was ?] look, we don't care about your grades in the first year. You're really here to adjust to university, to a different way of learning, and now, it was actually done in a way that somehow the competition was partly the students reinforcing the culture, the upperclassmen, as well.

If you could actually try to mitigate that, you'd go a long way towards mitigating the wiring together. But you can do it in your classroom too by deemphasizing the things that trigger that. And so when you call on people-- so for example, for that reason at MIT, I almost never what's called cold calling.

If you've ever seen that old movie The Paper Chase-- so The Paper Chase is about the what we could call today the terroristic Socratic method. So the law schools all use the Socratic method, where they read a case, and you ask them about things-- things about it. Mr. Jones, what do you think, what were the findings of law? Mr. Smith or Mrs. d'Arbeloff, what do you think about that, do you agree with that, or is that pure rubbish? So people were just put on the spot repeatedly, and they just produced terror in the students, and that was considered part of the hazing ritual, basically.

So at MIT, I just don't do that at all. Because I think it just triggers that. And I'd rather just create a structure whereby students feel ready to participate, by, for example, in the interactive teaching, giving them time to think to each other and then say something. So you can do stuff, and it's hard, and it's an important problem. Yeah? Sure.

AUDIENCE: It's worthwile to actually confront it head on, is just say, a lot of you may be feeling this way, and please know that it's normal, and in fact, I felt this way most of my life. Because I had a professor in my undergrad who actually came in and said to a group of us-- like, it was in a meeting. And she said, how many of you have felt like, basically, you're an imposter and it's just a matter of time [INAUDIBLE] discovers you? [INAUDIBLE] all of us raised our hands. And she said, I wanted you to know that it wasn't until I won my fourth major award in the field that I stopped feeling that way. So It's normal.

And that's run in my head for years. So even someone who was amazing as she was, and she was an amazing individual, has felt that way, and to know that you're not the only one [INAUDIBLE]. So do you think it's actually worth saying?

PROFESSOR: Yes, that's a really good point. So the point was that it's worthwhile making it explicit for the students and really naming the feeling that they're going through. Like look, you may be feeling really terrible and feeling like you're an imposter. Well, let it be known that I felt the same way too for a long time, and describe it to them, and talk to them about why they may be feeling that. Say look, there's a culture of competition in the society and the university, and it's actually harmful for your learning.

And share with them some readings about that. Help them see that yeah, this is something they're going to have to struggle with. But there's help to do that, and they're not alone in that. So yeah, definitely be explicit about things like that. I think more question, yeah?

AUDIENCE: Just in a follow-up, do you think that contextualizing the course is a way to fight that?

PROFESSOR: Good question. Do I think conceptualizing the course is a way to fight that? I think it is. Because when the course is not contextualized, it generally tends to be really abstractly focused. And the interest and the orientation towards doing things first abstractly is very rare. I don't have it, and I've gone really far in math and physics.

My office mate had it in graduate school. He was the only person I've ever met who would rather have a proof than a picture. And now he's a professor at Caltech, and he's also doing well. But it's pretty rare.

So by contextualizing the course, for almost everybody, actually I would say even for my friend and office mate, you're actually connecting to a much larger part of their mind and their experience. So you're actually making people much more intelligent and much more equal in that way. So that's one reason I would actually like to teach the Physics in Music class.

AUDIENCE: It made me think [INAUDIBLE] they teach drugs in the brain [INAUDIBLE] biology. So if you're a physicist and you're like, I can't do biology, you can take drugs in the brain [INAUDIBLE]. And they teach you biology by teaching [INAUDIBLE].

PROFESSOR: Interesting. Yeah, so it somehow gets around the phobia you might have.

AUDIENCE: [INAUDIBLE].

PROFESSOR: Pardon?

AUDIENCE: They do that at [INAUDIBLE].

PROFESSOR: Interesting. So I was there 10 years ago, and that wasn't true then. So the comment was that at Caltech now the required classes all have a contextual version, as well. And I think that's a great idea. It's also, from your point of view, it's really fun as a future teacher, because you get to think of all the interesting contexts in which your field is useful. It connects you back to the reason you did the field, originally.

And that's good to remember. Because you'll be going along, and you'll be absorbed in your research, trying to get tenure, whatever it may be, and there will be all these layers that are created on top, and you'll forget why you came into the field to begin with. And if you can capture that feeling, actually if you can connect to that feeling in you, then you'll actually be able to transmit that feeling to your students. And they'll have it, as well, and you'll actually be doing good for the field, good for you, and good for your students.

So with that, I'll wish you lots of success in your teaching. I hope that the principles and examples that we've done throughout the term you find useful, at least in one way. And I look forward to hearing from you later how you use or not use or any suggestions that you may have. So good luck.

[APPLAUSE]