The following content is provided under a Creative
Commons license.
Your support will help MIT OpenCourseWare continue to
offer high quality educational resources for free.
To make a donation or view additional materials from
hundreds of MIT courses, visit MIT OpenCourseWare at
ocw.mit.edu.
PROFESSOR: OK, welcome everyone.
It's 8:05, well, 9:05.
So today's lecture and discussion is about how to
make good problems, whether it's on homework problems,
exams, even problems in class.
Its general principles that you can use constructing
questions that you ask, so problems broadly conceived.
Now why problems?
So problems are, I think, one of the central ways of
thinking about teaching.
And I'll show you how problems fit into the grand scheme of
this course, of rethinking all the aspects of teaching.
So one way that I like to think about teaching is called
backward design.
So this is a way of designing a whole course, could even be
for a curriculum.
And doing problems is one aspect of that.
So why is it called backward design?
Well, it's backward from the usual way.
So I'll show you what backward is and then show you what the
usual way is.
So in backward design the highest-level thing you figure
out-- not necessarily the first thing you figure out,
but the highest level thing-- is your course goals.
And that then feeds to OK, if those are your course goals
how do you know whether students have reached them?
And that's problems.
Problems, tasks, projects, whatever it may be, things
that students are going to do.
And so generally speaking, things to do.
And then given that you've operationalized the goals in
these problems what are you going to do in class, so that
students actually are able to do these kind of things?
OK, so that is backward design because the
usual way is the following.
You say, OK, well, you have a bunch of lecture notes.
Where do you have those?
Well, either the last person who taught the course gives
you their lecture notes.
Or you just take the standard book in the subject and you
just do the lectures one per chapter or one week per
chapter, something like that.
And you just go through in the order.
So in the usual way you start with this.
And then you think oh, whoops, oh yeah, I've got to make some
problem sets.
And oh, some exams.
Well, what have we done this last couple weeks?
OK, that's what we're going to do.
So the usual way is you start with here.
And then you get here.
And you somehow never get here.
The course goals are, basically, to
get through the book.
Right?
That's the implicit course goal.
But it's never explicitly planned.
So that's a terrible way, but the normal way
of designing a course.
So this is backward design.
So this is due to Wiggins and McTighe.
OK, so here we're going to be discussing this today.
So we're doing this first.
And then the next session is this one.
And I think the session after that is, what do you do in
class, interactive teaching and things like that.
So why am I doing it in this order?
Well, in the levels of abstraction these
are the most abstract.
And this is the most concrete, OK, what question I'm going to
ask students in class now.
And this is things like goals.
I want them to really understand conservation.
They're really high level.
So I think if you start here, by trying to figure out course
goals, I haven't found that works very well for me because
it's too high level.
I can't think about course goals in absence of actual
problems and things I want students to be able to do.
So this has the right level between concrete and abstract.
So it's actually where I start my thinking.
Now, you may actually find you want to start somewhere else
on the continuum.
But this is what I found works quite well.
You start here, you operationalize what your goals
are in the problems that you think are interesting.
And then you look at the problems.
And that sort of gives you an idea of the course goals.
And then you think about the course goals.
And that helps you think of problems.
But I start here.
And then, once I have things like that, then class time is
actually much easier to plan.
Because class often is going to be problems like this,
related to this, maybe shorter.
But it's easier to plan here after that.
Whereas we start here, you don't know, you're
back in the old way.
So I don't want to start here.
This is too abstract, so I start here.
So that's why we're doing this now.
And then next week we're going to do this one.
Question?
AUDIENCE: So when you say that you're setting
goals for your class--
PROFESSOR: Pardon?
AUDIENCE: So if you say that, basically, setting the goals
for your class, or for your own course.
So when you design your course, basically you write
out some kind of goals you want to reach?
So how do you then do the problem?
Do you go in detail before you've even
started the whole class?
Or do you do it that on a week-to-week basis?
PROFESSOR: Oh, what do I tell the students
about the course goals?
AUDIENCE: Yeah, I mean, if you say, OK, basically, you have
your goals and then you design problems so you
can reach the goals?
But then do you also design these problems before you even
start the whole lecture?
The whole semester?
PROFESSOR: Oh, well, it's pretty rough.
I mean, usually the first time you teach the thing it's too
abstract to design--
I find--
everything in advance.
Maybe that's my extrovert nature, which is that I find
it hard to design things for people I don't know.
If I don't know the audience I don't even know what to say.
So I want to actually teach the course one time at least.
So then the audience becomes concrete for me.
So really, the first time through the course nothing is
ever ideal.
And you're doing everything on the fly.
But the second time through the course, you can actually,
basically, make all the problems ahead of time.
So the second time through the course you actually refine
your goals and your problems together.
So that shows you the structure of where we are and
where we're going to go and then later.
So this is two weeks from now.
Today, here.
So how do you make problems?
Well, the fundamental principle is, again, deduced
by induction from what happens with regular problems.
So if you remember, I gave you an example earlier of the
results of standard problem solving.
And I'll just remind you that we're putting up the
percentages again.
OK, so after doing lots and lots of standard
multiplication problems, students were--
if you remember-- unable to do the following problem.
So these 13-year-Olds and 17-year-Olds were asked to
estimate 3.04 times 5.3.
So here were the answers.
So they were given four possible answers.
And they could also not answer.
So this is no answer.
So that's the age 13.
So of the four possible answers--
or let's say no answer is possible as well-- so five,
they're 1% over 1/5 will get the correct answer.
So that's a bit depressing.
So you think, oh, OK, the 17-year-olds
are going to be better.
And they are, but they only have 37% correct.
OK, so here is a serious problem,
which is rote learning.
And that's the fundamental thing to
avoid when doing problems.
So these students-- especially the 17-year-olds--
were actually perfectly capable of doing exactly
multiplication.
So on the same test they were asked questions like multiply
2.7 by 8.32 and given just empty space on the page to
multiply it out.
About 80% did it right.
So it's not that they don't know how to multiply.
They just don't know what multiplication means, which is
a far more serious problem.
I'd rather they knew what multiplication meant and
didn't really understand, didn't
really know how to multiply.
Because the algorithm you can teach them later if they
understand what it means.
But if they don't even understand what it means that
basically it's 3 times 5, so it's got to be somewhere
around 16, they don't understand what multiplication
means or the number system means.
That's a far more serious problem.
And that's produced by the traditional kind of problems.
The traditional kinds of problems are, generally
speaking, too low level and can be solved,
to easily, by rote.
Even the complicated traditional problems can
eventually be turned into rote learning,
and that's the danger.
So the main goal in making good problems is--
I would say--
how to make problems that fight rote learning.
OK?
That produce long-lasting actual conceptual
understanding along with whatever else you're trying to
produce and teach.
But really, fight the rote learning because that is the
fundamental problem.
So a way of thinking about this is to categorize the
levels of thinking that you're expecting of students when
you're asking them to do a problem.
And for that there is work from 50 years ago now--
which is still useful--
which is Bloom's Taxonomy.
So Bloom's Taxonomy--
let's see, when was it?
I think it was 1956.
I think it was '56.
I'll put a handout on the website for you which
summarizes in one page.
But I'll summarize it even shorter here.
So what is it a taxonomy of?
Well, when you read the title the book, it's Bloom's
Taxonomy of Educational Objectives.
And it's not really clear if that's going to be useful for
anything at all.
Because you think, what the hell is that?
So the shortest way of understanding it is that it's
six levels.
And each level is higher than the one previous to it in
terms of the level of thought required of a student.
So it's not really a strict order.
You can have really hard questions that seem to be low
level just because they address misconceptions.
But generally speaking, it's a rough order.
So the order is knowledge questions.
Those are things like define, name, state, list, You know,
it's basically almost rote.
I mean, these are the kind of problems that everyone would
agree are rote, state the definition of
Newton's Second Law.
The second is comprehension.
So that's sort of like knowledge but
it's a higher level.
So that's being able to grasp the meaning of the material,
not just recite the facts.
But generally, that doesn't include the idea of being able
to transfer it to new situations.
That's where application comes.
So can you apply the material to something?
For example, often compute, demonstrate, show that, those
are typical verbs that are used when you
have application questions.
The next level is analysis.
So that's really trying to understand the structure of
the material.
So things like diagram, differentiate, infer, outline,
point out, identify, distinguish, discriminate, all
those kinds of verbs com into analysis kinds of questions.
Synthesis--
so this is breaking things apart.
This is the ability to put things back
together in new ways.
OK?
So design, explain, categorize, create, revise,
rewrite, reorganize, devise.
And the highest level is evaluation.
So judging the value of material, for what purposes is
this good versus that good?
Explain, contrast, summarize, support, recommend, evaluate,
criticize, appraise, so generally
quite high-level skills.
So this is the taxonomy and this is a higher level.
OK, so a useful exercise--
whenever you're thinking of problems-- is to try to place
them on this.
Now it's not that you want no-knowledge kind of questions
and you want only to ask these kind of questions.
Because if you just ask these kind of questions people will
not be ready for them.
And they're not prepared.
It doesn't actually teach anything.
You want a mix of them.
But generally speaking, what happens to often, is that most
of the questions that happen are usually in this zone,
maybe a few of those.
And then only later in people's careers do
they get over here.
For example, in engineering curricula you learn a ton and
ton of application, maybe comprehension, Maxwell's
equations, differentiation, integration.
Then we start to synthesize stuff.
You know, in your senior year you do
design and project classes.
But you don't have to wait all the way until then.
You can actually start putting stuff in much earlier.
And it doesn't have to be an entire class.
You can have tasks that are like that in a short time, on
a problem set in the middle of lecture time.
OK?
So questions about Bloom's Taxonomy?
OK, so what I'll do, is I'll give you an example of
questions that in freshman physics mechanics--
on the topic of Newton's laws--
that fit into each of those levels.
OK, so just to give you an example, to make it concrete,
to see what those kinds of questions are.
And some of the categorizations I'm going to
give you are debatable.
But the rough idea is that they go from high to low.
So this is on the subject of Newton's second and third law.
So a knowledge question, state Newton's second and third law.
Newton's second law, state Newton's third law, so you can
see, just because people can state it doesn't mean they can
actually do anything with it.
That's why this is way up just at the knowledge end.
Computers are actually really good at this.
Right?
You can look it up on Google.
But your computer can tell you what Newton's second and third
law are but you can't actually do anything with them.
So comprehension--
so give an example of Newton's Second Law, using Newton's
Second Law.
OK, this next one is an application.
It turns out to be, actually, quite a hard application.
But it's still an application.
So it's to apply Newton's second and third law to show
the following, which the following is suppose I have a
piece of chalk in my hand and I stand on a weighing scale.
I want the students to show that the weight that the scale
reads is equal to the sum of the two individual weights.
So I weigh myself.
And then I put the chalk on the scale.
And I add up those two weights.
And that should be the same as when I hold
the chalk in my hand.
OK?
So I'll diagram that as--
OK, so that's me plus chalk equals me plus chalk.
So this is a scale.
OK?
So there is the piece of chalk.
So now this is an example of where that's quite a hard
application question, even though it's kind of high up on
the Bloom's list, so towards the lower end.
Because this is a serious misconception
that students have.
Newton's second and Newton's third law, people are
completely confused about, you'll find.
They don't know when to use which.
And they think any time two forces are equal it's sort of
50-50, whether it's because of Newton's third law or Newton's
second law.
So this-- if you're addressing misconceptions--
makes the problem, actually, all the much harder.
So at analysis--
so the analysis comes out-- oh, question sorry.
AUDIENCE: What are Newton's second and third laws?
PROFESSOR: Ah, fair question.
So Newton's Second Law says force equals mass times
acceleration.
Third law is--
the way it's usually stated, which it's terrible--
action equals reaction.
Or for every action there's an equal and opposite reaction.
Oh, which reminds me, one of the questions from before,
which is what name would I give?
Because I said I slagged off the reaction force as the name
and the normal force as a name.
People ask me, what should you call it?
And I actually thought about that.
And I think the answer is contact force.
Because that tells you what the thing is.
It's a force of contact.
So Newton's third law says action equals reaction.
So now, the problem is that students have, is that
whenever they see two forces are equal they assume it's
because the action equals reaction.
Or they think Newton's third law is the reason for it.
So for example, the bouncing-ball question that we
talked about last time.
So in the bouncing ball the question is, when it's
stationary on the ground what are the forces on it?
And they'll often say, well, there's a weight downwards.
And there's the force upwards.
And they'll often say, well, the force upwards equals the
weight downwards.
These are both forces on the ball because
action equals reaction.
And they won't realize that action and reaction have to be
on different objects.
OK?
So they're fundamentally confused about the meaning of
action and reaction, which you can bring out with the
following analysis question--
When two forces are equal in magnitude how do you tell if
it's Newton's two or Newton three?
So how do you know if it's Newton two or Newton three?
So just ask them that.
Yes, question?
AUDIENCE: This might be a little bit off subject, but
should we use Newton's third law ever?
It seems like it's a natural consequence from
saying a equals 0.
PROFESSOR: Ah, oh, OK, interesting.
So OK, let me actually explain that one.
I'll put it over there because I didn't do the
boards right over here.
So that is exactly the issue brought out by this question.
And let me show you the difference.
And then there was another question.
That reminds me of another whole set of questions from
the sheets, which was how do you develop
your physics intuition?
So let me answer that in a moment.
After this and then we'll have a break.
OK, so the bouncing-ball question--
actually let's not do the bouncing ball.
Let's just do an object sitting on the ground.
Oh, great, can you pass them around?
Oh, Leeann's just coming around with a handout on
Bloom's Taxonomy.
And let's see, I'll do these after the break.
OK, suppose you have this object sitting on the ground
and it has Mass m.
OK, let's look at what the forces on it are.
Well, there's mg down--
and actually, in this case, the contact force, our normal
force is equal to mg.
OK, so now the question is, are these two equal because of
Newton's Second Law or Newton's third law?
That's what this question asks.
So it's actually worth understanding
the difference here.
So these guys are equal because a equals 0.
So this thing's just sitting on the ground, a equals 0.
So the net force is 0, so these two
forces have to cancel.
OK, so that's Newton's Second Law.
So then the question is, where does Newton's Third Law show
up in this?
Ah, Newton's Third Law shows up here.
Here is the Earth.
And here is your object.
Let's just separate them just so you can see them separate,
although this is sitting on there.
So this is mg.
Well, this is the mg because of gravity, right?
The earth gravity is attracted to mass, m.
Well, the mass is also attracted to the earth.
So what's the force on the earth from the mass?
AUDIENCE: mg.
PROFESSOR: mg, it has to be.
Now this mg equals that mg.
Call this f2 and this is f1.
So f1 equals f2 by Newton's third law.
So that's a fundamentally different set of forces.
So are equal and opposite.
They're an action/reaction pair.
So what Newton's third law really says--
when you understand it-- is that all forces in the world
come in pairs.
So if you ever see one force you have to hunt around for
his other half.
And the other half here is this.
So actually, there's another question you could use for
Newton's third law.
So I don't have it on the list here.
But I'll ask it right now, which is suppose someone comes
to you and proposes the following gravitational law--
f gravity equals g m1 squared m2 over r squared when the g
is the proper units.
So m1 squared times m2, is that a legal
force law for gravity?
No, because when you switch m1 and m2 as you're doing here--
you're switching the mass and the earth--
you actually change the force.
So then this would violate Newton's third law.
OK?
So Newton's third law, it actually is a constraint on
the laws of physics.
It's actually conservation of momentum.
And for momentum to be conserved you can't have any
arbitrary law of physics that you want.
Does that help clarify that?
AUDIENCE: Come up with a better set of taxonomy for
describing it that doesn't lead people-- it leads people
into thinking about that.
PROFESSOR: I know.
I agree.
So I think the terms are very important.
So that's why don't like this form, action equals reaction.
I actually would prefer it were stated all forces come in
pairs that are equal and opposite.
Right, and language does have a powerful effect.
Yes?
AUDIENCE: The technical [? route's ?] obvious, just in
calling the novel force the contact force, but that's not
refined enough.
Because both friction and novel are contact force?
PROFESSOR: Yeah.
AUDIENCE: [INAUDIBLE] constraint force because it
could constrain into the [INAUDIBLE].
PROFESSOR: Yeah, that's probably a better name.
Yeah, I agree with you.
So the comment was that contact force isn't quite
right because there's friction force as well.
And that's also a contact force.
And maybe a better name is constraint force.
I think that is a better name.
OK, so let me finish this list of two more examples of
Newton's Second and third law.
And then we'll have a break.
OK, synthesis.
So creating a problem using Newton's second and third law.
So for example, this is a problem like that.
But here I'm asking the students to make one up.
And here is a very hard evaluate question, which is
what, if any, are the limits of validity of Newton's Second
and third law?
So that's, actually, a very
interesting historical question.
Interesting historically because Einstein thought about
that question.
He wanted to know whether Newtonian gravitation and
special relativity were compatible.
And basically, they can't be because of Newton's third law.
Because gravity should propagate with some speed, not
the speed of light.
So you can't actually conserve momentum right away.
Suppose the sun does something strange.
Well, it takes a while for the earth to
actually respond to that.
And it's in that intervening time Newton's third law seems
to be violated.
So Einstein thought very carefully about that question
and came up with his theory of gravitation.
So this is quite a high-level question.
But it's to show the whole range of things you can ask
about the second and Newton's third law as you
climb Bloom's Taxonomy.
OK, so what we're going to do in the second part is we're
going to actually practice
rewriting a couple of questions.
And I'll show you a couple more examples.
And seeing how we can make questions worse and better,
because I find that's actually quite a good way to get
practice with designing questions is taking good
questions, making them bad, taking bad questions and
making them good.
Both ways are useful.
OK, so we'll do that.
Let's say it's 9:04, 9:10 let's say.
So it's 904 on that clock.
So see everyone, sorry it's 10:04 on that adjusted clock.
So see everyone at 10:10 or 10:11.
And meanwhile, here are the feedback sheets.
So I'll just put a bunch in the two back things.
So when you come back just grab those.
OK, so I'll just answer a couple questions which I
promised to answer.
So the comment made by several people about last week's
examples, say, of the rock and the rolling was that oh no,
that showed me that my physics intuition is
quite off, and bummer.
And so then if you're going to teach it, how do you make
sure, well, A, how do you learn to think intuitively
about these things?
And B, if you're going to teach it
how do you make sure--
related to that--
you actually have the intuition before you teach it.
There's two answers to that.
One is that it's very rare that anyone ever achieved a
goal that they didn't set out to do.
And then just happened upon it.
So I would say that's true, also, with developing
intuition in these areas.
You have to make it your goal.
So it's not going to just happen by chance.
So then questions, OK, if it is your goal how do
you go about it?
One way is anytime you do a calculation, at the end of the
calculation ask yourself this question.
So John Wheeler-- who was Feynman's adviser--
he recommended the following question, which I think is a
fantastic question.
So the question is if you could time travel back and
talk to your earlier self what one or two sentences would you
tell your earlier self before they started solving the
problem that would make the problem just
sort of flow smoothly?
OK?
So what sentence would you tell your earlier, pre-problem
solved self?
So this is after you solved the problem.
So that forces you to give a insightful summary of what you
learned from the problem.
So it forces you towards the intuitive way of reasoning.
And you'll find it's quite hard at first.
Because you don't have the intuitive library.
But at least it points you in the direction to be going.
OK, now what else?
Well, in chemistry there's various books that help
develop intuition.
They have all kinds of intuitive reasoning
questions in them.
And some are quite good.
Some of the books are quite good.
So the one in chemistry that I like is called Voyages in
Conceptual Chemistry.
And in physics one of the ones I like is called Thinking
Physics by Epstein.
I think Epstein's like that.
So what that is, is that's basically, one
question per page.
And it's multiple-choice intuitive reasoning question.
And they're about all kinds of interesting physics, like why
do tea kettles whistle?
If you shake bottle of soda up, why does it explode?
And so it's filled with really, really fascinating
puzzles and ways of thinking.
And then with explanations.
So just by thinking about them, reading the explanation,
trying to understand, OK, then asking yourself Wheeler's
question at the end of reading the explanation, even if you
didn't solve it, you can develop a lot of intuition.
I don't know a good one for math or biology.
And if anyone does, I definitely appreciate any
suggestions.
OK?
But there are resources in some fields.
And the other is just whenever you find paradoxes go after
them and share them with everyone.
And ask everyone you know,do you know any
paradoxes I can work on?
That'll also developed intuition.
OK, and then related to that, one comment was I've noticed
that teaching is the best way of learning.
How can you incorporate that into a course?
So yeah, that's true.
So sometimes you just have to accept that the first time you
teach the thing you're going to be learning a lot.
So that's fine.
And the second time you teach it, it'll be much better.
So you'll actually develop a lot of intuition the first
time you teach a course.
So for example, I found when I was a graduate student, I was
a teaching assistant for Physics 1A at Caltech, which
was like 801.
And I found I had a huge number of
misconceptions about tension.
I just had no idea what tension was.
For example, I thought tension was a force.
And now that seems ridiculous to me.
And it doesn't seem ridiculous to you that's because you're
in exactly the same state I was in before I was taught
physics 1A.
And I've got myself all in a twist when I was trying to
explain it.
And I finally tracked it down to actually, that particular
misconception.
So then I developed a bunch of questions for myself, which I
also asked students about tension.
So teaching is learning.
And then how do you incorporate that into a class
for the students?
Well, you can ask them to teach each other and argue
with each other.
So we'll talk about that in the session
on interactive teaching.
But yeah, it is the best way of learning.
And so if you can get the students to do that as well
you'll be increasing their learning.
And then the last question for the moment was what's my
response to the following article-- so it was in the
globe on February 15, 2009.
And it said, don't open with a joke.
So it was summarizing a paper called "Increased
Interestingness of Extraneous Details in a Multimedia
Science Presentation Leads to Decreased Learning." Journal
of Experimental Psychology, December, 2008.
So they summarized that.
So basically, what they found is that if you start with a
joke that's extraneous to the material it actually decreases
the learning.
And that, I think, is quite plausible.
So I recommend that you start with something interesting in
the beginning.
But it should be related to the material.
So the problem wasn't the joke.
The problem was that the joke wasn't
related to the material.
So the thing should be interesting.
Because you want to draw people in.
You don't want to conclude from that article that you
shouldn't start with some interesting.
But you should start with something interesting that's
related to the material.
OK, so redesigning questions.
So here is an actual question that we will redesign.
And I've left up the Bloom's Taxonomy there.
So actually, I have a bunch of questions.
We'll do a couple of them.
And we'll review, probably, one of the questions we did
earlier, which is about the cones.
OK, so the question to start with-- and we're going to go
that way and that way--
is find the eigenvalues of this matrix.
So for those whose linear algebra is rusty, the
eigenvalues of a matrix so a matrix, m, times some
eigenvector e is equal to lambda times e.
So if there's some vector that the matrix just brings back to
itself with some constant, that's the eigenvalue.
And this is the eigenvector.
So eigen meaning own in German.
So somehow this vector belongs to the matrix.
It's said to be an eigenvector of the matrix.
It's preserved when you hit it with the matrix.
You get back the thing itself.
And in that way it belongs to the matrix.
OK, so there are recipes for computing it.
And so here the question is, I'm asking students to use the
recipe for computing the eigenvalue.
OK, so now I'm going to ask you the following question,
which is to design--
well, first of all, where would you say that guy goes?
It's sort of comprehension.
You know, it's basically computing.
Maybe it's a bit of application.
But it's basically comprehension.
OK, so can we go lower on the taxonomy towards knowledge?
Can we go higher towards the higher-level reasoning?
Let's try to construct questions on this end.
OK, so everyone understand the thing to work out?
I'm going to ask you to try to think of stuff that could be
put here and here.
So this is lower level and this is higher.
Does anyone need me to explain--
or want me to explain-- more about eigenvalues or
eigenvectors?
OK, so find a neighbor or two.
And try to think of stuff that can go in
this box or this box.
And generally, the higher-level questions
requires more knowledge about the material, which is one
aspect of their being higher level.
You need a bigger margin of safety when you're trying to
design them.
But don't let that bother you.
See what you can construct.
And you can use the levels on Bloom's Taxonomy to try to
create some kind of questions around it.
By the way, the answer to this question, by the way, is i and
minus i, just in case you are curious.
OK, so let's see what examples you've come up with, so
related to this question.
Obviously, you'll change it around some.
Let's say either side, this side or this side.
Someone I haven't heard from?
Yes, can you tell me your name?
AUDIENCE: Mike.
PROFESSOR: Mike, yeah?
AUDIENCE: What is an eigenvalue?
PROFESSOR: OK, so what is an eigenvalue?
Where would you put that?
AUDIENCE: Lower.
[INAUDIBLE]?
PROFESSOR: Oh, yeah, good question.
It depends where you put this one.
This one is computing, which I roughly put that somewhere
between comprehension and application.
So right now I would say, we're somewhere around there.
So your question is to define eigenvalue?
OK.
I'm just short-handing it as defined lambda.
OK, so that's one question.
OK, another question.
Can you tell me your name?
AUDIENCE: Wendy.
PROFESSOR: Wendy, yes?
AUDIENCE: So related to that, is that you could put that
question higher.
If you asked them, what does an eigenvalue mean?
If you word it differently instead of [INAUDIBLE].
PROFESSOR: OK, which may have been what you meant as well
and I may have defined lambda.
OK, so you're saying, well, you can actually ask them to
say, what does eigenvalue mean?
OK, so how would you if they know what it means?
So what would you look for in something like that?
AUDIENCE: If they could describe it to you, or maybe
tell you a problem in which it'd be useful, in using it?
PROFESSOR: OK, so yeah, let me write that too.
Create a problem that would benefit from lambda, that
would need lambda.
OK, so for example here, create.
That's gone pretty far down towards synthesis.
I really should have flipped the list around.
But synthesis is quite high on Bloom's taxonomy.
So you're actually going to create a problem.
It's like the Newton's law example.
Yes?
AUDIENCE: [INAUDIBLE]
instead of saying, again, what does lambda mean?
You could say what do the results from finding lambda?
What are the results of that solution?
PROFESSOR: OK, so how does it help you to know the
eigenvalues?
Is that the question?
AUDIENCE: No, I'm just saying instead of the general asking
for the definition, what does lambda mean?
What is an eigenvalue?
What does it mean?
But after having solved the problem saying, what does your
answer mean?
PROFESSOR: Ah, OK, so even if you ask them this first you
can say, OK, interpret your answer.
Yeah, and that's a-- generally--
very good policy.
So interpret.
and where would I put that?
Eh, interpret is somewhere like analysis,
maybe a bit of synthesis.
But yeah, it's quite high on the list.
So interpret, that's another really key word.
In fact, it's often neglected, right?
We just ask students to solve the problem.
And wander, wander, wander.
They get the thing that looks like an answer.
They put a box around it and they're done.
But actually, you can massively increase the level
of a problem just by adding to the problem,
interpret your answer.
What does it mean, that that's an eigenvalue?
And maybe they would say something like this.
But even that would be better than just having done the
computational procedure and got an answer out of it.
Yes, can you tell me your name?
AUDIENCE: Ben.
PROFESSOR: Ben, yes?
AUDIENCE: I might ask, does that matrix have any
eigenvalues?
PROFESSOR: OK, so yeah, so let's put it over here.
Right, so there what you're doing is you're not presuming.
You're not key-wording what they should do.
When you do this-- especially if it's a
linear algebra course--
they've probably seen a bunch of things like this.
Like in a calculus course, integrate
these things by parts.
They just know I have to integrate by parts, whereas
here you're at least one step higher.
Because they don't know whether there's an answer to
the problem or not.
And they have to think, well, how do I know if they're
eigenvalues?
So there's actually one intervening level of
sophistication.
Other suggestions?
Yes, Adrian?
AUDIENCE: Just give them two numbers and say write a matrix
that has these as the eigenvalues?
PROFESSOR: OK, so let me continue the list over here.
OK, so create a matrix which has the following eigenvalues.
So say lambda equals 2 comma 3.
So create a matrix, create a meaning.
We're already high up in the taxonomy toward synthesis.
And actually, I would even use a related one, which is--
so create a real matrix with eigenvalues i comma minus i.
So that one, is quite related to this.
And this is the answer.
So it's quite a hard question.
Even if people can do this and find i comma minus i, this one
is very hard because you're asking to apply a constraint
that the thing be real.
So it's not just any matrix.
But it's that the thing be real.
And then you have to really understand quite a lot about
matrices to do that.
Yes, can you tell me your name?
AUDIENCE: Scott.
PROFESSOR: Scott.
AUDIENCE: So, I was thinking of a synthesis question which
would be is after multiplying the item the eigenvector by
the matrix how is it rotated?
PROFESSOR: OK, so what is the geometric effect on the
eigenvector of the matrix on the eigenvector?
AUDIENCE: Right, but the question is if they understand
what an eigenvalue is.
PROFESSOR: They should be able to answer that right away.
Right, but it really tests for understanding of what the
meaning is.
So it's another way of approaching the question of do
they understand eigenvalue means?
What is the geometric effect of the matrix on the
eigenvector?
And then hopefully, they'll be able to say well, actually, it
doesn't rotate it.
It just scales it.
And the eigenvalue is the amount of scaling.
OK, and that demonstrates quite a high level of
understanding too.
And This is one reason that oral exams, generally, are
quite good ways of evaluating.
Because you can ask these kind of questions.
And as soon as you see that they understand you move on to
something else.
Which of the flip side is that's why oral exams are very
disconcerting for the students.
Because you spend no time on the things they understand.
Right?
So as soon as you know they understand it, you move on to
something else.
So the student just feels like, what the
hell happened to me?
I was doing fine.
And they asked me about something else.
Yeah, that's exactly right.
That's why they asked you about something else.
They knew there was nothing more here to find out.
You understood it.
But that's why oral exams are such a good way-- so
efficient--
for evaluating.
So in Cambridge we used to do oral exam interviews for
admission for undergraduate.
So people would be admitted to the major right away.
So we had two 20-minute interviews.
And we found those were much more reliable than all the
exams that they did in high school, the
equivalent of AP exams.
Because you could really ask questions that really probed
understanding and right away see what people could do and
couldn't do.
So geometric effective of m on e creating a matrix, so you're
asking, so what's the geometric effect?
It's not just comprehension.
You're asking them to apply it and maybe, in a
bit in a new area.
They hadn't really thought of eigenvalues and eigenvectors
as geometric at all.
And now, all of sudden, you're asking them that.
You're asking them to transfer their knowledge, which is one
of the fundamental goals.
Yes?
AUDIENCE: Once you give them two matrices that has the same
eigenvalues you can ask them to see if they are
related in some ways.
PROFESSOR: Right, so for example, let's do this one.
So how are those two matrices related?
So they both have eigenvalues i and minus i.
This one obviously does because those are the diagonal
values and there's nothing else.
And this one, you have to do it by the magic procedure.
You don't have to.
And so how are they related?
So that is some kind of analysis question.
And it's quite a difficult analysis question.
It tests for really, do they understand what matrices mean?
So that's why I've underlined this word.
The questions all in this column are all somehow testing
that they understand what things mean.
So that ties back to the point I made earlier, which is that
the whole fundamental point of asking good questions is to
avoid rote learning.
Is that by asking good questions you're
fighting rote learning.
And rote learning is the bane of most education.
Right?
Rote learning is too often the result of education.
And it's the-- in my view--
fundamental problem with education.
So by asking questions that are different you're actually
starting to fight that and break the habit.
So for example, how does rote learning show up?
Well, in the bouncing-ball question, so the force is-- in
the instant that it's
stationary, while it's bouncing--
people say mg.
While this mg is right.
The gravity, but the contact--
let's say the constraint force--
in this case it is a very full-contact force is mg.
One of the reasons they say that is the misconception that
they think a equals 0 because v equals 0.
So that's one reason.
But the other reason is that they've done so many problems
with books resting on tables, balls resting on tables, where
the force is mg.
Right?
So they've now induced a new rule of physics, let's call it
Newton's fourth law.
That anytime you see a contact force it is mg.
So contact force equals mg maybe times cosine of theta.
[LAUGHTER]
PROFESSOR: Right?
or sine theta.
I guess it's mg--
well, it depends how you measure theta.
But theta is the tilt of the plane.
So if the plane is tilted 0 degrees just to ground, well
it's then mg.
So that kind of learning is the result of rote learning
because they've solved problems without understanding
what they mean.
So instead of figuring out that oh, actually, you could
understand it from Newton's Second Law and understanding
acceleration, they think oh, there's a new pattern here.
Right?
Oh, a new pattern is whenever I see contact forces it's mg
times maybe cos theta or sine theta.
Yes?
AUDIENCE: What's the exact question with that system and
what's the normal [INAUDIBLE]?
PROFESSOR: Yeah, so the question was so you drop the
ball, say from here.
And when it bounces off the steel table there's one
instant when it's stationary on steel table.
And what are the forces on it at that instant?
AUDIENCE: I remember from last week.
PROFESSOR: Yeah, no, fair enough.
I should have stated the question out again.
So this is stationary for an instant.
All right, so actually the ball is quite compressed here.
Yes?
AUDIENCE: Isn't it impossible to give an exact answer for
the force with the [INAUDIBLE]
if you don't have the duration?
PROFESSOR: It is, right.
So it is impossible to give the exact answer.
But what you ask them is, well, how big is it roughly?
So I always protect myself by saying roughly.
And then you can ask anything you want.
And because, really, all I'm interested in is do they think
it's 1 times mg or do they realize it's 10
to the 4 times mg?
And those are so far apart that I don't care about
factors of two or three or even 10.
Now, how can you know that it's 10 to the 4 times mg?
Well, that is quite a hard question.
That's way down there.
Even though you'll see some lists of Bloom's Taxonomy say
estimate is a low-level skill, not that
high in Bloom's Taxonomy.
It's almost like computing somewhere in comprehension.
I don't agree with that.
Because as you saw from the example where students
couldn't estimate 3.04 times 5.3, they're terrible at
estimation.
So estimation actually hits one of the big misconceptions
and reveals a lot of the misconceptions.
So I put estimation almost at the level of synthesis because
it forces you to really synthesize your understanding.
You can't hide behind formulas.
So the next thing I'll do is say, OK, let's estimate this
and see if you can do that.
And if they really understand the system they
can estimate it.
But it's quite hard.
And I find almost no students can do it on their own.
But the method of estimation is, as you say, you do need to
know the contact time.
So the acceleration is not 0 as you say.
But it's delta v over delta t, where delta t
is the contact time.
Well, it's going v and then it's pretty much going v up,
so that's about 2v.
But I don't care about factors of 2.
And then, what's the contact time?
Well, there's a couple methods for doing the contact time.
One is you listen for the pitch.
And whatever the highest frequency in the pitch is--
suppose it's 10 to the 4 hertz--
well, then 1 over that time, 1 over the frequency
is roughly a time.
So it's probably the contact time.
So that's one approach.
The other approach is to say well, actually, physically
what's happening?
The ball is hitting the bottom.
And this end compressing.
But this end keeps going.
Because it has no knowledge that the bottom of the ball
hit the ground.
How's it going to know about that?
Well, the bottom of the ball has to tell it.
How's it going to tell it?
Well, it has to send some kind of signal.
How does it send signal?
Sound.
So it sends a sound wave upward.
And the sound wave travels in steel about 5 or 10 kilometers
per second.
So you suppose it's a 5-centimeter ball, then it's
about 10 to the minus 5 seconds is your contact time.
So this is really short.
So suppose v is something like a meter per second, and divide
it by 10 to the 5 seconds.
You're talking about something like 10 to the 5 meters per
second squared, which is 10 to the 4 times the acceleration
due to gravity.
OK?
So I went through because I think it's interesting but
also to show that estimating actually requires generally,
very high-level skills.
And it's quite hard for the students.
Therefore you should do it.
So there are some people who say, well, it's very hard for
the students.
Don't do it.
But again, there's another rule of thumb in teaching,
which is nobody ever learned anything that you didn't
actually try to help them with.
Well, that's not quite true.
But if you never teach something they're not very
likely to learn it.
So if it's something hard but valuable you have to start
teaching at some point, which is why I put
up that graph earlier.
Oh, it's gone now.
Where in the earlier stages of teaching you do things more
approximately.
And then in the later stages you do it more exactly.
So what I've told you now is a slight lie.
So this brings up the question from last
time, what about lying?
Is it useful?
Yes, it's essential because this argument is simple enough
that once you understand it you can just
keep it in your head.
And it's a bit of a lie, the reason being that the ball
doesn't have a uniform cross section.
So as the ball compresses the contact area changes, whereas
if the ball were cube or a rod the contact area would stay
the same as it compressed.
So there's a correction factor because the
contact area changes.
Now, that correction factor is not very big.
It turns out to be the fifth root of some
dimensionless thing.
And so this sound-wave argument you put in to the 4/5
power and then you put in the other thing to the 1/5 power.
So it's almost correct for this spherical ball.
So in a later of course it actually
adjusts for that factor.
And then in a much later course, probably, what I would
do is calculate the exact 2 pi over whatever factor that
shows up as well, by way of increasing the symbolic
complexity.
But even this estimate, doing that is conceptually hard.
So I would say start that early.
AUDIENCE: I think the point I was trying to
make was that this--
I suspect that the students answered that wrong because of
real problem solving, and that they expect there to be a
solution given the data that you [INAUDIBLE].
PROFESSOR: Well, if you've worked with them a bit on
estimating they know that they're going to have to do
some thinking.
If you say compute the force I agree with you for sure.
They're going to think they have to be able to do it from
the data there.
And they'll just say it's got to be mg because that's the
only other force I know.
But if you say estimate then there is a bit of a queue that
they have to include some other information which might
not put in the problem.
And so that mitigates the rote learning.
But what I find, even then, is they usually still can't do it
because their misconceptions about acceleration are so deep
and about equilibrium and force.
They think f equals mv, basically.
OK, so let's just look at what we've done here.
So I would say you can define lambda.
You can define eigenvector.
You can even say state the computational procedure.
I'm not sure where I would put that.
Depending on how people learn it, this could be purely a
knowledge question.
Because people could state the procedure but not really
understand it, not be able to do anything.
Or actually, you could say program the procedure.
Well, that would be something over here probably.
Because to program it you probably have to understand
what you're doing pretty well.
So stating the procedure, it depends how you ask them to
state it, whether it's a low-level or
a high-level question.
So I'll put program over here.
OK, so we've gone down and up.
And again, I want to remind you, I'm not saying all your
questions need to be here.
But generally, the questions are too much over here and not
enough over here.
So this gives you a recipe for thinking of new questions.
Yes?
AUDIENCE: So throughout the course do you basically, maybe
start on the first three in the beginning and then you
move it more through until you reach the end of the course?
PROFESSOR: More fractal.
The question was as the course moves on do you move more
towards synthesis?
I actually try to make it fractal.
So within each unit and each day even, I try to maybe have
something, knowledge or comprehension just to make
sure everyone's tracking.
But try to have something interesting as
well, all the time.
So something synthesis, analysis, maybe some evaluate.
And so each problem set-- for example--
I'll put, generally, warm-up questions.
So basically they're marked warm up so that people know
OK, those are going to be knowledge and comprehension.
And you need that.
Because otherwise there's no point trying to do synthesis
if you don't have the base knowledge.
But just make sure you understand those,
just in your sleep.
And then regular problems, which tend to be somewhere in
the middle, say application or analysis.
And then bonus problems which are just optional.
But they're for people who either find all the other
problems easy or are really curious.
For them I put either synthesis or evaluation.
For example, in the approximation class the last
problem set had design an email indexing system.
So that was the bonus problem.
And there was a bunch of warm-up and regular problems
which were involving Unix text processing.
And then for the people who are really curious actually,
put it all together and make a really fast email-indexing
system so you can just find any email that you want just
by doing keyword search.
Sort of like a web search but just on your email inbox.
Because I actually use that myself.
Because I find folders are just hopeless.
It just takes all my time.
I have to figure out what folder things are in.
And that could be many.
So I just keep it all in one giant inbox with, I think,
150,000 messages in it.
And I just search through that.
And I have an index that's rebuilt every week or so.
And so I can search through it really fast.
So it's actually very practical problem and it gives
you an example of the levels of things you can ask.
So try to do it on a fractal scale.
Each class should have something, mix the levels each
homework, the exam.
So the general rule is to mix them up all the way.
OK?
So hopefully, from that, you feel confident that you have
tools that you can make interesting questions.
And this does have lots of payoffs, which one of them is
that questions that are enjoyable and educational,
students love them.
So when I practiced what I preached and made these kind
of problems, mix of problems for my classes, often students
have said, oh, can you please have more problems?
Which is very rare.
They said, oh, these are some of the most interesting
problems I'm likely to get at MIT.
And I used to read them out in my living of my
independent-living group just so that other people could
hear the interesting problems and we'd try
to solve them together.
And then I've had several emails after class finished,
saying, oh, you promised to post the optional problem set.
Where is it?
So it's because the problems, actually, somehow connect to
things that people find real and interesting.
So that's a mix of the perceptual.
Don't make it just purely symbol manipulation.
Try to connect to ways people look at the world.
But also, try to connect to higher-level reasoning skills.
Question?
AUDIENCE: Do you always use bonus problems?
PROFESSOR: I do.
I find that works really well.
Because it's like a pressure-escape valve.
It caters for the diversity of backgrounds in the class.
The people who are feeling very worried by the material,
they can just skip them, no problem.
And they don't feel like they have to do them.
And they don't feel like, oh my god, I can't do them.
I'm going to get points off.
And the people who find the other stuff easy, they find
something for them too.
So yeah, I try to always use bonus problems.
And I learned that from Knuth's book, called Concrete
Mathematics.
So actually, I think they have five levels of problems at the
end of every chapter.
There's warm ups.
There's homework problems.
There's exam problems, bonus problems,
and research problems.
So the research problems were the ones
that are not yet solved.
So at least you know they're not yet solved.
So you're going to be spending a while on them.
And then in the second edition I think some of them were
actually solved.
OK, so if everyone could just take a one minute and fill out
the feedback sheet?
Oh yeah, and let me just say, why I'm doing this now at
10:50, of the questions on the sheets before was the feedback
sheet, the person said, I've been using them in my class
and they've been really useful.
But I find that the response rate has been dropping.
What do you do about that?
And I have to say, that I found the same thing.
This year I found the same thing.
Not in this class but in my other class, whereas I didn't
find that last year.
And I was trying to figure out why.
And theory I have is that I was trying to put in too much
material in the class and not leaving the students time
before the five-minute mark to fill out the sheet.
So one thing, I've been trying to mend my ways and make sure
I end class at 9:50 or 10 minutes to the hour so the
students have at least one or two minutes to fill out the
sheet without feeling pressured to get
to their new class.
So I'll let you know how that goes over the next week.
So take a minute now and then that'll maybe make you think
of some questions which we can answer in
the next two minutes.
And then while you're doing that, just a couple
announcements.
So sorry I didn't say anything before about what to do with
the equation treatments that you shared with each other.
So just discuss them with each other.
You have a collaborator.
And then revise your treatment based on what
that person told you.
You don't have to turn in your revised treatment.
You guys are all graduate students, or
almost all of you.
You're all taking the class because you're interested.
I'm not trying to police everyone.
But the goal here, and what you should do, is take the
benefit of the collaboration, revise your equation
treatment, and share it with each other.
And then just bring in your original one next week.
And then I'll also have a couple of readings for you to
do for next week.
And also a short problem to work on, things to think about
to bring in next week as well.
OK, so sheets and then questions.
Question?
AUDIENCE: So I tried doing these sheets for a class that
I was teaching yesterday.
In fact, I found that I got a response rate of 15%.
If your response rate from the beginning's very low and
[INAUDIBLE] to increase that rate?
PROFESSOR: Right, so 15%, yeah, I find I'm getting that
around now too.
And I think what's happened is that I've got people in the
habit of not filling them out because I wasn't giving them
enough time.
So I may be hosed because now they're in the habit.
So I'm trying to mend it.
And I'll let you know if that fixes it by giving people
actual time before they feel rushed.
But it may be too late.
And I worry that it is too late.
Because last year I was actually more diligent about
making sure they had time.
And I found the response rate stayed about 60%, 70%.
But even I find the 15% very useful.
And I found that--
people even in the past, when the response rate was 30% in
another class people said, I didn't have a
question all the time.
But I knew when I did I could ask it.
So even the 15% isn't bad.
It's still useful.
But yeah, I would like it to be higher.
And I'll tell you how that goes.
Question?
AUDIENCE: [INAUDIBLE]?
Wiggins and--
PROFESSOR: McTighe, yeah.
AUDIENCE: Oh, McTighe.
Where are they based?
PROFESSOR: So where are they?
They're in New Jersey, and they actually consult on
educational course design and run workshops and courses.
And we actually invited Wiggins last
year to come and speak.
So I'll put a short reading selection from their book
called Understanding By Design for optional reading for
people so then you can learn more about that.
I find the basic form that they say very useful.
I find applying it the way they describe, sometimes I
just can't square it.
But it may resonate with you quite well.
And the overall approach resonates very
strongly with me.
So hopefully you will like it too.
So I'll put that on the website and some readings for
next time about course design.
Yes?
AUDIENCE: Just a comment, actually on your web page,
where the response sheet [INAUDIBLE]
and situation where only a few people were very dislike the
current [INAUDIBLE].
They don't like it this way and so they want to change it.
But the 80% who don't feel [INAUDIBLE] is
actually OK with it.
[INAUDIBLE] the change.
It makes most people unhappy.
PROFESSOR: Unhappy, right, so that can happen.
And so that's the silent majority problem.
So what I do when I get comments like people saying,
well, I wasn't happy about this.
Sometimes what I do is--
especially if it doesn't resonate with my intuition
about how things were going, and I thought
they were going well--
I'll say, well, there were a few comments about this issue.
What do people think?
And just take an informal straw pole
before I make big changes.
AUDIENCE: I think it helps to have a positive question on
[INAUDIBLE], like, what did you take
away from this lecture?
Or what did you really understand?
PROFESSOR: Right, I think that's right.
It helps to have a positive question.
So that's why I say, either what you take away, or what
helped or hurt?
So there's an opportunity to say something helpful.
And any other comments is also interpreted in a positive way.
Answers from Lecture 5 to questions
generated in Lecture 4.
PROFESSOR: OK, and questions from last time.
So they grouped into several ones.
One is how should you choose your levels of your problems--
say, in class or in homeworks-- among the
different levels of Bloom's Taxonomy?
Should it be, for example all towards the evaluation?
So your side would be here, sort of the high level.
A Mix?
A flat mix?
Should it be peaked towards the middle with some tails?
So that depends, partly, on your course design.
So to answer that question, you can't really answer that
question until you figure out what your course goals are.
So in some ways there's an argument for doing course
design first and then working on what problems to do and how
to design problems.
But actually, I like the other way around.
I like doing course design second.
Because actually, doing problems is concrete enough to
do something.
And then it leads to questions about course design, which
we'll answer how to mix the levels.
OK, another one was several people wanted to know, why is
tension not a force?
Or in fact, maybe is a force.
So let me explain that.
Because that actually gives you another example of sorting
out misconceptions.
So it's always useful.
So I'll show you the sequence of questions I used with
myself when I finally convinced that
tension wasn't a force.
And here's how I did it.
And then I used the same sequence of questions with
students to fight the misconception that's been
induced by many pictures of--
for example-- a pulley.
And here's a rope and there's tension.
Sorry, this is actually done the other way.
So here's the rope.
And there's a t labeled there.
And then on the mass there's a force labeled as t upward.
So when you see that you think, oh, tension's a force.
So you see that a whole bunch of times and you're pretty
sure of it.
So the sequence of questions is as follows--
say OK, here's a tree.
And I'm going to tie a rope to the tree, rope or a string,
it's a mass-less thing.
And I put a force on it.
OK, so I'm going to pull that tree.
The tree's not going to go anywhere.
We're going to pull on the rope with
100 newtons of force.
OK, so you can try this yourself too.
What's the tension in the string?
OK, so take 30 seconds and I'll take a tension.
OK, a tension, anyone?
AUDIENCE: 100 newtons.
PROFESSOR: OK, so in fact that is 100 newtons.
And most people will say 100 newtons.
OK, so now the next question is this--
here is a river and here's a rope.
And me and a friend each pull on the rope with 100 newtons.
OK?
OK, so discuss with your neighbor--
a, b, or c, what's the tension in the rope?
And we'll take a vote.
OK, so take 10 seconds, get your vote ready.
We'll take a quick straw pole.
Who votes for 0 newtons?
Who votes for 100 newtons?
Who votes for 200 newtons?
OK, so now here, how do you know what it's going to be?
Well, you make a force diagram.
And there's a way to convince yourself that there's no doubt
about what it should be.
And we just draw the forces on the rope here.
Oh, we've already done that.
Let's draw the forces on the rope up there.
Well, what are the forces on the rope here?
Well, it's 100 newtons from you.
But there must also be 100 newtons from the tree.
All right?
So in fact, these two situations are identical.
So yeah, 100 newtons seems pretty clear here.
So it has to be 100 newtons up there too.
OK, now that--
so far-- isn't the hard part.
But now is when you actually create the contradiction with
the idea of it being a force.
It can't be a force because if it were a force you would just
add up vectors.
Right?
So there's a couple answers you get if it was a force.
If you're a bit sloppy you'd just add up the magnitudes and
get 200 newtons.
If it's really a vector--
you're putting two vectors on it, two forces--
you should get 0 because they're two opposite forces.
But actually you get 100.
The only way to get 100 is that the tension has to be
something completely different from a force.
So that's how you know it's not a force.
But how do what it is?
Well, it turns out to be a tensor.
So it's not a vector.
Forces are vectors.
It adds differently than vectors do.
It adds like a tensor.
So tension is more like a pressure.
It doesn't have a particular direction.
Pressure is in all directions.
So things like that, they can't be described by vectors.
So you need something else.
And that's why we introduced tensors.
So now, in freshman physics I wouldn't give them a whole
story about tensors.
But I do actually teach this much of it so that they know
that is a different object.
And then what do I do about this?
Well, I won't write t unless it's for myself.
But for the students I think it's sloppy and it increases
misconception.
It's much better to write the force sub t, which is the
force due to tension.
OK?
So even a slight change like that can mitigate the rote
learning, where people think oh, tension's a force.
Because you always see it drawn like every other force.
There's mg down, There's t up, must be a force too.
AUDIENCE: [INAUDIBLE]?
PROFESSOR: What are the two objects you hit it with to
make a scalar?
It's the two directions that you're
interested in, basically.
So it would be the x direction or the direction along the
string and the direction along the string.
AUDIENCE: So it's the two direction then?
PROFESSOR: You can use the same direction twice.
So it's the direction along the string
and along the string.
There's no cross direction.
So it has no shear.
So there's no cross direction.
So it's some component, tension is one component of
the stress tensor.
So now, again, freshman don't need to know all the details.
This is back to the subject of lying they do need to not be
told fundamental lies.
Like for example, tension is a force.
OK, so now, how do you deal with misconceptions that show
up on problem sets?
This is a good question.
So the point was made well, in class you can sort it out
right then.
Suppose you ask a question, say, about the Ideal Gas Law.
And you find that people have confused the Ideal Gas Law and
the Adiabatic Gas Law.
Well, you can talk about it right then, which is a good
reason to do those things in class.
What happens on the problem set if you find there's a
bunch of misconceptions?
Well, one thing is, the first time you do the problem set,
yeah it's a bit of a problem.
Because you don't know what the misconceptions are unless
you've researched ahead of time.
But the second time you do the class you know what they are.
So you can address the misconceptions in
the solution set.
So when I'm not totally sleep deprived I always try to get
the solution set out the exact same day that the problem set
is turned in.
So the students can right away see.
They can turn in their problem set, pick up a solution set or
look at it online.
So they can right away get quick feedback.
And they can sort out the misconceptions that way.
So that's a good question.
So it is harder.
Yes, question?
AUDIENCE: What do you do about the fact that a lot of people
will just ignore the solution sets?
Or do you just consider that their problem?
PROFESSOR: So the question is, what do I do about people who
just ignore the solution sets?
Because a lot of students do that.
Yeah, partly I consider it their problem.
Because look, that's there for them to learn.
But partly, to the extent that they're ignoring the solution
set because they only cared about the grade.
So one cause of that is they really only
care about the grade.
So they just want to wait for the grader to give them back
their solutions, their problem set with a mark on it.
And they don't care what's right and wrong.
That, I think, partly, is our responsibility that we've
chosen boring problems.
So to fix that, I think, one of the key areas is to use
more of the Bloom's Taxonomy and make interesting problems.
And the flip side is to deemphasize the grading.
So actually, I like the PDF scale on homework.
I think we are too micromanaging, generally,
about homework, you know, grading it on a percentage
scale, on A, B, C, D. P, you did a reasonable.
And D, you blew it off.
You did something but it was a joke.
And F, you didn't turn it in.
I think that scale puts the right emphasis on the grading.
Look, we just want you to do the thing and try it.
And if the problems are interesting that allows people
to actually get involved in it.
So there's lots of studies that show that if you reward
people for things they already enjoy--
so you start paying them more for doing things they really
like-- they actually start liking them less.
So I'll talk about that as one of the political barriers to
educational change.
Because that's completely against the conventional
psychology of this society, where everything has to be
rewarded, merit-based pay and this and that.
So how do you go against that?
It's quite difficult.
But one way, in general, in those problem sets is to
deemphasize the grading.
Related method--
which I haven't tried yet but I'm planning to--
is reading memos on the solution set.
So I talked about reading memos before.
And maybe one of you will try it before me and can tell me
how it works.
A reading memo is--
I talked about-- where you have the students take a
reading and mark-- either online or
on a piece of paper--
things that were confusing or things that were interesting
and things they noticed.
Well, you can do that with the solution set too.
You say, OK, your assignment, homework 3A is to look at the
solution set for homework two and mark
anything that's confusing.
So that forces people to read the solution set.
And often what you'll find is that they'll find mistakes in
the solution set, which is good to know because you can
correct them right away.
But you'll see what parts are confusing to students or not.
Did that help answer your question?
Great.
Do I ever asked Wheeler's question explicitly?
Yes, sometimes.
I should do it more in class in lecture.
But I do it a lot to myself.
Whenever I solve a problem I think oh,
what was the key idea?
And then once I see the key idea happen many times I think
oh, that's a very key idea.
That should be taught explicitly in class.
So I do that.
And I'm going to talk about that principle of course
design today, basically organizing courses around
large themes and principles, which is one of the main
[? cures. ?]
OK, so what's wrong with tension as a force?
How far should I go down the taxonomy?
Basically, where in the course should I be?
Really low in the taxonomy early and higher later?
Well, I talked about that a bit last time.
The fractal picture is not bad.
So yeah, you have a few low-level examples and
higher-level examples.
But you do that throughout the course.
So you don't wait until the end of the course to get the
reward of the really interesting problems.
And then related to that, what about exams and homework?
Should they be the same or different?
My view is they should be the same given the constraint that
maybe the exam is done in a shorter time.
So if anything, the exam should be
easier than the homework.
Generally, homework people will spend six
or seven hours on.
When I was an undergraduate we had take-home exams.
And we would spend 24 hours on them.
You pick them up, you signed it out.
You'd turned it in in 24 hours.
So that was misery.
And it was interesting.
You learnt a lot.
But it was kind of miserable because it just wiped out a
whole day from your quarter.
And you had that for four classes.
So four days from your quarter were just gone.
So generally, exams are three hours, maybe two hours.
And so that's much less time than a homework set.
So if anything, exams should be easier,
definitely not harder.
Student loath when the exam does different things than the
problem sets.
And that is a general principle of course design,
that if you want to prepare them for being able to do
stuff-- for example on the exam--
you have to prepare them with things.
You have to "teach to the test." That's actually good.
You want your homeworks to be like the exam.
And that's fine if the test is good.
If the test tests real-world, interesting skills sure, then
it's fine to teach the test because they'll come out being
able to do that, no problem.
So the exam and homework-- as a rough rule of function--
will be pretty much the same.
Or if anything, the exam should be easier.
Oh yeah, the first time you teach a class.
So all this Bloom's Taxonomy and designing questions, it
takes a long time.
And it's really hard to make good questions.
It's easy to make define XYZ questions.
But what about the higher-level questions?
What do you do to mitigate that problem of a
huge amount of time?
Well, one of the cures is to cooperate with other teachers.
So you steal their good problems.
So I gave you the T.S. Eliot saying before, that talent
invents and genius steals.
So be a genius.
Whenever you see a good problem just use it.
And whenever someone wants a good problem, if you
have one, share it.
And now, should you guard your problems?
I think those days of guarding problems are gone.
They've gone by because of things like OpenCourseWare,
putting problem sets online.
It's just not possible anymore to guard problems.
I'm not sure it was ever a good idea.
But now it's not even practical.
So everything is online, all from the previous year.
So the problems can't be guarded.
And it's not practical to make up a whole set of new good
problems for the next year.
You'll never sleep.
So I just trust the students not to cheat by looking at the
old solution sets.
And if you're grading, that's yet another reason to minimize
how strict the grading is.
So if the grading is really, just that they make an effort
and try to learn something, then there's no benefit to
cheating and looking at the old solution sets.
So you actually make the incentive structure so that
they don't need to do all that.
Question?
AUDIENCE: Do you have to change your problem to a
certain extent in terms of plagiarism?
PROFESSOR: So the question is, do I have to change the
problems to a certain extent to avoid plagiarism?
I don't even care.
I don't even change them.
I think, again, it's sort of what Adrian has said, is it
their problem if they do it?
And to some extent it is their problem.
If they're going to cheat and look at the old solution set
and not acknowledge it.
There's only so much you can do.
And you then get in an arms race with them.
You change the problem to some extent.
But then someone else makes it.
So they can always subvert that.
One person can solve the new problem and give it to
everybody else.
So you have no hope of actually winning that war,
even if it were a moral war to try to win.
So I don't even change them.
And I say look, just acknowledge, just like you
would in science.
I think it's Larry Lessig, the copyright scholar,
professor of law.
He used to be at Harvard.
Now it's Stanford.
He founded Creative Commons and various fantastic
copyright projects.
So he said, well, there's a couple ways to get-- not
compliance--
people to do things.
One is you can do it through rules.
For example, this is illegal.
You just can't do this.
We'll punish you if you do this.
The other one is through norm.
So this is just not done.
In our community we just don't do things like that.
And generally, this kind of-- not exhortation--
but push works much better than the punishment kind.
So I just tell them, look, in science when you grow up and
become a scientist of course you acknowledge your peers and
people you learn from.
Because that's just polite.
It's just how we are.
We say, please.
We don't say gimme, gimme, gimme.
I'm working on teaching my daughter that right now.
She says, daddy read.
Daddy read.
Mommy read.
Elsa read.
I'm like OK, I'm fine with the Elsa read.
But daddy read, mommy read, that's nice.
And we love reading.
But how do you ask?
And now she says mommy read please, daddy read please.
And so it's the start.
So people already learnt that from young.
So you can tap into that.
Look, how do you act as a member of the community?
We're social animals.
So you don't have to put too much
pressure in that direction.
Whereas if you try to do it through a punishment you have
to work much harder.
The punishment one--
underneath--
has the idea that we don't trust the students.
Right?
And in that way it's common with changing the numbers on
the problems set.
It's that we don't trust you so we're changing the numbers.
So I try to avoid those things as much as possible and try to
go more towards this.
Yes?
AUDIENCE: Has anyone tried to have problem
sets without solutions?
So then there's no grading.
And then the problem sets are just for practicing.
PROFESSOR: Yeah, have problem sets for practice with no
solutions or no grading on the problem sets?
Have people tried that?
So there's actually whole educational
systems based on that.
Not so much in America, but in England, the Oxford and
Cambridge tutorial system.
You're given problems to work on.
So I was on both sides of that system.
You're given problems to work with your tutor.
And they're not graded.
I mean, your tutor helps you make sure that
you can do the problems.
But everything that's graded happens at the end.
Well, when I was an undergraduate it happened at
the end of your entire degree.
There was eight three-hour exams that counted for
everything.
And nothing else counted.
So it had some craziness.
That time at the end was pretty stressful.
But before that your tutor was actually your friend.
And your tutor wasn't the person doing the exams.
So you would do problems because you wanted to learn
and your tutor would help you so that you would do well on
the exam that other people were setting.
So you had a nice alliance with your tutor.
AUDIENCE: Does that work well?
PROFESSOR: I would say it works well.
At MIT it would work great.
Given [INAUDIBLE]
that people are too overloaded.
And they would say, oh well, if there's not going to be any
grading at all, I don't have to do anything, while I'd
really like to do this problem set.
But I have four other classes that are making me work 20
hours a week.
And I don't have any time.
So then because of the other pressures
they might fall away.
But they would like to do it.
So that's why I compromise by just saying, you have to do
the thing and make an effort.
And then I trust that once they start trying it they'll
enjoy doing it.
So I tried to make it as light as I think you can get away
with at MIT, given the pressures from
all the other teachers.
But in the ideal world I think that's what we would do here.
There wouldn't be that pressure.
AUDIENCE: [INAUDIBLE]?
PROFESSOR: Yeah, in Oxford and Cambridge it works very well,
provided you have a good tutor.
And there's many students who are there just because it's a
thing to do, to go to Oxford and Cambridge.
But if you factor out those students who are just there
for the stamp of the place, for the other students it
works fantastically well.
I mean, the tutorials are a great discussion time.
You can really kindle people's curiosity because it's really
not based on grading, it's based on helping.
AUDIENCE: [INAUDIBLE]?
PROFESSOR: It's so hard to do.
I mean, I don't know of any comparative
studies within one culture.
And I don't think you could really do them well between,
say, the English universities and the American just because
there's so many other social variables that are so
different like the population that goes to university, which
universities are talking about the exam system itself, so I
don't know of any studies that have been done across cultures
and that I would trust.
And within I don't know of any.
But it's a good question.
I know studies in general about rewards, which is that
the more you reward people for things that they like the less
they like doing them.
So I might give you some readings for that for very
last penultimate session.
And for the reading that I'm going to give you for the next
session is going to be about tutoring, actually.
So I'll put that on the website today.
OK, questions?
Yes?
AUDIENCE: [INAUDIBLE] of a question.
So I was wondering, if you do exams or homework problems
throughout the semester, you can, maybe, also better
monitor how the students develop compared to having one
big exam on the end.
And then maybe using [INAUDIBLE] lectures.
PROFESSOR: Right, so having continuous assessment is one
phrase for it.
It allows you to monitor, make sure things are going OK.
So all of a sudden you don't find, oh my god, the person
didn't know anything.
They got a third, which is like straight D's in your
whole undergraduate.
So in Oxford and Cambridge things are graded on first,
second, third.
And all of a sudden you had had no idea the student was
going to get a third, or they were sick or
something like that.
So I think, yeah.
So homework problems and exams have a really good purpose in
assessment and helping students get feedback on how
they're doing as well.
So for that purpose, grading is useful.
And one solution to that dilemma-- because I don't like
to grade exactly because it stresses them
out with the grade--
is to give them the solution set right away.
So they can get feedback from the solution set whether they
understood things, not whether they had a 90 or and 80, but
whether it made sense to them.
And that's, I think, a much more important thing.
But then the other part of grading, which is that it's
used publicly--
as part of your record and written to other people--
that part of grading I think is not right.
I don't think it's morally right to, basically, reveal
information about students to other people without their
permission.
I don't think that should be the purpose of the university.
But the other purpose of grading, helping students know
where they are.
And helping us know where they are so we can help them more,
I think, is a completely valid purpose.
OK, so there was another suggestion which was to use as
many non-physics and math examples as possible.
So I'll try to reform my ways for the non physicists and
mathematicians.
It's just those come most naturally for me.
OK, so I think that was the main--
yes, question?
AUDIENCE: [INAUDIBLE]
for evaluating the students or for teaching the students.
PROFESSOR: Pardon?
AUDIENCE: Homeworks?
PROFESSOR: Yes, for evaluating versus for teaching?
AUDIENCE: Yes, because that's one thing to evaluate, right?
[INAUDIBLE]?
PROFESSOR: It's both.
So there's two, yeah.
So the other purpose is that they learn and they do things.
That's for sure.
So you want them to do that.
And you can do that in class and homework.
And the take-home exams that we had when I was an
undergraduate had that purpose too.
You spent 24 hours really learning stuff.
So that purpose is very valid too.
But it's also very valid to know where the students are.
So for example, that basically closes the feedback loop.
You're doing all this stuff.
And now you're seeing the result of it.
And you want to know, well, did any of it take?
Did any of it stick?
And if none of it did, well, maybe you want to change what
you're doing.
AUDIENCE: Oh, but you cannot evaluate a
person every week, right?
And that's what drives students to copy solutions
because they care about the grade.
[INAUDIBLE].
PROFESSOR: That's why distinguish
two senses of evaluate.
There's evaluate for the purpose
of helping the students.
And there's evaluate for the purpose of publishing their
grades elsewhere.
And it's the publishing their grades elsewhere
that worries them.
So you want to minimize that part.
And you want to do as much evaluation towards helping
them as possible.
So a solution set-- for example, one thing you can do
is say OK, here's the solution set.
Ask me about anything that's confusing.
And then you can put some of the responsibility for this
evaluation onto them so it's self evaluation.
But you're still using the homework for evaluation.
It's just you're not trying to say look,
that's the main thing.
I'm going to use that to tell everyone else about it.
That's what worries them and produces copying.
So you want to minimize that as much as possible.
And it's hard.
And that's the subject of the political obstacles to
educational change session that'll be in the
second-to-last lecture.
OK, so yeah, I don't pretend any of these have easy
answers, unfortunately.
Oh, there was another suggestion, which is that I
make a web forum where people can ask questions afterwards.
Because it's hard sometimes think of a
question on the fly.
And that's true.
And the problem is it's not easy to make web forms on the
MIT.edu site.
So I'll try to figure out a way to do that.
But for security reasons you can't really easily make web
forms that just append to a file or do something like
that, although I'm going to look into it and see what
workarounds I can find.
Yes?
AUDIENCE: [INAUDIBLE]
scripts?
PROFESSOR: Yeah, you can use scripts at MIT.edu.
So I have to figure out how to use that.
But yeah, I had that in the back of my mind.
But thanks for pointing that out.
Yeah?
AUDIENCE: What about forms?
PROFESSOR: Pardon?
AUDIENCE: Forms?
PROFESSOR: Forms?
AUDIENCE: [INAUDIBLE]?
PROFESSOR: Yeah, so then you get email.
So the only problem with that is that you get a ton of spam.
So there are all these spam bots that float around and
just fill out forms and send junk emails all the time.
So maybe that can be filtered out and it's not so bad.
So but yeah, that's one of the few things you can do on
MIT.edu, directly on the web.MIT.edu.
So there's a very solutions to it.
I'll try to think of something around that because it is a
good suggestion.
OK.