Lec 3: The laternal geniculate nucleus and the visual cortex

• Download English-US transcript (PDF)

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

PROFESSOR: Good afternoon, everyone. Let us get started. The first thing we're going to do today is I'm going to finish up what I didn't get a chance to finish the last time, because we ran over a little bit. And what we are going to cover to begin with is the lateral geniculate nucleus of the thalamus. This is a structure in the thalamus that receives an extensive input from the retina from retinal ganglion cells. And it is a beautiful structure that has several layers to it.

And here is a picture of one from a monkey. And this is a coronal section. I think you all know a little bit about the anatomy of the brain. If you cut slices this way, it's coronal cuts. If you cut it this way, it's sagittal cuts. OK? So this is a coronal cut. A thin slice. And it shows what the lateral geniculate nucleus looks like when it's cut about in its middle.

And what you can say is a bunch of layers here. There's six layers as labeled here. And the top four layers are called the parvocellular layers. And the bottom two are called the magnocellular layers. And the reason they have these names is because the cells-- this was stained. You can't see it that clearly. But the cells are much bigger in the bottom two layers than in the top four layers. And that's why these are called parvo, and these are called magno. Now, one of the important discoveries that had been made, and I'll belabor that in a minute, is that these top four layers get input from the midget cells of the retina, and the bottom two get input from the parasol cells. And we'll be talking about that a great deal in the next few lectures.

Now, each of these layers gets input from one eye. And there's an alternation. If you go from layer six, this is input for a contralateral line. Ipsilateral. Contralateral. Ipsilateral. Then there is a reversal. And this layer is again ipsilateral. And this layer is contralateral. So that is the basic layout.

And I might as well anticipate to tell you that if you study the receptive field organization of these cells, they are very similar to those that you see in the retina. You have midget cells. Parasol cells. You have center-surround antagonism. So that there is not much of a transform at this level in the brain, as the inputs progress from the retina to the lateral geniculate nucleus. But once we get up to the cortex, we will see many major transforms that I will come to shortly.

Now, the layout of the lateral geniculate nucleus varies quite a bit from species to species. In the market we have this kind of arrangement. And this is very similar to the kind of arrangement we have in our own heads. But then if you look at-- well, let me make one more point here about this.

There's a beautiful experiment that's been done to confirm that indeed the inputs to the parvo and magnocellular layers are coming from different cells in the retina. And so an experiment was done in which a labeling material was put into either the parvocellular layers or the magnocellular layers, as you can see here on the right side. Then those labeling substances were transported back to the retina. And they labeled the cells that project to the axons into these two regions.

And what you can see here is that these are smaller cells that project to the parvocellular layers, and much bigger cells that project to the magnocellular layers. And these correspond, as other studies have shown in more detail, to the parasol cells. And these correspond to the midget cells.

So that's an important distinction here of the inputs. And the other thing that's important to realize here is that we talked about how the eye is organizing, and the way it projects to the nasal and temple parts of the retina. The left lateral geniculate nucleus sees the right half of the visual field, and right geniculate sees the left half of the visual field.

Now, let me say just a few words about the fact that there are different kinds of geniculates. But they're not that different. So here's an example of a tree shrew lateral geniculate nucleus. It's shaped differently, but it still has these nice six layers. And the bottom two layers again are magnocellular, and the top four layers are essentially parvocellular.

But then when recordings were made, an interesting specialization was seen in this animal, which was not that obvious in others. If you look at this here, examine the question to what degree are the cells in these top four layers on or off cells. And what you see here, that when you look at layers three, four, five, and six, there's a huge distinction here. Virtually all the cells in three and four are off cells, and all the cells in five and six are on cells. So there's a specialization in the layers as to whether the input is from the on cells or off cells.

And yet another animal has a different kind of arrangement. I'm not going to go into more details about this, but again there's six layers here. But here they don't seem to have a distinction between magno and parvocellular layers. Instead we have two layers, five and four, where the cells are very small. And the rest of the layers are quite large. This is the galago. All right? Now most thoroughly studied had been the cat, and the monkey. And that's the kind of stuff we are going to talk about more as we progress in the course. All right. So that in a very summary form is the essence of what that lateral geniculate nucleus is like.

Now, what I want to do is I'm going to have a quick summary of what I've covered in the first session. And here, first of all again, is a drawing of what the cells look like in the retina. These are the photoreceptors. Remember the light is coming in from the bottom up. OK? And so if you look at that, we find that all the photoreceptors hyperpolarize the light, and they produce only graded potentials.

My last important fact is that all these cells use glutamate as the neurotransmitter that is released at the bottom here of these cells, which then innervates the subsequent elements in this retinal picture here, which are the horizontal cells and the bipolar cells. So if you look at those, the horizontal cells are all hyperpolarized to light just to the photoreceptors. And they also only produce graded potentials.

But then when we proceed, and look at the bipolar cells, there are two major classes of bipolar cells that I want to deal with, and more. These are the important ones from our point of view. The so-called on and off bipolars. And that arises, because the on bipolars have signed inverting synapses with which they connect to the photoreceptors. Whereas the off bipolars have signed conserving synapses.

So the way to think of this, if the photoreceptor hyperpolarizes and depolarizes, going up and down as the light goes in and out, an off cell will mimic this, and go like this. But the on cell will do the opposite. OK? So we have created in the retina, I shouldn't say we, but evolution has, from a single-ended system since they were hyperpolarized, a double-ended system at the level of the bipolar cell. And that's what then creates these famous on and off cells that we are going to talk about in a lot more detail the next time to try to figure out why do we have these cells.

All right. So that then is a summary of the upper layers. And then when they come down to the amacrine cells, some of these amacrine cells produce action potentials. Some don't. There are many different kinds, as I mentioned the last time. There are more than 20 different kinds. Some are on. Some are off. Some are on-off. And some don't even give you action potentials. Then finally when you come down to the ganglion cells, they all give you action potentials. And the major two classes that we are going to deal with a lot, even though there are many more, we'll talk about some of them later, are the midget cells and the parasol cells. OK.

So then to have an overall summary of what we covered the last time, first of all, we had the right brain receives input from the left visual hemifield, and the left from the right hemifield. That, you understood when I explained to you what the wiring is like in the retina. And then I pointed out, just seeing that, we have these five major classes of retinal cells. We have the photoreceptors themselves, which are the rods and the cords. And then we have the horizontal cells, the bipolar cells, the amacrine cells, and the retinal ganglion cells. Hopefully, by repeating this stuff, eventually this will stick into your brain.

All right. So then the receptive fields, or the retinal ganglion cells, sometimes referred to as RGCs, have antagonistic centers around organization. And then when we look at the question of adaptation, you can explain why on earth did this evolve. This complex arrangement. Then there are several classes of retinal ganglion cells. We have talked about the on and off, the midget and parasol, and we will later on talk about several other classes.

Now, all photoreceptors and horizontal cells hyperpolarize the light. At that level you have a single-ended system. But then when you come to the bipolar cells, about half of them are hyperpolarizing, and half of them are depolarizing. And the opposition arises, because the on bipolar cells have an inverting synapse, which is due to their intersynaptic junction where their receptors are made of molecules, which is called the mGluR6. And we'll talk about that in more detail next time.

The action potentials in the retina are generated by only amacrine cells and retinal ganglion cells. The lateral geniculate nucleus that we have talked about is a laminated structure, but desegregating lamina varies with species just as I pointed out to you. The parvocellular layers receive input from the midget cells, and the magnocellular layers from the parasol cells in the monkey, and in the human. OK? The inputs in the left and right are segregated into lamina, as I have already pointed out to you.

Lastly, the receptive few properties of lateral geniculate cells are similar to those that you see in the retina ganglion cells. You don't have any major transforms. OK. So that's then what we have an initial understanding of when it comes to the retina, and the lateral geniculate nucleus. We are now going to proceed, hang on for a minute, to the visual cortex. All right. So we are going to talk about the visual cortex.

OK. So the first thing we are going to talk about that pertains to the visual cortex is V1, this initial area that gets most of the input directly from the lateral geniculate nucleus. And we're going to look at this first from an anatomical point of view. And then we're going to look at it progressively more from a functional point of view. OK.

So here once again is a monkey brain. You're going to see this monkey brain over and over again, and gradually will become familiar with it. I want to point out to you that this here is a central sulcus. Just like we have a central sulcus, so do monkeys. And then this area here is called the lunate sulcus. And then in front here, we are going to talk about this region. It's called the frontal eye fields. There are two. The soft side, the principalis, and the arcuate. So you're going to see this over and over again. There's not too many things to remember here. But it will grow, because I will have to talk about more and more areas as we progress in the course.

Now, here is area V1. And as I pointed out to you before, this region is in the monkey lissencephalic, meaning it's mostly flat. And because of that, its spatial layout is fairly easy to understand. So the first thing you want to do is to understand this spatial layout of the structure. And to do that people have done all sorts of experiments.

Here is one that has done the spatial layout. Initially, the way this was done was kind of a difficult undertaking. They used single microelectrodes, and they would systematically move the microelectrodes here, and all across. And then they would map out where the receptive field was. OK? So if they did that, they found that in this region here the fovea is represented. And then as you progress towards the center of the brain, what you find is the center of the hemisphere I should really say. You progress to about 80 degrees out.

And this is the horizontal meridian. And this below here, and above it, is the vertical meridian that goes around. OK? So what is represented, as you had already seen, you're representing each hemisphere, half the visual field. So what you have is this. OK? This is the horizontal meridian. This is the upper part. This is the lower part. OK? But that is inverted on the retina, as well is inverted on the visual cortex.

All right. So that's the basic layout. But now what happened, as it always happens in science, new methods were developed to study this in more detail, and more reliably. And one of the methods that was developed is to do a combined action, and then chemical experiment, in which the eyes were stimulated. Then they were fixed, and one of the neurons were active. Took up the substance that was injected into the bloodstream, which then could be labeled subsequently.

So I'm going to show you what this arrangement is. This is what the monkey is looking at. This is the half of the visual field that is being studied, because you're looking at the contralateral hemisphere. And then what is being done in these experiments is that you alternate the black and white stripes back and forth. So you can see if that's sort of a flicker. And you keep doing this for an hour or two. And as a result of this, the cells that are activated by this take up this substance that is being used, which is called 2DG, 2-Deoxyglucose. And then subsequently you can label that.

So now what we are going to do, we can look at what the label looks like. And here is this beautiful example of this work. Here is a foveal area. This is about 7 or 8 degrees out. OK? So if you go back-- oops. Sorry. This outermost half disk is one that you see here. OK? So now we have a very clear understanding, which we can do quantitatively, of what the spatial layout is of the visual field on the retinal surface.

And one important point that I want you to remember today, and we will discuss it again when we talk about visual prosthesis, is that much more brain area is devoted to central vision, because of course the many more cells in the fovea than in the periphery. And the cortex itself is of constant thickness. It's about roughly 2 millimeters in thickness. And so you need to give it more space in this lissencephalic brain to accept all that input from the foveal region. And so much more area is devoted to foveal vision than to peripheral vision.

All right. So that then is the spatial layout. And then if we look at a cross section of the visual cortical regions here, this includes several cortical areas, this one right here, and this region here is so-called area V1. Now, an interesting discovery was made by a person called Gennari, who discovered when he used this kind of Nissl stain that when you look at V1, there seems to be almost what looks like an extra stripe, which gets labeled. And as soon as we get here, as you can see, this point, it stops here. And that defines, this extra layer, this stria of Gennari defines area 17 anatomically. If you get here, you suddenly get to area V2, so V2 starts here. So even with a simple anatomical technique, you can tell what is area V1, and area V2, and so on.

OK. So now, if we take a cross section of the visual cortex, which I've said is about 2 millimeters in thickness, OK? People have divided this region, this so-called gray matter. Why is it called gray matter? It's called gray matter, because most of the cells there are not coded. Whereas if you go below, you have all the coded fibers that are coming in.

So this gray matter then has been divided into six subdivisions. Layers one through six. In this case from the top down instead of in the genicular point, so from the bottom up. And these layers, subsequently, were realized they're more than six layers, really. And that's why it became 4A, 4B, 4CF, or 4C-beta. Now the 4C-alpha and beta is a region that gets a lot of input directly from the lateral geniculate nucleus. And what the nature of that input is, I'm going to show you here.

Once again, here's the lateral geniculate nucleus. The six layers. The four parvocellular, and two magnocellular layers. And then what we can do, is we can trace how that connect. This has been done both physiologically and anatomically. And it shows that the four top layers terminate for the most part, almost exclusive, but not 100%, but pretty closely into layers of so-called 4C-alpha. Whereas when you come to the two bottom layers, OK? The so-called magnocellular layers, they terminate in 4C-beta. So some of these things are almost like an inversion from here to there. OK?

And then another thing we haven't talked about before is that also some cells, even though they're not numerous, that reside in between the layers of the lateral geniculate nucleus, are called the koniocellular cells. But you don't have to worry about that now. You don't have to remember that at this point. But we'll talk about it later.

But when you do look at them, what you find is that they project into the upper part of the visual cortex, both from the parvocellular interlaminar layers, and the magnocellular interlaminar layers. So as you might expect, things are complicated when it comes to the brain. And there are just all kinds of connections, and you're trying to make sense of it, and understand the reasons as to why we have these connections, and what the functions are.

All right. So now, we are going to take a big step forward, and begin to look at the functional aspects of area V1. OK? Oops. Sorry. Hang on. Here we are. We are going to look at the receptive field organization of cells. So how do you do that? What you do is you can stick either a single electrode, or multiple electrodes into the visual cortex. And then you can map out the receptive fields to see how they respond. OK?

Now, this was an interesting story in the beginning when this kind of search went on, because there were two major groups that were trying to understand what the visual responses are like in the visual cortex. And this was not even that long ago. This was in the 1930s, '40s, and '50s. So what was done in two areas, and one was in Germany, what they did there, they kind of followed in the spirit of Keffer Hartline. And they shone light into the eye, diffused light. And they recorded from V1, and they couldn't derive any cells to the extent that some people said, I think people have made a major mistake. That stuff back here, that's not a visual area. It's something else.

But then, pretty much at the same time, another group, initially at Johns Hopkins, and subsequently at Harvard, did similar experiments. But what happened was, it's almost an amusing story. The people who did that were Hubel and Wiesel. And when they first went to Johns Hopkins, they worked with a person's name I've mentioned before, I think, Kuffler, who discovered center-surround antagonism. And Kuffler did experiments similar in technique, in terms of activating the eye, that had been done by Keffer Hartline, which shine light into the eye. Not necessarily diffused light, but spots of light that he would move around on the retinal surface. This was a very complicated piece of equipment.

And so when Hubel and Wiesel went to become postdocs with Kuffler, they said, my god, what is this? We don't know how to work this stuff. Let's do something different, so we can handle it. And so what they decided to do, and this may sound crazy-- Let me explain one more thing. When these experiments were done in the cat, with the way the experiment was done, that the cat was put upside down on a table like this. Like that. And the light was shone into it. OK? With a piece of equipment from above, and you could even look at it through a microscope to see how that light was shone into the eye.

So Hubel and Wiesel said, you can't handle this. But I tell you what we'll do instead. You'll take a bed sheet, and put it up on the ceiling. And then we'll take a projector, and move the light around like this. OK? Now, the projector they used was like an old fashioned camera. And you could put a slide in there, and move the slide in and out. OK? And so they would, first of all, just had the whole lights come on. And they kept recording, and they wouldn't get anything. They said, oh my god. Those people in Germany are right. These cells don't respond to light. What's going on?

And then one day, as they were fiddling around with this, Torsten Wiesel pulled out a slide to put a different slide in. And when he pulled that slide out, they displayed the sound of the action potentials. Like that. What on earth was that? And so they did it repeated. [HIGH-PITCHED BEEPING SOUND] Like that.

And guess what that resulted in? That finding [INAUDIBLE] logic that resulted in the Nobel Prize he and Hubel got, because they discovered that cells' individual cortex are orientation selective. OK? Meaning that each cell responds to a particular orientation of an edge. And this was an incredible transformation, which we saw in the retina, and what you had seen in the lateral geniculate nucleus.

So what they then did, they became systematic about it, and mapped out the receptive field organization of cells. And they discovered two majors classes of cells, they called the simple cells, and the complex cells. The simple cells were distinctive in the sense, first of all, all these types of cells were orientation specific. I want to find the appropriate orientation by moving the edge around until you get the best response. Then you could do the very small spot. You could activate the cell, but not as effectively as with the ball moving, of course, but still quite well. And they found that there was a center region in this kind of simple cell that gave an on response, and a surround region that gave an off response. And then they found another class of simple cells, in which the left half in this case was on, and the right half was off.

And there were variations of this, I'll mention those in a bit more detail later, indicating that the input from the on and off systems may be separate, or they [? sub ?] some interaction that makes for their specificity for orientation. Now then, they also discovered another class of cells called complex that on the whole tended to have a bit larger receptive fields. And in these cells, even when you use small spots, no matter where you stimulate it, you got both on and off responses intermingled. So you did not have a spatial separation of the on and the off responses. Now what exactly we mean by on and off responses we will discuss for the whole session next time.

OK. So now, let me explain to you how you do these experiments. Once you get a little bit more sophisticated, and instead of just using a projector that you reflect the light with by hand, you can do this on a computer. OK? And that indeed was a big step forward. I would say in the early 1960s, maybe late '50s or early '60s, people became computerized as we had especially here at MIT.

And the first set of computers that were used to enable us to quantitatively measure these attributes in the visual cortex were the so-called PDP-11/10s. Subsequently, PDP-11/20s. Subsequently then PDP-11/34s. Some of you may have heard about these ancient old computers that practically nobody uses anymore, because of all the advances that have been made in computer technology.

So what you did then using a variety of means, I don't have to describe that in detail, is to move a bar of light across the receptive field in different orientations. So here's an example here. You first use very small spots, and you find the receptive field. And then after, because the cells do respond to very small spots of light. But they don't respond at all to larger spots. And once you've found the receptive field, what you can do is you can take a bar, and move it across it in different orientations, like this.

So here's a direct example. Like that. OK? So as soon as it's been across the field, the cell responded vigorously. And so if we then did it systematically, as I've shown here, you can generate an orientation tuning function, it is called. So you can quantitatively establish just how sharply specific a cell like this is to different orientations. So to do that here is an example of that in a second.

Here is an example. What you can see here is when you move the ball down, you get a huge response here. And then we get rapidly progressively less of a response. And then you can establish by taking the half height, or whatever point here, what the width is of this function. And then you can plot that for hundreds of cells to establish just how sharply they are tuned, or how different types of cells, say simple and complex, how sharp they are tuned.

Now, that was only quite amazing. But then if you look at this figure, I want you guys to be my best detectives. I bet you all of you are best detectives, right? So if you are, you have to tell me what else is in this figure that tells you something more about this cell that is not just orientation.

Yes.

AUDIENCE: Also, the direction as well.

PROFESSOR: A-ha. There we go. Best detective. OK. Is that your middle name? Best detective?

AUDIENCE: Sure.

PROFESSOR: OK. Very good. So what you see here, if this cell only were orientation selective, you should get another peak here, because you have the same orientation, but moving upward, like this. OK? But you don't have anything like that. So what does that mean? That means that this particular cell is not only orientation specific, but it is also direction specific. Now that's quite something.

And the overwhelming majority of cells in the visual cortex in V1 are direction selective, as well, in addition to being orientation selective. And then when you go to other areas, as we'll see in just a minute, there are some areas in which virtually all the cells are direction selective. So direction cell activity becomes an inherent central attribute of visual processing, as revealed by the fact that all the cells, I shouldn't say all, but so many of the cells in the visual cortex have that attribute. OK.

So now, what we can do is ask the next question. Now, are there other attributes in the cells of the visual cortex that are selective [INAUDIBLE] some other attribute of the visual scene? So far we have orientation, and we have direction.

So another one that has been studied is spatial frequency selectivity. Now, how do you do that? The most common way this had been done is to use sinusoidal gratings. An example of that is shown here. But you could use also [? square-wave ?] gradings, or sinusoidal gratings work extremely well for a variety of reasons. We will talk about that later.

And what you do then on each trial, as you move this back and forth across the receptive field in the optimal orientation, you can vary the spatial frequency. You can make it extremely high, or extremely low. This is obviously very low. OK? And if you do that systematically once again, using a computer system that can establish to what degree are cells specific to different spatial frequencies. Everybody understand that? OK.

So let me show you what happens. Here is an example of a simple cell, and here's an example of a complex cell. Here we have different spatial frequencies on each line, as specified in here. And it shows that both of these types of cells respond to some in-between level of spatial frequency selectivity, and don't respond at the extremes. So that means that these cortical cells, in addition to being orientation and direction selective, are also spatial frequency selective. And that had lead to many interesting ideas about what is the coding operation in the visual cortex that enables you to see. And that we will discuss some more later.

OK. The other important feature here that is worth mentioning is that simple cells respond selectively to each phase, whereas complex cells respond in a more general fashion. OK? All right.

So now, we can summarize, and I'm going to add a couple more things to it, of what the major so-called transforms are in the visual cortex. And this is remarkable as you go from the geniculate to the visual cortex, and suddenly there are all these transforms that somehow will enable you to see things better. So the first one of course, we talked about a lot, is orientation. Next one is direction. Then spatial frequency. Those are the three I gave you examples of.

But now, one thing I haven't mentioned so far is that you have also a transform in terms of binocularity. As I mentioned to you that already, in the geniculate you have specificity by layers of input from the left and right eyes, which means that those cells in each of those layers are driven only by one eye. OK? They have a monocular. But now when you come up to the cortex, the numerous cells once you get above the major input layers, 4CL from there or below, most of the cells there can be driven binocularly, meaning they get an input from both eyes. And we shall examine what that is.

And another thing that we have that you already must have inferred on the basis looking at the receptive fields, that there is a dramatic ON/OFF convergence. In complex terms, it's so complete that it's totally intermingled. So the cell responds to on and off by the way you present it. The simple cells also respond to on and off, but there's a small spatial separation between the on and off sub region.

Now in addition, that we will encounter also later, some of the cells, not all of the cells, but some of the cells in the visual cortex get a convergent in both the midget and parasol cells, while others get an input only from one or the other. So that further highlights the extensive increase of complexity, and analysis that is being performed in the visual cortex.

OK. So now, to sort of schematize about this, what we can say is the following. We can say here is that we have a cell in primary visual cortex, in V1. OK? And many of these cells, most of them, get an input from both the left and the right eyes, as shown here. They also get an input, many of them, that are convergent from the parasol and midget system.

And then the output of these cells, in some fashion or other, should be able to tell us, either from the same cell or from separate cells that we've examined, tell us about luminance, color, orientation, spatial frequency, depth and motion. So those are some of the major tasks a cortical cell performs, so that you can see. So that then is a very general scheme that we are then going to examine in considerably more detail.

All right. So now, we take another step forward. The question came up. How are these things in general, these attributes and whatnot, arranged in the visual cortex? Is everything just helter skelter, or is there spatial separation among various attributes? And this is yet another major line of research that Hubel and Wiesel had done, for which they got the Nobel Prize, as well as for their other discoveries. I think they got the Nobel Prize in 1983, I believe it was. OK.

So this is called cytoarchitecture. OK? Your ability to understand what the layout is of the functional attributes of the cells in the visual cortex. All right. So let me, first of all, tell you the initial experiment Hubel and Wiesel had done. What they did is they would inject a label into one eye. And then if they waited a week or so, you would get transneuronal transport, meaning that the label would go to the geniculate, and that label would transport to the cells in the geniculate, and then would go up to the visual cortex.

And so when they looked at this, put the input into the left eye, what they found was in the cortex of the cat in this case, you see these alternations, you can label them unlabeled regions. And the labels in unlabeled regions are pretty much equally distributed. The assumption therefore is, and that was proven of course, that if you were to label both sides, then you would get a continuous label. These include areas, or if you labeled the other eye, you would get the dark areas labeled. So this established then that we have an orderly arrangement of, if you will, ocular dominance columns. That's what it is called. Ocular dominance columns.

Now, this is a cross section. So now, what you can do is do a similar experiment. Unless you're doing a cross section, you can take the brain, and make horizontal cuts across it. Now, the brain is slightly curved, of course, depending on the species how much. And so, when you make these very thin cross sections, they will only tell you about that particular cross section. But then you can put them together. You can take a montage.

So let me explain that to you. Here is a single section of the left eye columns that are labeled in the monkey. So in other words to repeat, you inject a label into the left eye. Then that is transported transneuronally to the visual cortex, and this is how it lights up. But now what you can do is you can take each of these sections. There are many sections. I told you the cortex is about 2 millimeters in thickness. OK?

So we can take a huge number of sections across, depending on how thick or thin you want to cut it. And then you can label each of these, and then you can superimpose them. And this is what we call a montage. In this case we have five layers that have been superimposed, so you can see it.

Now, these columns here, so-called, it's a column because it goes through the thickness of the cortex, are then ocular dominance columns. OK? Ocular dominance columns. We will next examine whether they're also columns for orientation. That's why I said orientation. OK.

Anyway, so we have this arrangement. And these often have been called zebra stripes, because it looks like a zebra. And that helps as a mnemonic device to remember this. Now, what you can do is to examine this a bit more systematically. And if you do that, we can draw together a huge section of the visual cortex.

This is the fovea again, in this case. This is about 80 degrees out. And so this is a section that was put together in the Hubel Wiesel Laboratory at Harvard, showing you that the thickness of them is constant throughout. And then to understand what they're talking about in terms of size, here we have David Hubel's thumbprint. OK? So if you look at your thumbprint, the spatial frequency here is about twice roughly, twice that you have here. So again, if you look at your thumb like this, it can give you a sense of how unbelievably fine these columns are in the visual cortex. OK? So that's basically the layout.

So now, the next big question came up in the work that Hubel and Wiesel originally did, was whether you also had columns for orientation. So how do you do that? Well, another technique has evolved, and everything that you discover in this business heavily depends on you being able to come up with a new technique, or somebody else, and use a technique they invented. And when you do that, almost inevitably you can make a major discovery. But if you're not sensitive to new technologies, then it's unlikely that you're going to make a major new discovery. So you've got to keep your nose to the grindstone, and constantly look, and say what is the latest discovery in many techniques. All right.

So if you do that, one of the remarkable techniques that had been developed is to use a substance called 2-deoxyglucose that I'm not going to go into details about it. It's radioactively labeled as well, and what you inject in the bloodstream. And then what happens is that those cells in the brain that are highly active, absorb more of this 2-deoxyglucose than those cells that are inactive. And so if you do that, the experiment you would do to look at the selectivity for orientation, is you take an animal, unless it is paralyzed, and present for an hour vertically oriented set of bars, or signs of gratings that keep moving, and keep moving, and keep moving. And then those cells that are, in this case going this way, have selected the horizontal orientation. Those will take up a lot more of this 2-deoxyglucose, and therefore that would be heavily labeled.

So then the question is, what does that look like? And so if you do that kind of experiment, let me skip that, here's an experiment like that, and this happens to be the tree shrew. It shows that in this case, when you use one particular orientation that you get these stripes, which reflect those cells in the visual cortex in the column arrangement that respond best to that orientation. So therefore, if this is horizontal, the ones in between with these cells are vertical. All right?

So if you then do this systematically in the monkey, you can see exactly what that looks like. And then you can ask the big question that was posed. What is the relationship between orientation selectivity columns and ocular dominance columns? So that was then done by a set of experiments, also in the Hubel-Wiesel Laboratory. And I'm proud to say that the prime person who did that experiment was one of my former students, who got his PhD in my laboratory, Michael Stryker. And so he did these experiments.

And so in a way, the experiment was done to test a hypothesis that was proposed by Huber and Wiesel. And let me interject at this point, that one of the important things to realize is that you do not want to fall in love with your hypotheses, because in most cases the hypothesis that you dream up, so to speak, out of thin air, will end up being wrong when it comes to the brain. So in this case, this also happened.

This is the model they came up with. They proposed that you had what we call an ice cube model, which in one direction specifies orientation columns, and in the other direction, ocular dominance columns. So this is the ice cube model. And now that you had this model, this hypothesis, you could test it. And the way this was tested was to use the same procedures I described to you, but do it in the same brain. All right?

And so here's an example of here we have a part of the brain, again looking at it from the top down, which had been labeled for both orientation and ocular dominance columns. And then you take that same brain, and you label it for orientation columns. So now, if your hypothesis is correct, let me clarify this, if you say that they are right angles to each other, then you would expect that if you have a bunch of orientation columns, the ocular dominance columns should be at right angles to it. Right? And so if you label this, and then can draw it out, you can see whether or not that hypothesis is correct.

And when you did that, here is a real example of what they did. One of these is orientation, and the thick ones and the thin ones is ocular dominant, maybe the other way around. It doesn't matter. But at any rate, if the hypothesis is correct, you'll expect something like what you see here. They're at right angles to each other. OK? But if you look at it carefully, you can see the endless locations where they're not at right angles to each other. All right?

And because of that, even though that's a very attractive hypothesis, people have come up with other hypotheses saying that this hypothesis is really questionable. And an alternative hypothesis that had been proposed arose in part by yet another new discovery that had been made. And this discovery was made by a woman called Margaret Wong-Riley. But this particular picture has been done by processing, again this is a horizontal view looking down at the brain, by Marge Livingstone, who is at Harvard, and had done all the collaborative work with Hubel.

So what you see here, are the so-called cytochrome oxidase patches. OK? Now, when this was first discovered, people said, my god. They had never seen anything like this before. This is the only kind of stain that showed this up. Let me explain to you. This is also an activity label. The cytochrome oxidase. And we selectively label those cells that are most active, rather than those cells which are inactive.

And so this is what these patches look like. And so people kept saying, why do we have this patches? We've never seen them before in the visual cortex. What on earth are they for? What do they tell us about? So all kinds of hypotheses evolved. And people asked the question, since we have this thing, what if we recorded in one of these patches, as opposed to outside of them. And so when this was done by several people-- let me just show you if you have a much larger section.

Here again is a fovea that's about 80 degrees out. This is what these patches look like when you do it on the high contrast. OK? So indeed, they're extremely frequent. They're very orderly. And so the question, of course, is whether these batches are relative to the columns that we talked about. And what was discovered, it if you look at the ocular dominance columns, these patches are always in the middle of a column.

So in other words, to make that clear, if you have here a column like that, the patches would be like this. OK? And the next one over, again the patches would be like that. OK? They would not be between patches. So there's a direct relationship between the ocular dominance columns, and the layout of these patches. OK?

All right. So now, as a result of this work, people have come up with yet, and let me finish, another hypothesis, which is called the radio model. They argued that the cells that are in the center of these patches are largely unoriented, and that the orientation selectivity goes around in a radio fashion, as indicated here in the visual cortex within each of the ocular dominance columns. OK?

AUDIENCE: I'm good now.

PROFESSOR: Are you good now?

AUDIENCE: Thank you.

PROFESSOR: All right. So that's the second model that has emerged. And this model also was highly questioned, because the evidence was not that you had these nice clear radio orientations for orientation selectivity. So then finally another technique had emerged, which was carried out again at Harvard, at Harvard Medical School by a fellow called Blaisdell. He developed a technique that actually other people have used, but then he applied it to the visual cortex, that is called optical recording.

So you could record optically from the visual cortex straight down, and then it could vary orientation, and ocular dominance. And when you did that, this is what a typical example looks like. OK? Those red lines show what the layout is of the orientation selectivity of the cells, and the patches here are those regions where you don't have orientation selectivity. OK?

So this then indicated that you do not have a computer like layout in the visual cortex going one way, and the other way for orientation direction. But you have it a bit messier. And so this particular arrangement then was called, or at least I called it, the swirl model. OK? But actually, to call it a model is incorrect. This is a fact. The others are models. OK?

So now to put it all together, here is the Hubel and Wiesel ice-cube model, where the orientation selectivity [INAUDIBLE] right angles to each other. Then we have the radio model. And finally we have the swirl model, which is the way it really is. All right. So that then in essence tells you about the cytoarchitecture of the visual cortex.

Now we are going to take another big step forward. We are going to look at extrastriate cortex. As I've mentioned to you before, when you look at the visual cortex, it comes actually in many subdivisions, and maybe as many as 30 visual cortical areas, of which we will only talk about a few of them, because it's totally overwhelming. And these areas were presumed to involve, maybe correctly so, initially at least, increasingly complex visual analyses. All right?

Now, one of the prime ideas that, or hypotheses again if you will, to my mind turned out to be incorrect, is that each of these areas in the brain specializes in analyzing a certain aspect of vision. So what are certain aspects of vision? Just a few examples are analyzing color, analyzing shape, analyzing motion, and so on. And so they argue that all these visual areas specialize in one of those. So that was an interesting idea, and so people began to study extrastriate cortex in more detail. And I'm going to say few words about it, but let me go back even further in history.

Let's go back to a time in the early 1800s. There were two very famous people there, which were called Gall and Spurzheim. And they came up with the idea which eventually was called phrenology. How many of know phrenology? Oh, well. My goodness. All of you do. OK. So phrenology, in essence, claimed that there are specific areas in the brain, as shown here, that specialize in certain aspects of the information processing.

And how did they come up with this? The way they came up with it was to palpate the skull. And wherever there were big bulges, they felt that there was a lot of that attribute that a person had. And if it was small, then the person didn't have too much of it. And so they played with it, and played with it, and they came up in an 1812 publication with essentially 35 basic visual attributes of processing in the brain, for the cortex of the brain in humans. And the interesting thing about this is it gives you a sense of history and how much we have changed in our lives, as to what these specialized areas were conceived to be.

Now, to make that a little bit clearer for you, let me stop here for a second. And what I want to do here is I want to enlarge this, so you can see it better.

OK. So here are some of these areas. And if you look at these, you will be almost taken aback as to what the hell the names are here. I mean, some of the common names here that you can see would be, for example, let's see if I can put them in the right order here. Actually, the thing was that they called these not only propensities, but they called them sentiments. So in those days sentiments were very important. There's not much brain allocated to sentiments anymore.

But at any rate, such things that existed as amativeness, cautiousness, benevolence, veneration, wonder and ideality. Those are just a few examples here that you can spot. Amativeness right there. And indeed you can ask the question, really would that much brain be devoted to amativeness? And certainly that's not the case anymore. So maybe our brains are totally different from the way they were back in 1812. I doubt it. And so therefore, I think these things are rather fanciful, and very far-fetched hypotheses. And as I mentioned before, that's par for the course. So many hypotheses end up being wrong, and the best way to overcome that is to do solid experiments to find out what is really going on.

OK. So now, we move on. And we need to understand what the more modern techniques are that enable us to specify things about these higher cortical areas. The first one of these is architectonics. That's obvious what it is. That's simply to look at the brain, and identify the various areas in a systematic fashion.

The second one is to look at connections. You do anatomical studies to determine which areas to connect to what areas in what fashion. Another one is topographic mapping. We talked about that already. One of those was to actually do single cell recording systematically moving the electrode across, or to do the two digit type studies.

Another one we can do is what we call physiological characterization. That's also known to you. When you looked at the cells in V1, you established that they're orientation direction selective. That's your physiological characterization of the cells in V1. And then you can ask the question, well, what about V2, V4, and so on, these other areas. What are the cells like there? What do they respond to well? And so on.

And then, another very important technique is to say, OK, what if we removed a particular area that had been identified. What kind of loss do you have in vision, or in general in the brain? If you move a particular area, what kind of deficit arises? And if that's a specific deficit, you can infer that that particular area plays an important role in the analysis that you can no longer do.

That can also be done instead of making specific lesions. It's less accurate, but you can do it by studying humans who have had various kinds of febrile accidents. And that's one of the things that our former chairman, Hans-Lukas Teuber, has done extensively. Studying people after the Second World War who have sustained specific brain injuries to see what kinds of deficits they had suffered on the basis of that you could infer what various brain areas do.

And the last one here that I want to mention is imaging, of course. We've talked about that. You can present certain specific stimuli to activate the brain. And then once you process that using magnetic resonance imaging, for example, functional magnetic resonance imaging can tell you how important, for example, areas are in recognising faces. And some of that work is being done, actually here, by Nancy Kanwisher, for example, in our department.

OK. So to do this then systematically, you have to have a sort of idea of what kinds of functions do we want to study. And so you want to break it down visual functions. And one way to break them down, I'm not saying this is overall satisfactory to everyone, they can talk about so-called basic visual capacities, and more higher level ones. When you talk about basic visual capacities, they say well how well can you see color. How well can you distinguish differences in brightness? How good are you at seeing basic patterns? Textures? Motion? Depth? OK? So those would be your basic visual functions.

And then when you come to intermediate visual capacities, things become much more complex, of course. You come up with constancy. How come that when I look at something that's nearby, and something that's further way, I can recognize it's the same thing? OK? Or how can we select things in the visual scene? How can we recognize things, just like recognizing faces? How can we make transpositions? How can we make comparisons? And how can we locate things in space? So those would be some of those so-called intermediate visual capacities. And I'm not even going to mention high level visual capacities, because they are even more complicated, and we know even less about them.

OK. So now, let's go and lay out the visual areas. Just the very basics of it, because it's unbelievably complicated. Here we have a human brain. And back when this was put together, this posterior area here, which is the primary visual cortex, called area 17 then, we now call that V1. And 18 would be V2. And 19, V3, and so on. OK.

So now again, we go back to the monkey brain. And when we look at the monkey brain, what you see here is again the central sulcus. You're going to see this over and over again, and it's going to stick into your head. Here's the lunate sulcus. And here is area V1. This is where we had examined the properties of these cells that we had just talked about. Then if you move on, right at the edge here of the lunate, V2 starts, and then goes from under the gyrus. And then in this region here, we have what is called area V4. So those are some of the basic areas. And you are going to see a few things about them.

Now, people have studied this extensively, and they came up with frightening diagrams of this. This is a flattened monkey brain. This is V1. And then you have a whole bunch of areas that are following succession here. I'm not going to label many of these. And we talked about V2, V4. But there are many, many more. I'll bring up a few more in a minute.

And then, if you look at the interconnections that had been made, it's totally frightening. There's so many hundreds, and hundreds, and hundreds of connections going every which way. So it is very difficult to say that particular area receives inputs only from one other area, or something like that. There's just a tremendous amount of interconnections, indicating that any analysis is likely to take place involving thousands, and tens of thousands of thousands of neurons being active in many, many different brain areas.

OK. Now the major cortical visual areas that we shall consider are V1, that we already did, V2, V3, V4 and MT. Then in the temporal region of the brain, we have inferotemporal cortex. And then in the private region, we have the lateral intraparietal area, the ventral area, and MST, which is called the Medial Superior Temporal area. And then lastly, in the frontal lobe, we have the frontal eye fields. Now, we will talk about each of these at various levels in the course. Today, we will just briefly talk about V2, V4 and MT. OK.

So now, a couple of general principles that emerge from this kind of work. First of all is that the size of the receptive fields in these different areas changes dramatically. In the visual cortex there's aptitudes that are very small. I mean, bigger than in the retina, or in the geniculate, but still very, very small. But then when you get to V2, they're about three times bigger on the average in diameter. And when you come to V4, they're huge. OK? And that's true throughout, that the specificity of the location of the receptive fields decreases as you progress up into higher cortical areas.

Now, there's another very interesting fact that had been discovered, which again nobody hypothesized. Namely, that the way these areas are laid out next to each other is not what you would have thought. So let me give you an example. Suppose what we do is we take an electrode, and record in V1, and then we go in progressive steps across V1, where the receptive fields move out from the fovea into the periphery. And if you do that, here's a receptive field of that cell. [INAUDIBLE], right there.

The next one is a little bit further removed. The next one is a little bit further removed. And here's the next one, and that one is there. OK? So what you have is a progression of receptive fields. They get bigger, and they move from the center out.

So now the big question is what happens when you now come in to V2. And think about it for a minute. What do you think? Where do you think the receptive field will be when you just ride across from V1 to V2 in the map that I had shown you before? Well, you will be surprised. I think you will be surprised. You will find that the next receptive field, close as this, is at the same location, almost, very close the same location. And of course, the receptive field is about three times bigger.

Now, what happens when you now move one over? Well, the receptive field comes back towards the center. And again, and again like that. So these areas connect to each other in reverse, so to speak. In other words, if they didn't one would connect to here. Two would connect to there, and so on. But instead, four and five could have short connections. Three and six, longer, and so on.

So we have this very curious arrangement. And to this day I don't have a clue as to why in the course of evolution this happened. I mean, there all kinds of hypotheses. This may take less elaborate wiring. Not as long wiring overall, or something, because if it were the other way, then all these wires would be long. So that could be a reason, but one is not sure. And to my knowledge, no experiment has yet been done to truly explain why we have this curious arrangement.

Now, when you look at area V2, we have yet another very interesting factor. When you come back to area V1, this would use V1. We have those famous cytochrome oxidase patches. But then when you get to area V2, instead of what you have, you have these elongated bars. OK? And if you look closely, you can see that there's a tendency for the bars to go from thick to thin with an inter-bar area.

And so when this was discovered, people began to record to say why is this different from here. What does it signify? And so when they did that systematically, they came up with a model, which may not be entirely correct. But the claim was that the thin stripes in V2 get inputs predominantly from there patches that you have in V1. And the thick stripes get input from the so-called parasol system, meaning that they would be heavily involved in things like what the parasol cells do, namely play an important role in emotion perception. And the interstripes here called, or the pale stripes sometimes called, they get input in this particular model from the orientation specific cells. OK?

So now, once this was done and proposed, people begin to do experiments to record in the thick and thin stripes of V2 to see what the properties are of the cells there, recognizing of course that the size of the receptive fields is uniformly about three times bigger than in V1. So when they did that, they looked at orientation. And they found that in V2, most of the cells in the thick stripes are orientation specific, but also many in the thin stripes, and many also in the pale stripes. So there didn't seem to be a huge distinction.

Then when it came to end stopping, I didn't talk about that. Let me mention what end stopping means. If you take a bar of light, and you move it across a receptive field, you get a vigorous response when it's a fairly short bar. But then when you make the bar a lot longer, you get less of a response, because of some surrounding [INAUDIBLE]. So that's what's called, by Hubel and Wiesel, end stopping.

And so that attribute is one that is again fairly similar in the three areas. When you come to color, in the thin stripes there is much more color sensitivity than the fixed stripes. When it comes to direction, again not that much difference. Maybe more in the thick stripes. Disparity, more in thickness. Disparity refers to depth perception, as we have talked about. So the prime message here is that you do not have a complete clear separation of function in those three stripes that had been identified in V2, virtually inputs from V1.

So now, we are going to move on to V4. And when you talk about area of V4, we'll talk about that again in more detail later on. We have a huge increase of complexity of the response properties of cells. And here is area V4, as you can see. A lot of recording has been done in this area. Hundreds of papers have been published. And I can conclude on the basis of that.

First of all, here the receptive fields are even a lot bigger than in V2. And the receptive field properties are far more complex than in either V1 or V2. And the response properties are dynamic. If the monkey is paying attention, or is looking for something, the cells will find a lot more than when he's not looking for something. And yet another discovery was made that the amount of activity is also modulated by how you move your eyes as you're looking around in the visual scene.

And then, an important conclusion from that is that this is not just a color area. And the reason I mention that is because a good many years ago an experiment was done in England. I shan't name the person who was doing it, who claimed that this is a color area. Now, one of the reasons he made that claim was that he removed area V4, or studied humans who lacked area V4. And he found that they had difficulties in telling colors.

Well, that was nice. But the problem was, and this is a good lesson for all of us, that the only thing he tested was for color, because that was his hypothesis. Had he tested for a multitude of other functions, he would have found that area V4 has much more significant deficits when it's removed for many other visual functions than for color. So again, we'll talk about that input with more detail later on when we talk about very specific processing of digital attributes.

Now then the next areas I want to deal with, the last areas we are going to discuss, are going to be areas MT and MST. MT stands for Middle Temporal area. And MST stands for Middle Superior Temporal area. And again, to look at the brain, what you can see here is a superior temporal sulcus. And if you went into it, it's about 13 millimeters deep. On the posterior side, we have area MT. And on the interior part of it we have area MST.

Now, this is a remarkable area. An incredible amount of work had been done on it. And the major discovery that had been made is that the cells in MT and MST respond predominantly to direction. Virtually, all the cells are direction specific. And also, this is in part believed to be due to the fact that this area gets half their input from the parasol system.

So to look at this in more detail, here is a receptive field, quite large and empty. And if you move a bar of light across in this direction, you get a huge response. This is the cumulative response histogram. But if you move it in the opposite direction, you won't actually get an inhibition. Tremendous direction specificity. And you get the same in MST, but there the receptive fields are just gigantic. But again, direction specificity is just as specific as it is in MT. So these two areas play a central role in motion analysis, as we shall see in more detail later on.

All right. So now, if you look at the spatial layout of this, what you do is you can move an electrode into the area. And so you move it across the represented area by going into the sulcus. And what you can see here is it's a systematic progression of directions as you move the electrodes. Here the distance is expressed in micrometers. OK?

Now, if you map that out you can create a layout of the area, area MT. And what you can see is there's a systematic kilometer arrangement, or different direction specificities. So this is a general principle, by the way, of the way the cortex is organized. It first actually was discovered by Mountcastle in the somatosensory system. OK? Showing that there's a kilometer arrangement there, and then subsequently it was shown that this is also true for many other areas, including vision and audition.

All right. So now, the last thing I want to mention very briefly, we'll get back to it, is so-called inferotemporal cortex. And here once again, we have a map of the monkey cortex. And this down here is a temporal area here. And this is where the inferotemporal cortex resides. And this has been studied extensively by people like Charlie Gross, and more recently with imaging techniques by people in the department here. It was shown that this area has a lot to do with object recognition, face recognition. A very complex high level area. OK.

So now, I want to summarize what I have covered today. First of all, the contralateral visual hemifield is laid out topographically in V1 in each hemisphere. You know that by now, cold.

Secondly, the major transforms in V1 are orientation, direction, spatial frequency, selectivity, binocularity. You have an ON/OFF convergence, and you have also in many cells a convergence of the midget and parasol systems. Then V1 is organized in a modular fashion. Another way to put it is that you have a kilometer organization, and that we talked about three models that had been proposed. One of them, the original one by Hubel and Wiesel, is the ice cube model. Then we had the radio model, and the swirl model. And I pointed out to you that the swirl model is not really a model. It's a fact. That's how it's laid out. OK.

And then I said also that there are more than 30 visual areas that make more than 300 interconnections. Extrastriate areas do not specialize in any one single function, contrary to what had been very, very popular, maybe even 10 or 15 years ago. The receptive field's size in neurons increases greatly in progressively higher visual areas. That is a very highly solid fact.

Then, area MT is involved in the analysis of motion. And as we shall see later on, it also contributes to perception of depth, and to flickering stimuli. Area V4 engages in many aspects of visual analysis, and neurons have dynamic properties. Attention and eye movements modulate the way those cells respond. And then in inferotemporal cortex, high level visual analysis takes place that includes object recognition, and therefore also the recognition of faces.

So that then summarizes what I had to say. And to conclude in general, we can say that the cells in the cortex have one very important increase in their mode of functioning compared to lower areas, in that the cells are multifunctional. So any one cell can tell you information already in V1 about direction, as well as orientation and spatial frequency.

So if one cell can do many, many different things then that's a good thing, because if each cell in the brain specialized in one thing, it used to be the hypothesis that was called your famous grandmother cell. That there was a cell in the brain that represented your grandmother. Now, if that were the case, that individual cells were future selective, you would need a brain as big as this room to accommodate the ability to what you can do that you can do anyway.

So cells in the brain are multifunctional. They carry out many, many different analyses, just like when computers do complex mathematical analyses. And because of that it's very difficult, of course, especially when you study with high visual areas, to learn just how these cells function, because to understand that you would need to record at the same time from virtually all the cells in an area to see what each of them does, from which you can derive what the actual analysis is.

So indeed, we have a very complex task ahead of us in trying to understand how this very complex interaction among neurons eventually results in your ability to analyze various aspects of the visual scene. And that's true not only for vision. It's true for many other areas. In vision it's very, very complex, because you have all these incredible number of cells. As I mentioned before, you have more than a million retinal ganglion cells that project from the one eye, from each eye, into the brain. And then this multiplies in these higher visual areas. Then you have millions and millions, billions of cells to perform all of these analyses.

Now by contrast, when you look at the auditory system, and I'm not saying this to belittle it, I'm just saying it's probably easier to understand it, because in the auditory system the fibers that project from the cochlear nucleus amount to about 30,000 in each cochlear to the central nervous system. So we're talking about 20, 30-fold higher level of number of neurons involved in the visual system. And that's of course because vision for us is just a very, very important attribute, perhaps more so than many other attributes that we have in the nervous system. OK.

So that's what I have to cover today. And then next time, you're going to sort of move back. We're going to talk about the on and off channels to try to understand why on earth did they evolve. OK?