19. Neural Integrators

• Download English-US transcript (PDF)

MICHALE FEE: Today we're going to continue talking about the topic of neural-- recurrent neural networks. And last time, we talked about recurrent neural networks that give gain and suppression in different directions of the neural network space. Today we're going to talk about the topic of neural integrators. And neural integrators are currently an important topic in neuroscience because they are basically one of the most important models of short-term memory.

So let me just say a few words about what short-term memory is. So and to do that, I'll just contrast it with long-term memory. So short-term memory is memory that just lasts a short period of time on the order of seconds to maybe a few tens of seconds at most, whereas long-term memories are on the order of hours, or days, or even up to an entire lifetime of the animal.

A short-term memory has a small capacity, so just a few items at a time you can keep in short-term memory. The typical number would be something like seven, the classic number, sort of seven plus or minus two. You might have heard this, so just about the length of a phone number that you can remember between the time you look it up in the-- well, you know, we all have phone numbers on speed dial now, so we don't even remember phone numbers anymore. But in the old days, you would have to look it up in the phone book and remember it long enough to type it in.

OK, whereas long-term memories have very large capacity, basically everything that you remember about all the work in your classes that you remember, of course, for your entire life, not just until the final exam. Short-term memories are thought to have an underlying biophysical mechanism that is the persistent firing of neurons in a particular population of neurons that's responsible for holding that memory, whereas the biophysical mechanism of long-term memories is thought to be physical changes in the neurons and primarily in the synapses that connect neurons in a population.

So let me just show you a typical short-term memory task that's been used to study neural activity in the brain that's involved in short-term memory. So this is a task that has been studied in nonhuman primates. So the monkey sits in a chair, stares at the screen. There is a set of spots on the screen and a fixation point in the middle, so the monkey stares at the fixation point.

One of those cues turns on, so one of those spots will change color. The monkey has to maintain fixation at that spot. The cue turns off then. So now the animal has to remember which cue was turned on. And then some delayed period later, which can be-- it's typically between three to six or maybe 10 seconds, the animal-- the fixation cue goes away, and that tells the animal that it's time to then look at the cued location.

And so in this interval between the time when the cue turns off and the animal has to look at the location of that cue, the animal has to remember the direction in which that cue was activated, or it has to remember the location of that cue.

Now, if you record from neurons in parts of the prefrontal cortex during this task, what you find is that the neural activity is fairly quiet during the precue and the cue period and then ramps up. The firing rate ramps up very quickly and maintains a persistent activity during this delay period. And then as soon as the animal makes a saccade to the remembered location, then that neural activity goes away because the task is over and the animal doesn't have to remember that location anymore.

So that persistent activity right there is thought to be the neural basis of the maintenance of that short-term memory. And you can see that the activity of this neuron carries information about which of those cues was actually on. So this particular neuron is most active when it was the cue in the upper-left corner of the screen that was active, and that neuron shows no changes in activity when the cued location shows no change in activity during the memory period, during the delay period when the cued location was down and to the right.

So this neuron carries information about which cue is actually being remembered. And of course, there are different neurons in this population of-- in this part of prefrontal cortex. And each one of those neurons will have a different preferred direction. And so by looking at a population of neurons then during the delay period, you could figure out and the monkey's brain can remember which of those cues was illuminated.

OK, so the idea of short-term memory is that you can have a stimulus that is active briefly. And then for some period of time after that stimulus turns on, there is neural activity that turns on during the presentation of that stimulus and then stays on. It persists for tens of seconds after the stimulus actually turns off. So that's one notion of short-term memory and how neural activity is involved in producing that memory. And the basic idea here is that that stimulus is in some way integrated by the circuit, and that produces a step in the response. And once that stimulus goes away, then that-- the integral of that stimulus persists for a long time.

All right, now, short-term memory and neural integrators are also thought to be involved in a different kind of behavior. And that is the kind of behavior where you actually need to accumulate information over time. OK, so sometimes when you look at a stimulus, the stimulus can be very noisy. And if you just look at it for a very brief period of time, it can be hard to figure out what's going on in that stimulus.

But if you stare at it for a while, you gradually get a better and better sense of what's going on in that stimulus. And so during that period of time when you're looking at the stimulus, you're accumulating information about what's going on in that stimulus. And so there's a whole field of neuroscience that relates to this issue of accumulating evidence during decision-making. OK, so let me show you an example of what that looks like.

So here's a different kind of task. Here's what it looks like for a monkey doing this task. The monkey fixates at a point. Two targets come up on the screen. The monkey at the end of the task will have to saccade to one or the other of those targets depending on a particular stimulus. And a kind of stimulus that's often used in tasks like this is what's called a "random dot motion stimulus."

So you have dots that appear on the screen. Most of them are just moving randomly, but a small number of them move consistently in one direction. So for example, a small number of these dots move coherently to the right. And if the motion stimulus is more to the right, then the monkey has to then-- once that motion stimulus goes away, the monkey has to make a saccade to the right-hand target.

Now, this task can be very difficult if a small fraction of the dots are moving coherently one way or the other. And so what you can see is that the percentage correct is near chance when the motion strength or the percent coherence, the fraction of the dots that are moving coherently, is very small. There's almost a-- there's a 50% chance of getting the right answer. But as the motion strength increases, you can see that the monkey's performance gets better and better. And not only does the performance get better, but the reaction time actually gets smaller.

So I'll show-- I found a movie of what this looks like. So this is from another lab that set this up in rats. So here's what this looks like. So the rat is poking its nose in a center port. There's the rat. There's a screen. There's a center port right in front of it that the rat pokes its nose in to initiate a trial. And depending on whether the coherent motion is moving to the right or left, the rat has to get food reward from one or the other port to the left or right. So here's what that looks like.

[VIDEO PLAYBACK]

[BEEP]

 

[CLINK]

[BEEP]

 

[CLINK]

 

[BEEP]

 

[CLINK]

 

[CLINK]

[BEEP]

 

[CLINK]

So this is a fairly high-motion coherent stimulus, so it's pretty easy to see. But and you can see the animal is performing nearly perfectly. It's getting the right-- it's making the right choice nearly every time. But for lower-coherence stimuli, it becomes much harder, and the animal gets a significant fraction of them wrong.

[END PLAYBACK]

OK, all right, I thought that was kind of amusing. Now, if you record in the brain in-- also in parts of frontal cortex, what you find is that there are neurons. And this is data from the monkey again, and this is from Michael Shadlen's lab, who's now at Columbia. And what you find is that during the presentation of the stimulus here, you can see that there are neurons whose activity ramps up over time as the animal is watching the stimulus.

And so what you can see here is that these different traces, so for example, the green trace and the blue trace here, show what the neurons are doing when the stimulus is very weak. And the yellow trace shows what the neurons do when-- or this particular neuron does when the stimulus is very strong. And so there's this notion that these neurons are integrating the evidence about which way this-- these random dots are going until that activity reaches some sort of threshold.

And so this is what those neurons look like when you line their firing rate up to the time of the saccade. And you can see that all of those different trajectories of neural activity ramp up until they reach a threshold at which point the animal makes it's choice about looking left or right. And so the idea is that these neurons are integrating the evidence until they reach a bound, and then the animal makes a decision.

The weaker the evidence is, the weaker that evidence accumulates. The more weaker the coherence, the more slowly the evidence accumulates and the longer it takes for that neural activity to reach the threshold. And so, therefore, the reaction time is longer. So it's a very powerful model of accumulat-- evidence accumulation during a decision-making task.

Here's another interesting behavior that potentially involves neural integration. So this is navigation by path integration in a species of desert ant. So these animals do something really cool. So they leave their nest, and they forage for food. But they're foraging for food. It's very hot. So they run around. They look for food.

And as soon as they find food, they head straight home. And if you look at their trajectory from the time they leave food, they immediately head along a vector that points them straight back to their nest. And so it suggests that these animals are actually integrat-- look. The animal's doing all sorts of loop-dee-doos, and it's going all sorts of different directions. You'd think it would get lost. How does it maintain? How does it represent in its brain the knowledge of which direction is actually back to the nest?

One possibility is that it uses external cues to figure this out, like it looks at the-- it sees little sand dunes on the horizon or something. You can actually rule out that it's using sensor information by after the point where it finds food, you pick it up, and you transport it here to a different spot. And the animal heads off in a direction that's exactly the direction that would have taken it back to the nest had it been in the ori-- in the location before you moved it.

So the idea is that somehow it's integrating its distance, and it's doing vector integration of its distance and direction over time. OK, so lots of interesting bits of evidence that the brain does integration for different kinds of interesting behaviors.

So today I'm going to show you some-- another behavior that is thought to involve integration. And it's a simple sensory motor behavior where it's been possible to study the circuitry in detail that's involved in the neural control of that motor behavior. And the behavior is basically the control of eye position. And this group, this was largely work done that was done in David Tank's lab in collaboration with his theoretical collaborators, Mark Goldman and Sebastian's Seung. OK, so let me just show you this little movie.

[VIDEO PLAYBACK]

 

OK, so these are goldfish. Goldfish have an ocular motor control system that's very similar to that in mammals and in us. You can see that they move their eyes around. They actually make saccades. And if you zoom in on their eye and watch what their eyes do, you can see that they make discrete jumps in the position of the eye. And between those discrete jumps, the eyes are held in a fixed position.

OK, now if you were to anesthetize the eye muscles, the eye would always just sort of spring back to some neutral location. The eye muscles are sort of like springs. And in the absence of any motor control of any activation of those muscles, the eyes just relax to a neutral position. So when the eye moves and it's maintained at a particular position, that has-- something has to hold that muscle at a particular tension in order to hold the eye at that position.

[END PLAYBACK]

So there are a set of muscles that control eye position. There's a whole set of neural circuits that control the tension in those muscles. And in these experiments, the researchers just focused on the control system for horizontal eye movements, so motion, movement of the eye from a more lateral position to a more medial position or rotation, OK, so eye posi-- horizontal eye position.

And so if you record the position of the eye, and look at-- this is sort of a cartoon representation of what you would see-- you see that the eye stays stable at a particular angle for a while and then makes a jump, stays stable, makes a jump, and stays stable. These are called "fixations," and these are called "saccades."

And if you record from motor neurons that innervate these muscles, so these are motor neurons in the nucleus abducens, you can see that the neural activity is low, the firing rate is low when the eyes are more medial, when the eyes are more forward. And that firing rate is high when the eye is in a more lateral position because these are motor neurons that activate the muscle that pulls the eye more lateral.

Notice that there is a brief burst of activity here at the time when the eye makes a saccade to the-- into the more lateral direction. And there's a brief suppression of activity here when the eye makes a saccade to a more medial position. Those saccades are driven by a set of neurons, by a brain area called "saccade burst generator neurons." And you can see that those neurons generate a burst of activity prior to each one of these saccades.

There are a set of neurons that activate bursts-- activate saccades in the lateral direction, and there are other neurons that activate saccades in the medial direction. And what you see is if you-- is that these saccade burst neurons are actually-- generate activity that's very highly correlated with eye velocity.

So here you can see recording from one of these burst generator neurons generates a burst of spikes that goes up to about 400 hertz and lasts about 100 milliseconds during the saccade. And if you plot eye velocity along with the firing rate of these burst generator neurons, you can see that those are very similar to each other. So these neurons are generating a burst, drives change in the velocity of the neurons of the eye.

OK, so if we go from neurons that have activity that's proportional to position, and we have neurons that have activity that's proportional to velocity, how do we get from velocity? So the idea is that you have burst saccade generator neurons that project to these neurons that project to the muscles. You have to have something in between.

If you have neurons that encode velocity and you have neurons that encode position, you need something to connect those to go from velocity to position. How do you get from velocity to position? If I have a trace of velocity, can you calculate the position by doing what?

AUDIENCE: Integrating.

MICHALE FEE: By integrating. So the idea is that you have a set of neurons here. In fact, there's a part of the brain, and in the goldfish it's called "area one," that take that burst saccade generator neuron burst, integrate it to produce a position signal that then controls eye position. All right, so if you record from one of these integrator neurons while you're watching eye position, here's what that looks like.

[VIDEO PLAYBACK]

 

And so here's the animal's looking more lateral to the nose. The goldfish's mouth is up here. So that's more lateral. That's moving more medial there, more lateral, more--

[END PLAYBACK]

OK, so this neuron that we were just watching was recorded in this area, area one. Those neurons project to the motor neurons that actually innervate the muscles to control eye position. And they receive inputs from these burst generator neuron. OK, so if you look at the activity of one of these integrator neurons, that's a spike train during a series of saccades, and fixations is a function of time. This trace shows the average firing rate of that neuron.

This is just smoothed over time, so you're just averaging the firing rate in some window. You can see that the firing rate steps up, that the firing rate jumps up during these saccades and then maintains a stable, persistent firing rate. So the way-- think about this is that this persistent firing right here is maintaining a memory, a short-term memory of where the eye is, and that sends an output that puts the eye at that position.

OK, and so just like we described, we can think of these saccade burst generator neurons as sending an input to an integrator that then produces a step in the position, and then the burst generator input is zero during the [INAUDIBLE]. So the integrator doesn't change when the input is zero. And then there's effectively a negative input that produces decrement in the eye position.

OK, we started talking last time about a neural model that could produce this kind of integration. And I'll just walk through the logic of that again. So our basic model of a neuron is a neuron that has a synaptic input. If we put a brief synaptic input, remember we described how our firing rate model of a neuron will take that input, integrate it briefly, and then the activity, the firing rate of that neuron will decay away.

So we can write down an equation for this single neuron, tau dv/dt is equal to minus v. That's due to this intrinsic decay plus an input. And that input is synaptic input. But what we want, a system where when we put in a brief input, we get a persistent activity instead of a decaying activity. And I should just remind you that we think of this intrinsic decay and this intrinsic leak as having a time constant of order 100 milliseconds.

And I should have pointed out actually that in this system here, these neurons have a persistence of order of tens of seconds. So even in the dark, the goldfish is making saccades to different directions. And when it makes a saccade, that eye position stays stable for-- it can stay stable for many seconds. And if you can do this in humans, you can ask a person to saccade in the dark and try to hold their eyes steady at a given position, and a person will be able to saccade to a position.

Just you can imagine closing your eyes and saccading to a position. Humans can hold that eye position for about 10 or 20 seconds. So that's sort of the time constant of this integrator in the primate, so that's also consistent with nonhuman primate experiments.

OK, so this has a very long time constant. But we want a neural model that can model that very long time constant of this persistent activity that maintains eye position. All right, but the intrinsic time constant of neurons is about 100 milliseconds. So how do we get from a single neuron that has a time constant of 100 milliseconds to a neural integrator that can have a time constant of tens of seconds?

All right, one way to do that is by making a network that has recurrent connections. And you remember that the simplest kind of recurrent network is a neuron that has an autapse. But more generally, we'll have neurons that connect to other neurons. Those other neurons connect to other neurons.

And there are feedback loops. This neuron connects to that neuron. That neuron connects back, and so on. And so the activity of this neuron can go to other neurons, and then come back, and excite that neuron again, and maintain the activity of that neuron. So we developed a method for analyzing that kind of network by [INAUDIBLE] a recurrent weight matrix, recurrent connection matrix that describes the connections to a neuron A in this network from all the other neurons in the network, A prime, input to neuron A from neuron A prime.

And now we can write down a differential equation for the activity of one of these neurons. dv/dt is minus v that produces this intrinsic decay, plus synaptic input from all the other neurons in the network summed up over all the other neurons plus this external burst input. So how do we make a neural network that looks like an integrator? But how do we do that?

If we want our neuron, the firing rate of our neuron to behave like an integrator of its input, what do we have to do to this equation to make this neuron look like an integrator? So what do we have to do? To make this neuron look like an integrator, it would just be tau dv/dt equals burst input. Right? So in order to make this network into an integrator, we have to make sure that these two terms sum to zero.

So in other words, the feedback from other neurons in the network back to our neuron has to exactly balance the intrinsic leak of that neuron. Does that makes sense? OK, so let's do that. And when you do that, this is zero.

The sum of those two terms is zero. And now the derivative of the activity of our neuron is just equal to the input. So our neuron now integrates the input. So now the firing rate of our neuron, so there should be a v is equal to 1 over tau, the integral of burst input.

So we talked last time about how you analyze recurrent neural networks. We start with a recurrent weight matrix. So again, these Ms describe the recurrent weights within that network.

We talked about how if M is a symmetric matrix, connection matrix, then we can rewrite the connection matrix as a rotation matrix times a diagonal matrix times a rotation, the inverse rotation matrix, so phi transpose lambda phi where, again, lambda is a diagonal matrix, and phi is a rotation matrix that's [INAUDIBLE] two, in this case, in the case of two-- a two-neuron network, then this rotation matrix has as its columns the two basis vectors that we can now use to rewrite the firing rates of this work in terms of modes of the network.

So we can multiply the firing rate vector of this network times phi transpose to get the firing rates of different modes of that network. And what we're doing is essentially rewriting this recurrent network as set of independent modes, independent neurons, if you will, that described the modes with recurrent connectivity only within that mode. So we're rewriting that network as a set of only autapses. And the diagonal elements of this matrix are just the strength of the recurrent connections within that mode.

All right, so for a network to behave as integrator, most of the eigenvalues should be less than 1, but one eigenvalue should be 1. And in that case, one mode of the network becomes an integrating mode, and all of the other modes of the network have the property that their activity decays away very, very rapidly. So I'm going to go through this in more detail and show you examples.

But for a network to behave as an integrator, you want one integrating mode, one eigenvalue close to 1 and most of the-- all of the other eigenvalues much less than 1. So if you do that, then you have one mode that has the following equation that describes its activity, tau, and let's say that's lambda 1 that has eigenvalue of 1.

So tau dc1/dt, dc/dt equals minus c, that's the intrinsic decay of that mode, plus lambda 1 c1 plus burst input. And if lambda 1 is equal to 1, then those two terms cancel. Then the feedback balances the leak, and that mode becomes an integrating mode.

So when you have a burst input, the activity in that mode increases. It steps up to some new value. And then between the burst inputs, that mode obeys-- the activity of that mode obeys the following differential equation. There's no more burst input between the bursts.

dc/dt is just equal to lambda minus 1 over tau times c1. And if lambda is equal to 1, then this, then dc/dt equals zero, and the activity is constant. Does that makes sense? Any questions about that? Yes, Rebecca.

AUDIENCE: OK, so why does it [INAUDIBLE] need to balance [INAUDIBLE]

MICHALE FEE: Yes, that's exactly right. If this is not true, if, let's say that-- what happens if lambda is less than 1? If lambda is less than 1, then this quantity is negative. So if lambda is 0.5, let's say, then this is 0.5 over tau, minus 0.5 over tau. So dc/dt is some negative constant times c. Which means if c is positive, then dc/dt is negative, and c is decaying.

Does that makes sense? If lambda is bigger than 1, then this constant is positive. So if c is positive, then dc/dt is positive, and c continues to grow. So it's only when lambda equals 1 that dc/dt is zero between the burst inputs.

OK, so let's look at a really simple model where we have two neurons. There's autapse recurrence here, but it's easy to add that. And let's say that the weights between these two neurons are 1. So we can write down the weight matrix. It's just 0, 1; 1, 0 because the diagonals, the diagonals are 0, OK, 0, 1; 1, 0.

The eigenvalue equation looks like this. You know that because the diagonal elements are equal to each other and the off-diagonal elements are equal to each other because it's a symmetric matrix, then the eigenvalue, the eigenvectors are always what?

AUDIENCE: [INAUDIBLE]

MICHALE FEE: 45 degrees, OK, so 1, 1 and minus 1, 1. So our modes of the network, if we look in this state space of v1 versus v2, the two modes of the network are in the 1, 1 direction and the 1, minus 1 direction. What are the eigenvalues of this network?

OK, so for a matrix like this with equal diagonals and equal off-diagonals, the eigenvalues are just the diagonal elements plus or minus the off-diagonal element. I'll just give you a hint. This is going to be very similar to a problem that you'll have on the final. So if you have any questions, feel free to ask me. OK?

OK, so the eigenvalues are plus or minus 1. They're 1 and minus 1. And it turns out for this case, it's easy to show that the value for this mode is 1, and the eigenvalue for this mode is minus 1. And you can see it. It's pretty intuitive.

This network likes to be active such that both of these neurons are both on. When that neuron's on, it activates that neuron. When that neuron's on, it activates that neuron. And so this network really likes it when both of those neurons are active. And that's the amplifying direction of this network.

And the eigenvalue value is such that the amplification in that direction is large enough that it turns that into an integrating mode. All right, so I'll show you what that looks like. So the eigenvalues again are 1 and minus 1. If you just do that matrix multiplication, you'll see that that's true. lambda is 1, and lambda is minus 1.

You can just read this off. This first eigenvalue here is the eigenvector for the first mode. This eigenvalue is the eigenvalue for that vector for that mode. So here's what this looks like. So this mode is the integrating mode. This mode is a decaying mode because the eigenvalue is much less than 1. And what that means is that no matter where we start on this network, the activity will decay rapidly toward this line. Does that makes sense?

No matter where you start the network, activity in this direction will decay. Any state of this network that's away from this line corresponds to activity of this mode, and activity of that mode decays away very rapidly. So no matter where you start, the activity will decay to this diagonal line.

So let me just ask one more question. So if we put an input in this direction, what will the network do? So let's turn on an input in this direction and leave it on. What does the network do? Rebecca?

AUDIENCE: [INAUDIBLE]

MICHALE FEE: Good. So we're going to turn it on and leave it on first. The answer you gave is the answer to my next question. The answer is when you put that input on and you turn it off, then the activity goes back to zero. That's exactly right.

But when you put the input-- when you turn the input in this direction on, the network will-- the state will move in this direction and reach a steady state. When you turn the input off, it will decay away back to zero. If we put an input in this direction, what happens?

AUDIENCE: It just keeps going on.

MICHALE FEE: It just keeps integrating. And then we turn the input off. What happens?

AUDIENCE: It [INAUDIBLE]

MICHALE FEE: It stops, and it stays. Because the network activity in this direction is integrating any input that has a projection in this direction. Yes.

AUDIENCE: So [INAUDIBLE] steady state [INAUDIBLE] to F1, so if anything that has any component in the F1 direction will either grow or [INAUDIBLE] over 90 degrees [INAUDIBLE] F1?

MICHALE FEE: Yep.

AUDIENCE: Would it [INAUDIBLE]

MICHALE FEE: Like here? So if you put an input in this direction, what is the component of that input in the integrating direction? If we put an input like this, what-- it has zero component in the integrating direction, and so nothing gets integrated. So you put that input. The network responds. You take the input away, and it goes right back to zero. If you put an input in this direction, all of that input is in this direction, and so that input just gets integrated by the network. OK?

What happens if you put an input in this direction? Then it has a little bit of-- it has some projection in this direction and some projection in this direction. The network will respond to the input in this direction. But as soon as that input goes away, that will decay away. This, the projection in this direction, will continue to be integrated as long as the input is there.

So let me show you what that looks like. So I'm going to show you what happens when you put an input vertically. What that means, input in this direction means that we have an input to H1. Input to this neuron is 0, but the input to that neuron is 1. That corresponds to H0 being-- H1 direction being 0, and the H2 direction being 1 that has a projection in this direction and this direction.

And here's what the network does. OK, sorry. I forgot which way it was going. So you can see that the network is responding to the input in the H1 direction. But as soon as that input goes away, the activity of the network in this direction goes away as soon as the input goes away. But it's integrating the projection in this direction.

So you can see it continues to integrate. And than you put an input in the opposite direction, it integrates until the input goes away, and it stops there. OK, let me play that again. Does everyone get a sense for what's going on?

 

So now we have a input that has a projection in the minus F1 direction. And so it's the network is just integrating that negative number. OK, is that clear? OK, all right, so that's a neural integrator. It's that simple. It has one mode that has an eigenvalue of 1. And all of its other modes have small eigenvalues or a negative.

OK, so notice that no matter where you start, this network evolves. As long as there's no input, that network just relaxes to this line, to a state along that line. So that line is what we call an "attractor" of the network. The state of the network is attracted to that line.

Once the state is sitting on that line, it will stay there. So that kind of attractor is called a "line attractor." That distinguishes it from other kinds of attractors that we'll talk in the next lecture. We'll talk about when there are particular points in the state space that are attractors. OK, no matter where you start the network around that point, the state evolves toward that one point.

OK, so the line of the line attractor corresponds to the direction of the integrator mode, of the [INAUDIBLE] mode. So we can kind of see this attractor in action. If we record from two neurons in this integrator network of the goldfish during this task, if you will, where the [INAUDIBLE] saccades to different directions, so here's what that looks like.

So again, we've got two neurons recorded simultaneously, and we're following the [INAUDIBLE] rate [INAUDIBLE] versus [INAUDIBLE]. And Marvin the Martian here is indicating which way the goldfish is looking [INAUDIBLE]. OK, any questions about that?

So the hypothesis is that-- so I should mention that there-- I didn't say this before. There are about a couple hundred neurons in that nucleus in area one that connect to each other, that contact each other. What's not really known yet-- it's a little hard to prove, but people are working on it.

What's not known yet is whether the connections between those neurons have the right synaptic strength to actually give you lambda, give you an eigenvalue of 1 in that network. So it's still kind of an open question whether this model is exactly correct in describing how that network works. But Tank and others in the field are working on proving that hypothesis.

You can see that one of the challenges of this model for this persistent activity is that in order for this network to maintain persistent activity, that feedback from these other neurons back to this neuron have to be-- have to exactly match the intrinsic decay of that neuron. And if that feedback is too weak, you can see that lambda is slightly less than 1. What happens is that neural activity will decay away rather than being persistent. And if the feedback is too strong, that neural activity will run away, and it will grow exponentially.

So you can actually see evidence of these two pathological cases in neural integrators. So let's see what that kind of mismatch of the feedback would look like in the behavior. So if you have a perfect integrator, you can see that the-- you'll get saccades. And then the eye position between saccades will be exactly flat.

The eye position will be constant, which means the derivative of eye position will be zero between the saccades. And it will be zero no matter what eye position the animal is holding its eyes. So we can plot the derivative of eye position as a function of eye position, and that should be zero everywhere if the integrator is perfect.

Now, what happens if the integrator is leaky. Now you can see that, in this case, the eye is constantly rolling going back toward zero. So but if the eye is already at zero, then the derivative should be close to zero. If the eye is far away from zero, then the derivative should be-- if the eye position is very positive, you can see that this leak, this leaky integrator corresponds to the derivative being negative.

So if e is positive, then the derivative is negative. If e is negative, then the derivative is positive. And that corresponds to a situation like this. Positive eye position corresponds to negative derivative. And you can see that the equation for the activity of this mode which then translates into eye position is just e to the minus a constant times t.

If you have an unstable integrator, if this lambda is greater than 1, then positive eye positions will produce a positive derivative, and you get x runaway growth of the eye position, and that corresponds to a situation like this-- positive eye position, positive derivative, negative eye position, negative derivative. And then that equation for that situation is either the plus constant times t.

All right, so you can actually produce a leaky integrator in the circuit by injecting a little bit of local anesthetic into part of that nucleus. And so what would that do? You can see that if you inject lidocaine or some other inactivator of neurons into part of that network, it would reduce the feedback connections onto the remaining neurons. And so lambda becomes less than 1, and that produces a leaky integrator when you do that manipulation. So this experiment is consistent with the idea that feedback within that network is required to produce that stable, persistent activity.

Now, you can actually find cases where there are deficits in the ocular motor system that are associated with unstable integration. And this is called congenital nystagmus. So this is a human patient with this condition. And the person is being told to try to fixate at a particular position.

But you can see that what happens is their eyes sort of run away to the edges, to the extremes of eye position. So they can fixate briefly. The integrator kind of runs away, and their eyes run to the edges, to the extremes of the range of eye position. And it's thought that that one hypothesis for what's going on there is that the ocular motor integrator is actually in an unstable configuration, that feedback is too strong.

So exactly how precisely do you need to set that feedback in order to produce a perfect integrator? So you can see that the getting a perfect integrator requires that lambda minus 1 is equal to 0. So lambda is equal to 1. But if lambda is slightly different from 1, we can actually estimate what the time constant of the integrator would be.

So you can see that the time constant is really tau over lambda minus 1, tau over lambda minus 1. So given the intrinsic time constant tau n, you can actually estimate how close lambda has to be to 1 to get a 30-second time constant, OK? And that turns out to be extremely close to 1. In order to go from a 100-millisecond time constant to a 30-second time constant, you need a factor of 300 or, if the neural time constant is even shorter, maybe even 3,000 precision in setting lambda equal to 1.

So this is actually one of the major criticisms of this model, that it can be hard to imagine how you would actually set the feedback in a recurrent network so precisely to get a lambda equal to give you time constants on the order of 30 seconds. Does anybody have any ideas how you might actually do that? What would happen?

Let's imagine what would happen if we were-- we make saccades constantly. We make several saccades per second, not including the little microsaccades that we make all the time. But when we make a saccade, what happens to the image on the retina?

AUDIENCE: [INAUDIBLE]

MICHALE FEE: Yeah, so and if we make a saccade this way, the image on the retina looks like the world is going whoosh, like this. And as soon as it stops and our eyes-- if our integrator is perfect when the saccade ends, our eyes are at a certain position. What happens to the image on the retina? If our eyes make a saccade, and stop, and stay at a certain position and the velocity is zero, then what happens to the image on the retina? It becomes stationary.

So but if we had a problem with our integrator-- let's say that our integrator was unstable. So we make a saccade in this direction, but our integrator's unstable, so the eyes keep going. Then what would the image on the retina look like if we would have a motion of the image across the retina during the saccade? And then if our eyes kept drifting, the image would keep going. If we had a leaky integrator and we make a saccade, the image of the world could go whoosh, and then it would start relaxing back as the eyes drift back to zero.

So the idea is that when we're walking around making saccades, we have immediately feedback about whether our integrator is working or not. And so, OK, I'm going to skip this. So the idea is that we can use that sensory feedback that's called "retinal slip," image slip, to give feedback about whether the integrator is leaky or unstable and use that feedback to change lambda.

So if we make a saccade this way, the image is going to go like this. And now if that image starts slipping back, what does that mean we want to do? What do we need to do to our integrator, our synapses in our recurrent network if after we make a saccade, the image starts slipping back in the direction that it came from?

We need to strengthen it. That means we have a leaky integrator. We would need to strengthen or make those connections within the integrator network more excitatory. And if we make a saccade this way, the world goes like this and then the image continues to move, it would mean our integrator is unstable. The excitatory connections are too strong. And so we would have a measurement of image slip that would tell us to weaken those connections.

A lot of evidence that this kind of circuitry exists in the brain and that it involves the cerebellum. David Tank and his colleagues set out to test whether this kind of image slip actually controls the recurrent connections or controls the state of the integrator, whether you can use image slip to control whether the integrator network is unstable or leaky, whether that feedback actually controls it. Rebecca.

AUDIENCE: [INAUDIBLE] is the [INAUDIBLE] between slip and overcompensation with [INAUDIBLE] versus unstable integrator, the direction of [INAUDIBLE]

MICHALE FEE: Yes, exactly. So if we make a saccade this way, the world on-- the image on the retina is going to, whoosh, suddenly go this way. But then if the image goes-- OK, in the unstable case, the eyes will keep going, which means the image will keep going this way. So you'll have-- I don't know what sign you want to call that, but here, it's they did a sign flip.

Here the case of decay. So dE/dt is less than zero. That means that the eyes are going back, which means that after you make a saccade, the image goes this way, and then it starts sliding back.

AUDIENCE: So it'll return to--

MICHALE FEE: Return, yeah. So if dE/dt is negative, that means it's leaky. The image slip will be positive. And then you use that positive image slip to increase the weight of the synapses. So you change the synaptic weights in your network by an amount that's proportional to the negative of the derivative of eye position, which is read out as image slip. OK, is that clear?

OK, so they actually did this experiment. So they took a goldfish, head fixed it, put it in this arena. They made a little-- you put a little coil on the fish's eye. So this is a standard procedure for measuring eye position in primates, for example. So you can put a little coil on the eye that measures-- you measure-- OK, so you put a little coil on the eye, and you surround the fish with oscillating magnetic fields.

So you have a big coil outside the fish on this side, another coil on this side, a coil on the top and bottom, and a coil on front and back. And now you run AC current through those coils. And now by measuring how much voltage fluctuation you get in this coil, you can tell what the orientation of that coil is. Does that makes sense?

So now you can read that out here and get a very accurate measurement of eye position. And so now when the fish makes a saccade, you can read out which direction the saccade was. And immediately after the saccade, you can make this spot, so there's a like a disco ball up there that's on a motor that produces spots on the inside of the planetarium.

Notice the fish makes a saccade in this direction. What you do is you make the spots drift back, drift in the direction as though the eyes were sliding back, as though the integrator were leaky. Does that make sense? So you can fool the fish's ocular motor system into thinking that its integrator is leaky.

And what do you think happens? After about 10 minutes of that, you then turn all the lights off. And now the fish's integrator is unstable. So here's what that looks like. There's the spots on the inside. There's the disco ball. That's a overview picture showing the search coils for the eye position measurement system.

And here's the control. That's what the fish-- the eye position looks like as a function of time. So you have saccade, fixation, saccade, fixation. That right there, anybody know what that is? That's the fish blinking. So it blinks.

Give feedback-- OK, here they did it the other way. So they give their feedback as if the network is unstable, and you can make the network leaky. If you give feedback as if the network is leaky, so it makes a saccade, and now you drift the spots in the direction as if the eye were sliding back to neutral position, and now you can make the network unstable. So it makes a saccade, and the eyes continue to move in the direction of the saccade. Saccade, and it runs away. Any questions about that?

So that learning circuit, that circuit that implements that change in the synaptic weights of the integrator circuit, actually involves the cerebellum. There's a whole cerebellar circuit that's involved in learning various parameters of the ocular motor control system that produces these plastic changes. OK, so that's-- are there any questions? Because that's it.

So I'll give you a little summary. So the goldfishes do integrals. There's an integrator network in the brain that takes burst inputs that drive saccades. And the integrator integrates those bursts and produces persistent changes in the activity of these integrator neurons that then drive the eyes to different positions and maintain that eye position. So we've described a neural mechanism, which is this recurrent network, a recurrent network has one eigenvalue that's 1 that produces an integrating mode, and all the other eigenvalues are close to-- are less than 1 or negative.

The model is not very robust if you have to somehow hand-tune all of those [INAUDIBLE] to get a lambda of 1. But there is a mechanism that uses retinal slip to tell whether that eigenvalue is set correctly in the brain and feeds back to adjust that eigenvalue to produce the upper lambda, the proper eigenvalue in that circuit so that it functions as an integrator and using visual feedback.

And I just want to mention again, so I actually got most of these slides from Mark Goldman when he and I actually used to teach an early version of this course. We used to give lectures in each other's courses, and this was his lecture. He later moved to-- he was at Wellesley. So we would go back and forth and give these lectures.

But he moved to Davis. So now I'm giving his lecture myself. And the theoretical work was done by Sebastian Seung and Mark Goldman. The experimental work was done in David Tank's lab in collaboration with Bob Baker at NYU.

OK, and so next time, we're going to-- so today we talked about short-term memory using neural networks as integrators to accumulate information and to perform-- to generate line attractors that can produce a short-term memory of continuously graded variables like eye position. Next time, we're going to talk about using recurrent networks that have eigenvalues greater than 1 as a way of storing short-term discrete memories. And those kinds of networks are called Hopfield networks, and that's what we're going to talk about next time. OK, thank you.