1 00:00:15,023 --> 00:00:16,440 MICHALE FEE: So today, we're going 2 00:00:16,440 --> 00:00:18,780 to introduce a new topic, which is related 3 00:00:18,780 --> 00:00:20,850 to the idea of fine-tuning curves, 4 00:00:20,850 --> 00:00:23,650 and that is the notion of receptive fields. 5 00:00:23,650 --> 00:00:28,350 So most of you have probably been, 6 00:00:28,350 --> 00:00:31,710 at least those of you who've taken 9.01 or 9.00 maybe, 7 00:00:31,710 --> 00:00:36,630 have been exposed to the idea of what a receptive field is. 8 00:00:36,630 --> 00:00:40,740 The idea is basically that in sensory systems 9 00:00:40,740 --> 00:00:44,160 neurons receive input from the sensory periphery, 10 00:00:44,160 --> 00:00:47,880 and neurons generally have some kind of sensory stimulus 11 00:00:47,880 --> 00:00:50,700 that causes them to spike. 12 00:00:50,700 --> 00:00:53,910 And so one of the classic examples 13 00:00:53,910 --> 00:00:57,360 of how to find receptive fields comes from the work 14 00:00:57,360 --> 00:01:00,570 of Huble and Wiesel. 15 00:01:00,570 --> 00:01:03,150 So I'll show you some movies made 16 00:01:03,150 --> 00:01:06,720 from early experiments of Huble-Wiesel 17 00:01:06,720 --> 00:01:10,620 where they are recording in the visual cortex of the cat. 18 00:01:10,620 --> 00:01:13,140 So they place a fine metal electrode 19 00:01:13,140 --> 00:01:20,050 into a primary visual cortex, and they present. 20 00:01:20,050 --> 00:01:24,570 So then they anesthetize the cat so the cat can't move. 21 00:01:24,570 --> 00:01:26,220 They open the eye, and the cat's now 22 00:01:26,220 --> 00:01:29,340 looking at a screen that looks like this, where 23 00:01:29,340 --> 00:01:31,560 they play a visual stimulus. 24 00:01:31,560 --> 00:01:36,600 And they actually did this with essentially a slide projector 25 00:01:36,600 --> 00:01:38,280 that they could put a card in front 26 00:01:38,280 --> 00:01:40,800 of that had a little hole in it, for example, 27 00:01:40,800 --> 00:01:44,852 that allowed a spot of light to project onto the screen. 28 00:01:44,852 --> 00:01:46,560 And then they can move that spot of light 29 00:01:46,560 --> 00:01:49,740 around while they record from neurons in visual cortex 30 00:01:49,740 --> 00:01:54,590 and present different visual stimuli to the retina. 31 00:01:54,590 --> 00:01:56,780 So here's what one of those movies looks like. 32 00:01:59,732 --> 00:02:03,072 So you're hearing the actual potential 33 00:02:03,072 --> 00:02:04,690 of a neuron visual cortex. 34 00:02:19,530 --> 00:02:21,010 So you can see the neuron generates 35 00:02:21,010 --> 00:02:23,230 lots of spikes when you turn a spot of light 36 00:02:23,230 --> 00:02:26,384 on a particular part of the visual field. 37 00:02:33,170 --> 00:02:36,910 So they will basically play around with spots of light 38 00:02:36,910 --> 00:02:40,240 or bars of light and see where the neuron spikes a light, 39 00:02:40,240 --> 00:02:46,165 and then they would draw on the screen-- 40 00:02:48,835 --> 00:02:52,885 I think they're going to draw in a moment here-- 41 00:02:57,250 --> 00:03:00,250 what they would call the receptive field of the neuron. 42 00:03:00,250 --> 00:03:04,590 So you can see that this neuron responds with high firing rate 43 00:03:04,590 --> 00:03:10,600 when you turn on a stimulus in that small region there. 44 00:03:10,600 --> 00:03:22,200 Notice that the cell also responds when you turn off 45 00:03:22,200 --> 00:03:25,480 a stimulus that is right in the area 46 00:03:25,480 --> 00:03:29,770 surrounding that small receptor field. 47 00:03:29,770 --> 00:03:33,670 So the neuron has parts of its receptive field that 48 00:03:33,670 --> 00:03:37,610 respond with increased firing when you apply light, 49 00:03:37,610 --> 00:03:39,890 and they also have parts of the receptive field that 50 00:03:39,890 --> 00:03:44,692 respond with higher firing rate when you remove light. 51 00:03:48,520 --> 00:03:52,840 That was actually a cell, I should have said, 52 00:03:52,840 --> 00:03:59,860 that's in the thalamus that projects to visual cortex. 53 00:03:59,860 --> 00:04:01,550 So that was a thalamic neuron. 54 00:04:01,550 --> 00:04:04,030 Here's what a neuron in cortex might look like. 55 00:04:11,710 --> 00:04:15,940 So they started recording in the thalamus. 56 00:04:15,940 --> 00:04:20,079 They saw that those neurons responded to spots of light 57 00:04:20,079 --> 00:04:22,612 in small parts of the visual field. 58 00:04:22,612 --> 00:04:24,320 They were actually recording from neurons 59 00:04:24,320 --> 00:04:25,865 in the visual cortex. 60 00:04:25,865 --> 00:04:27,490 They got kind of-- they couldn't really 61 00:04:27,490 --> 00:04:29,115 figure out what the neurons were doing, 62 00:04:29,115 --> 00:04:31,660 and they pulled the slide out of the projector, which 63 00:04:31,660 --> 00:04:34,420 made an edge of light moving across the visual field. 64 00:04:34,420 --> 00:04:36,880 And the neuron they were recording from at that moment 65 00:04:36,880 --> 00:04:40,660 responded robustly when they pulled the slide out. 66 00:04:40,660 --> 00:04:42,310 And they realized, oh, maybe it's 67 00:04:42,310 --> 00:04:45,070 an edge that the neuron is responding to. 68 00:04:45,070 --> 00:04:48,780 And so then they started doing experiments with bars of light. 69 00:04:52,810 --> 00:04:53,760 Here's an example. 70 00:04:53,760 --> 00:04:56,010 So you can see the neuron responds when you turn 71 00:04:56,010 --> 00:04:58,030 a light on in this area here. 72 00:04:58,030 --> 00:05:01,460 But is responds when you turn light off in this area here. 73 00:05:01,460 --> 00:05:04,250 And so you can see they're marking 74 00:05:04,250 --> 00:05:08,490 a symbol for positive responses, positive responses to light 75 00:05:08,490 --> 00:05:14,190 on here, and negative responses or increased firing 76 00:05:14,190 --> 00:05:16,450 when you turn the light off. 77 00:05:16,450 --> 00:05:18,860 So there's different parts of the receptive 78 00:05:18,860 --> 00:05:21,700 field that have positive and negative components. 79 00:05:21,700 --> 00:05:23,640 But you can see that the general picture here 80 00:05:23,640 --> 00:05:26,370 is that the process of finding receptive fields 81 00:05:26,370 --> 00:05:29,200 at this early stage was kind of random. 82 00:05:29,200 --> 00:05:31,050 You just tried different things and hoped 83 00:05:31,050 --> 00:05:32,220 to make the neurons spike. 84 00:05:36,990 --> 00:05:41,150 And we're going to come back to this idea of finding 85 00:05:41,150 --> 00:05:44,900 receptive fields by trying random things, 86 00:05:44,900 --> 00:05:49,340 but in a more systematic way, at the end of the lecture today. 87 00:05:49,340 --> 00:05:52,170 So here's what we're going to be talking about. 88 00:05:52,170 --> 00:05:55,620 So you can see that Hubel and Wiesel were 89 00:05:55,620 --> 00:05:59,340 able to describe that receptive field by finding 90 00:05:59,340 --> 00:06:02,040 positive and negative parts and writing symbols 91 00:06:02,040 --> 00:06:03,720 down on a screen. 92 00:06:03,720 --> 00:06:06,840 We're going to take a more mathematical approach 93 00:06:06,840 --> 00:06:11,190 and think about what that means in a quantitative model of how 94 00:06:11,190 --> 00:06:13,860 neurons respond to stimuli. 95 00:06:13,860 --> 00:06:16,890 And the basic model that we'll be talking about 96 00:06:16,890 --> 00:06:21,270 is called an LN model, linear/nonlinear model. 97 00:06:21,270 --> 00:06:23,760 And we're going to describe neural responses 98 00:06:23,760 --> 00:06:28,110 as a linear filter that acts on the sensory stimulus followed 99 00:06:28,110 --> 00:06:31,500 by a nonlinear function that just says neurons can only 100 00:06:31,500 --> 00:06:32,950 fire at positive rates. 101 00:06:32,950 --> 00:06:36,360 So we're going to have our neurons spike when that filter 102 00:06:36,360 --> 00:06:40,190 output is positive, but not when the filter output is negative. 103 00:06:40,190 --> 00:06:42,630 And we're going to describe spatial receptive fields 104 00:06:42,630 --> 00:06:44,490 as a correlation of the receptive 105 00:06:44,490 --> 00:06:45,668 field with the stimulus. 106 00:06:45,668 --> 00:06:47,460 And we're also going to talk about the idea 107 00:06:47,460 --> 00:06:50,700 of temporal receptive fields, which 108 00:06:50,700 --> 00:06:53,850 will be a convolution of a temporal receptive 109 00:06:53,850 --> 00:06:57,190 field with the stimulus. 110 00:06:57,190 --> 00:06:59,520 So the firing rate will be a convolution 111 00:06:59,520 --> 00:07:02,580 of a receptive field with the temporal structure 112 00:07:02,580 --> 00:07:04,600 of the stimulus. 113 00:07:04,600 --> 00:07:07,120 We're going to then turn to the-- 114 00:07:07,120 --> 00:07:09,130 combine these things into the concept 115 00:07:09,130 --> 00:07:13,510 of a spatial temporal receptive field that simultaneously 116 00:07:13,510 --> 00:07:16,300 describes the spatial sensitivity 117 00:07:16,300 --> 00:07:21,100 and the temporal sensitivity of a neuron, as an STRF, 118 00:07:21,100 --> 00:07:22,090 as it's called. 119 00:07:22,090 --> 00:07:25,270 And we'll talk about the concept of separability. 120 00:07:25,270 --> 00:07:29,170 And finally, we're going to talk about the idea 121 00:07:29,170 --> 00:07:33,950 of using random noise to try to drive neurons, 122 00:07:33,950 --> 00:07:36,440 to drive activity in neurons, and using what's 123 00:07:36,440 --> 00:07:42,050 called a spike-triggered average to extract the stimulus that 124 00:07:42,050 --> 00:07:43,790 makes a neuron spike. 125 00:07:43,790 --> 00:07:45,980 And we're going to use that to compute-- 126 00:07:45,980 --> 00:07:48,020 we're going to see how to use that to compute 127 00:07:48,020 --> 00:07:52,550 a spatial temporal receptive field in the visual system 128 00:07:52,550 --> 00:07:56,330 or a spectral temporal receptive field in the auditory system. 129 00:07:59,930 --> 00:08:01,640 So let's start with this. 130 00:08:01,640 --> 00:08:04,520 What are spatial and temporal receptive fields? 131 00:08:07,090 --> 00:08:13,600 So we just saw how you can think of a region of the visual space 132 00:08:13,600 --> 00:08:18,740 that makes a neuron spike when you turn light on or makes 133 00:08:18,740 --> 00:08:20,470 a neuron spike when you turn light off. 134 00:08:23,820 --> 00:08:26,150 And at the simplest level, you can 135 00:08:26,150 --> 00:08:28,070 think of that in the visual system 136 00:08:28,070 --> 00:08:29,990 as just a part of the visual field 137 00:08:29,990 --> 00:08:32,390 that a neuron will respond to. 138 00:08:32,390 --> 00:08:36,140 So if you flash of light over here, the neuron might respond. 139 00:08:36,140 --> 00:08:38,330 If you flash of light over here, it won't respond. 140 00:08:38,330 --> 00:08:40,760 So there's this region of the visual field where 141 00:08:40,760 --> 00:08:45,050 neurons respond, but it's more than just a region. 142 00:08:45,050 --> 00:08:51,590 There's actually a pattern of features within that area that 143 00:08:51,590 --> 00:08:53,930 will make a neuron spike, and other patterns 144 00:08:53,930 --> 00:08:58,280 will keep the neuron from spiking. 145 00:08:58,280 --> 00:09:03,290 And so we can think of a neuron as having some spatial filter 146 00:09:03,290 --> 00:09:05,000 that has positive parts, and I'll 147 00:09:05,000 --> 00:09:06,860 use green throughout my lecture today 148 00:09:06,860 --> 00:09:08,930 for positive parts of a receptive 149 00:09:08,930 --> 00:09:12,170 field, and negative parts. 150 00:09:12,170 --> 00:09:14,960 And this is a classic organization 151 00:09:14,960 --> 00:09:17,300 of receptive fields, let's say, in the retina 152 00:09:17,300 --> 00:09:23,030 or in the thalamus, where you have an excitatory central part 153 00:09:23,030 --> 00:09:26,576 of a receptive field and an inhibitory surround 154 00:09:26,576 --> 00:09:28,880 of the receptive field. 155 00:09:28,880 --> 00:09:31,640 So we can think of this as a filter that 156 00:09:31,640 --> 00:09:34,070 acts on the sensory input. 157 00:09:34,070 --> 00:09:39,140 And the better the stimulus overlaps with that filter, 158 00:09:39,140 --> 00:09:40,550 the more the neuron will spike. 159 00:09:40,550 --> 00:09:44,700 So let's formalize this a little bit into a model. 160 00:09:44,700 --> 00:09:47,630 So let's say we have some visual stimulus that 161 00:09:47,630 --> 00:09:52,380 is an intensity as a function of position x and y. 162 00:09:52,380 --> 00:09:57,870 We have sum filter, G, that filters that stimulus. 163 00:09:57,870 --> 00:10:00,950 So we put the stimulus into this filter. 164 00:10:00,950 --> 00:10:05,720 This filter, in this case, just looks like, in this case, 165 00:10:05,720 --> 00:10:10,350 an excitatory surround in an inhibitory center. 166 00:10:10,350 --> 00:10:13,770 That filter has an output, L, which 167 00:10:13,770 --> 00:10:17,190 is the response of the filter. 168 00:10:17,190 --> 00:10:20,990 Then we have some nonlinearity. 169 00:10:20,990 --> 00:10:24,370 So we take the response of the filter, L, 170 00:10:24,370 --> 00:10:27,970 we add it to some spontaneous firing rate, 171 00:10:27,970 --> 00:10:33,900 and we take the positive part of that sum 172 00:10:33,900 --> 00:10:35,790 and call that our firing rate. 173 00:10:35,790 --> 00:10:38,940 So that would be a typical output nonlinearity. 174 00:10:38,940 --> 00:10:41,910 It's called "a threshold nonlinearity," 175 00:10:41,910 --> 00:10:45,210 where if the sum of the filter output 176 00:10:45,210 --> 00:10:48,150 and the spontaneous firing rate is greater than 0, 177 00:10:48,150 --> 00:10:50,790 then that corresponds to the firing rate of the neuron. 178 00:10:53,670 --> 00:10:58,130 So in this case, you can see that as a function of L-- 179 00:10:58,130 --> 00:11:00,550 I should have labeled that axis L-- 180 00:11:00,550 --> 00:11:03,140 as a function of L, when L is 0, you 181 00:11:03,140 --> 00:11:05,360 can see the neuron has some spontaneous firing 182 00:11:05,360 --> 00:11:06,740 rate, R naught. 183 00:11:06,740 --> 00:11:08,930 And if L is positive, the rate goes up. 184 00:11:08,930 --> 00:11:12,380 If L is negative, the rate goes down until the rate hits 0. 185 00:11:12,380 --> 00:11:18,380 And then once the neuron stops firing, it can't go negative. 186 00:11:18,380 --> 00:11:22,040 So the neuron firing rate stays at 0. 187 00:11:22,040 --> 00:11:29,210 And then once you have this firing rate of the neuron, what 188 00:11:29,210 --> 00:11:32,210 is it that actually determines whether the neuron will spike? 189 00:11:32,210 --> 00:11:34,580 So in most models like this, there 190 00:11:34,580 --> 00:11:37,160 is a probabilistic spike generator 191 00:11:37,160 --> 00:11:41,930 that is a function of the rate output of this nonlinear 192 00:11:41,930 --> 00:11:42,680 output. 193 00:11:42,680 --> 00:11:46,070 It's basically a random process that generates spikes at a rate 194 00:11:46,070 --> 00:11:48,600 corresponding to this R. 195 00:11:48,600 --> 00:11:50,930 And in the next lecture, we're going 196 00:11:50,930 --> 00:11:55,790 to come back and talk a lot more about what spike trains look 197 00:11:55,790 --> 00:11:58,130 like, how to characterize their randomness, 198 00:11:58,130 --> 00:12:01,310 and what different kinds of random 199 00:12:01,310 --> 00:12:04,760 processes you actually see in neurons. 200 00:12:04,760 --> 00:12:07,700 A very common one is the Poisson process, 201 00:12:07,700 --> 00:12:09,980 where there's an equal probability per unit 202 00:12:09,980 --> 00:12:13,520 time of a neuron generating a spike, 203 00:12:13,520 --> 00:12:17,510 and that probability is controlled by the firing rate. 204 00:12:17,510 --> 00:12:19,897 We'll come back to that and discuss it more. 205 00:12:19,897 --> 00:12:20,480 Any questions? 206 00:12:20,480 --> 00:12:21,147 Yes, [INAUDIBLE] 207 00:12:21,147 --> 00:12:23,980 AUDIENCE: Is something biologically [INAUDIBLE] 208 00:12:23,980 --> 00:12:28,080 something like if the overlap [INAUDIBLE] it's just it's 209 00:12:28,080 --> 00:12:30,350 more excitatory? 210 00:12:30,350 --> 00:12:31,100 MICHALE FEE: Yeah. 211 00:12:31,100 --> 00:12:33,680 So we're going to come back in, I think, 212 00:12:33,680 --> 00:12:35,240 a couple lectures where we're going 213 00:12:35,240 --> 00:12:38,660 to talk about exactly how you would build a filter like this 214 00:12:38,660 --> 00:12:40,070 in a simple feed forward network. 215 00:12:40,070 --> 00:12:41,900 So at the simplest level, you can just 216 00:12:41,900 --> 00:12:44,630 imagine you have a sensory periphery that 217 00:12:44,630 --> 00:12:49,160 has neurons in it that detect, let's say, 218 00:12:49,160 --> 00:12:51,020 light at different positions. 219 00:12:51,020 --> 00:12:54,950 Those neurons send axons that then impinge on, 220 00:12:54,950 --> 00:12:57,590 let's say, the neuron that we're modeling right here. 221 00:12:57,590 --> 00:13:00,170 And the pattern of those projections, 222 00:13:00,170 --> 00:13:03,140 both excitatory and inhibitory projections from the periphery, 223 00:13:03,140 --> 00:13:05,270 would give you this linear filter. 224 00:13:05,270 --> 00:13:08,930 And then this nonlinearity would be a property of this neuron 225 00:13:08,930 --> 00:13:11,060 that we're modeling. 226 00:13:11,060 --> 00:13:11,700 Yes, Jasmine? 227 00:13:11,700 --> 00:13:15,220 AUDIENCE: [INAUDIBLE] 228 00:13:15,220 --> 00:13:17,880 MICHALE FEE: Great, exactly. 229 00:13:17,880 --> 00:13:20,430 So it's going to turn out that we're 230 00:13:20,430 --> 00:13:23,490 going to treat this as a linear filter. 231 00:13:23,490 --> 00:13:26,880 The output of this filter will be calculated 232 00:13:26,880 --> 00:13:33,090 for a spatial receptive field as the correlation of this filter 233 00:13:33,090 --> 00:13:34,500 with the stimulus. 234 00:13:34,500 --> 00:13:38,760 But in the time domain, when we calculate a temporal receptive 235 00:13:38,760 --> 00:13:41,453 field, we're going to use a convolution. 236 00:13:41,453 --> 00:13:42,870 And we'll get to that in a minute. 237 00:13:42,870 --> 00:13:44,100 That's the very next thing. 238 00:13:44,100 --> 00:13:45,407 Great question. 239 00:13:45,407 --> 00:13:45,990 Anything else? 240 00:13:51,140 --> 00:13:53,080 So that's called a LN model. 241 00:13:53,080 --> 00:13:55,110 I should have put that on the slide-- 242 00:13:55,110 --> 00:13:57,390 linear/nonlinear model. 243 00:14:01,740 --> 00:14:04,590 So let's describe that mathematically. 244 00:14:04,590 --> 00:14:06,780 So let's say we have a two-dimensional receptive 245 00:14:06,780 --> 00:14:07,590 field. 246 00:14:07,590 --> 00:14:10,590 We're going to call that G of x and y. 247 00:14:10,590 --> 00:14:14,710 So remember, we had intensity as a function of x and y. 248 00:14:14,710 --> 00:14:17,310 There's our stimulus input. 249 00:14:17,310 --> 00:14:22,170 And we're going to ask, how well does that stimulus overlap 250 00:14:22,170 --> 00:14:23,430 with this receptive field? 251 00:14:23,430 --> 00:14:27,900 And we're going to describe the receptive field as a function 252 00:14:27,900 --> 00:14:31,830 on this space, x and y. 253 00:14:31,830 --> 00:14:35,250 And our linear model is going to be 254 00:14:35,250 --> 00:14:40,950 how well the stimulus matches or overlaps 255 00:14:40,950 --> 00:14:42,720 with the receptive field. 256 00:14:42,720 --> 00:14:45,960 And we do that just by multiplying the receptive 257 00:14:45,960 --> 00:14:50,910 field times the stimulus and integrating over x and y. 258 00:14:50,910 --> 00:14:54,480 x and y, just think of it as a position on the retina. 259 00:14:58,410 --> 00:15:00,600 So let's look at this in one dimension. 260 00:15:00,600 --> 00:15:02,700 So remember, this was a receptive 261 00:15:02,700 --> 00:15:05,520 field that has a positive central region 262 00:15:05,520 --> 00:15:06,960 and an inhibitory surround. 263 00:15:06,960 --> 00:15:08,730 So if we just take a slice through that 264 00:15:08,730 --> 00:15:11,390 and plot G as a function of x, you 265 00:15:11,390 --> 00:15:16,170 can see that there is a positive central lobe and inhibitory 266 00:15:16,170 --> 00:15:19,800 surround, inhibitory side load. 267 00:15:19,800 --> 00:15:23,190 That's a very, very common receptive field early 268 00:15:23,190 --> 00:15:25,650 in the visual system, in the retina 269 00:15:25,650 --> 00:15:28,800 and in the lateral geniculate nucleus. 270 00:15:28,800 --> 00:15:33,000 So in one dimension, we just take this receptive field, G, 271 00:15:33,000 --> 00:15:35,820 multiply it by the stimulus pattern, 272 00:15:35,820 --> 00:15:37,920 and integrate over position. 273 00:15:37,920 --> 00:15:41,580 That is L. That's the output of the linear filter. 274 00:15:41,580 --> 00:15:44,700 We're going to add that to a spontaneous firing rate, 275 00:15:44,700 --> 00:15:48,340 and that gives us the firing rate of our neuron. 276 00:15:48,340 --> 00:15:50,420 And you can see that that's like-- 277 00:15:50,420 --> 00:15:53,900 that this product, an integral over x, 278 00:15:53,900 --> 00:15:56,800 is just like a correlation-- 279 00:15:56,800 --> 00:16:05,450 G of I times intensity of I summed over I. 280 00:16:05,450 --> 00:16:07,440 So let's walk through what that looks like. 281 00:16:07,440 --> 00:16:09,880 So here's G of x, the receptive field. 282 00:16:09,880 --> 00:16:14,060 Let's say that's our intensity profile. 283 00:16:14,060 --> 00:16:16,540 So we're going to have a bright spot of light surrounded 284 00:16:16,540 --> 00:16:19,750 by a darker side lobe. 285 00:16:19,750 --> 00:16:21,880 So the way to think about this is, 286 00:16:21,880 --> 00:16:24,700 in visual neuroscience experiments, usually 287 00:16:24,700 --> 00:16:27,610 the background is kind of gray. 288 00:16:27,610 --> 00:16:34,140 And you'll have bright spots, like here, and dark spots, 289 00:16:34,140 --> 00:16:36,000 like there. 290 00:16:36,000 --> 00:16:38,010 And the rest will just be gray. 291 00:16:38,010 --> 00:16:42,420 So that's how you get positive and negative intensities here, 292 00:16:42,420 --> 00:16:48,150 because they're relative to some kind of gray background. 293 00:16:48,150 --> 00:16:50,792 And so now we can just multiply those two together. 294 00:16:50,792 --> 00:16:53,000 And you can see that when you multiply positive times 295 00:16:53,000 --> 00:16:55,010 positive, you get positive. 296 00:16:55,010 --> 00:16:58,490 And when you multiply negative times negative, 297 00:16:58,490 --> 00:17:00,490 you get positive. 298 00:17:00,490 --> 00:17:03,590 And when you integrate over position x, 299 00:17:03,590 --> 00:17:07,670 you get a big number. 300 00:17:07,670 --> 00:17:10,930 You get some positive number. 301 00:17:10,930 --> 00:17:13,480 So that neuron, that stimulus, would 302 00:17:13,480 --> 00:17:15,730 make this a neuron with this receptive 303 00:17:15,730 --> 00:17:19,310 field likely to spike. 304 00:17:19,310 --> 00:17:21,650 Let's consider this case. 305 00:17:21,650 --> 00:17:24,829 Now, instead of a small spot of light centered 306 00:17:24,829 --> 00:17:28,580 over the excitatory lobe of the receptive field, 307 00:17:28,580 --> 00:17:34,330 you have a broad spot of light that 308 00:17:34,330 --> 00:17:37,540 covers both the excitatory and inhibitory lobes 309 00:17:37,540 --> 00:17:38,560 of the receptive field. 310 00:17:38,560 --> 00:17:39,518 What's going to happen? 311 00:17:43,540 --> 00:17:44,040 Yeah? 312 00:17:44,040 --> 00:17:45,460 AUDIENCE: [INAUDIBLE] 313 00:17:45,460 --> 00:17:46,210 MICHALE FEE: Yeah. 314 00:17:46,210 --> 00:17:49,900 You're going to get a positive times of positive here, 315 00:17:49,900 --> 00:17:51,860 and a negative times a positive. 316 00:17:51,860 --> 00:17:54,790 And so you're going to get negative contribution 317 00:17:54,790 --> 00:17:58,060 from the side lobes, and those things 318 00:17:58,060 --> 00:18:03,370 can exactly cancel out when you integrate over position. 319 00:18:03,370 --> 00:18:06,880 And so you can get a very small response. 320 00:18:06,880 --> 00:18:08,900 If you have-- 321 00:18:08,900 --> 00:18:10,700 I'm not going to go through this example, 322 00:18:10,700 --> 00:18:11,720 but it's pretty obvious. 323 00:18:11,720 --> 00:18:14,960 If you were to have light here, and then dark 324 00:18:14,960 --> 00:18:18,300 here, and then light there, you would 325 00:18:18,300 --> 00:18:23,580 have only these negative side lobes activated. 326 00:18:23,580 --> 00:18:26,760 You would have no contribution from this excitatory lobe, 327 00:18:26,760 --> 00:18:28,995 and the integral would actually be negative. 328 00:18:28,995 --> 00:18:31,120 And so the firing rate of the neuron would go down. 329 00:18:37,050 --> 00:18:39,900 Do you remember seeing those different points 330 00:18:39,900 --> 00:18:42,210 in that first movie where you saw 331 00:18:42,210 --> 00:18:44,700 the donut of light turning on? 332 00:18:44,700 --> 00:18:49,080 The neuron kind of shuts off when you turn that light on. 333 00:18:49,080 --> 00:18:49,793 Yes? 334 00:18:49,793 --> 00:18:51,960 AUDIENCE: So I'm just curious what 335 00:18:51,960 --> 00:18:54,420 we're signing up 0 [INAUDIBLE]. 336 00:18:54,420 --> 00:18:55,170 MICHALE FEE: Yeah. 337 00:18:55,170 --> 00:18:59,460 So 0 is just this gray background. 338 00:18:59,460 --> 00:19:03,900 It's some intermediate level of light intensity-- 339 00:19:03,900 --> 00:19:04,990 pretty straightforward. 340 00:19:04,990 --> 00:19:07,830 So that's a spatial receptive field right there. 341 00:19:11,280 --> 00:19:16,620 We refer to this correlation process, this linear filter 342 00:19:16,620 --> 00:19:19,320 as linear, because you can see that if you 343 00:19:19,320 --> 00:19:24,400 put in half the light intensity, let's say, 344 00:19:24,400 --> 00:19:26,420 you get half the product. 345 00:19:26,420 --> 00:19:30,730 And when you integrate, you get half the neural response. 346 00:19:30,730 --> 00:19:35,050 If you take the stimulus and you cut it in half so that you only 347 00:19:35,050 --> 00:19:39,760 apply light and dark to half the receptive field, 348 00:19:39,760 --> 00:19:43,810 then you'll also get half the response of the neuron. 349 00:19:43,810 --> 00:19:46,180 Because this will contribute to the integral and this 350 00:19:46,180 --> 00:19:49,270 won't, and so you'll get a neural response 351 00:19:49,270 --> 00:19:50,300 that's half as big. 352 00:19:50,300 --> 00:19:54,510 So in this model, the response varies linearly 353 00:19:54,510 --> 00:19:57,610 with this overlap of the receptive field 354 00:19:57,610 --> 00:19:59,250 and the intensity. 355 00:19:59,250 --> 00:20:02,670 So that's where the term linear comes from. 356 00:20:02,670 --> 00:20:04,140 Any question about that? 357 00:20:04,140 --> 00:20:07,650 Correlation is a linear operation. 358 00:20:13,370 --> 00:20:15,740 So the next thing we're going to talk about 359 00:20:15,740 --> 00:20:17,760 is temporal receptive field. 360 00:20:17,760 --> 00:20:20,570 So we just talked about spatial receptive fields. 361 00:20:20,570 --> 00:20:25,400 Neurons are also very sensitive to how things vary in time. 362 00:20:25,400 --> 00:20:28,220 So we're going to take the same concept. 363 00:20:28,220 --> 00:20:32,150 Instead of a stimulus that's a function of position 364 00:20:32,150 --> 00:20:33,890 on the retina, let's say, we're going 365 00:20:33,890 --> 00:20:36,890 to take a stimulus that's a function of time. 366 00:20:36,890 --> 00:20:41,360 And we're going to operate on that temporal stimulus 367 00:20:41,360 --> 00:20:46,300 with a filter that [INAUDIBLE] a temporal sensitivity. 368 00:20:46,300 --> 00:20:48,220 We're going to get the output of that filter, 369 00:20:48,220 --> 00:20:49,960 add it to a spontaneous firing rate, 370 00:20:49,960 --> 00:20:54,460 and we're going to get a time-dependent firing rate. 371 00:20:54,460 --> 00:20:56,607 So let me just show you what this looks like. 372 00:20:56,607 --> 00:20:58,440 So let's say that you have a stimulus that's 373 00:20:58,440 --> 00:20:59,680 fluctuating in time. 374 00:20:59,680 --> 00:21:01,950 So imagine that you have a neuron that 375 00:21:01,950 --> 00:21:05,820 has a spatial receptive field that's just a blob, 376 00:21:05,820 --> 00:21:09,210 and you shine just a positive bump. 377 00:21:09,210 --> 00:21:12,120 And you shine light on it, and the intensity of that light 378 00:21:12,120 --> 00:21:14,130 varies. 379 00:21:14,130 --> 00:21:16,440 And this is the intensity that you 380 00:21:16,440 --> 00:21:20,310 apply to that spatial receptive field as a function of time. 381 00:21:20,310 --> 00:21:27,390 And again, 0 is some kind of average gray level. 382 00:21:27,390 --> 00:21:32,040 And so you can go dark, dark, or bright around that. 383 00:21:32,040 --> 00:21:38,910 So now, neurons generally have receptive fields in time, 384 00:21:38,910 --> 00:21:42,120 and this is what a typical receptive field might 385 00:21:42,120 --> 00:21:44,520 look like, a temporal receptive field might look like, 386 00:21:44,520 --> 00:21:47,310 for a neuron. 387 00:21:47,310 --> 00:21:50,990 Neurons are often particularly driven 388 00:21:50,990 --> 00:21:55,520 by stimuli that go dark briefly and then go 389 00:21:55,520 --> 00:22:00,630 bright very suddenly, and that causes a neuron to spike. 390 00:22:00,630 --> 00:22:06,820 So we can imagine that temporal receptive field, 391 00:22:06,820 --> 00:22:11,150 and sliding it across the stimulus, 392 00:22:11,150 --> 00:22:14,960 and measuring the overlap of that temporal receptive field 393 00:22:14,960 --> 00:22:18,506 with the stimulus at each time. 394 00:22:18,506 --> 00:22:20,800 Does that makes sense? 395 00:22:20,800 --> 00:22:23,640 And so you can see that most of the time 396 00:22:23,640 --> 00:22:25,920 that negative bump and positive bump 397 00:22:25,920 --> 00:22:29,310 are just going to be overlapping with just 398 00:22:29,310 --> 00:22:30,450 lots of random wiggles. 399 00:22:30,450 --> 00:22:32,430 But let's say that the stimulus has 400 00:22:32,430 --> 00:22:35,190 a negative part and then a positive part, 401 00:22:35,190 --> 00:22:38,730 dark, then bright. 402 00:22:38,730 --> 00:22:42,150 You can see that at this point that filter 403 00:22:42,150 --> 00:22:44,890 will have a strong overlap with the stimulus. 404 00:22:44,890 --> 00:22:45,390 Why? 405 00:22:45,390 --> 00:22:48,630 Because the negative overlaps with the negative, and that 406 00:22:48,630 --> 00:22:51,240 product is positive. 407 00:22:51,240 --> 00:22:54,430 Positive overlaps with positive, that product is positive. 408 00:22:54,430 --> 00:23:04,090 And so when you integrate over time you get peak. 409 00:23:04,090 --> 00:23:04,970 Does that make sense? 410 00:23:04,970 --> 00:23:09,580 This is, what I'm plotting here, is the product of these two 411 00:23:09,580 --> 00:23:13,140 functions at each different time step 412 00:23:13,140 --> 00:23:17,310 as I slide this temporal receptive 413 00:23:17,310 --> 00:23:19,290 field over the stimulus. 414 00:23:19,290 --> 00:23:22,218 Does that make sense? 415 00:23:22,218 --> 00:23:23,260 Any questions about that? 416 00:23:23,260 --> 00:23:24,790 I'm going to go through that. 417 00:23:24,790 --> 00:23:27,980 We're going to work on that idea little bit more, 418 00:23:27,980 --> 00:23:31,206 but if you have any questions, now's a good time. 419 00:23:31,206 --> 00:23:40,243 AUDIENCE: [INAUDIBLE] 420 00:23:40,243 --> 00:23:41,660 MICHALE FEE: So you're asking, why 421 00:23:41,660 --> 00:23:45,650 is it correlation in the spatial domain? 422 00:23:45,650 --> 00:23:48,740 Because-- well, let me answer that question 423 00:23:48,740 --> 00:23:53,030 after we define what this is mathematically. 424 00:23:53,030 --> 00:23:54,570 So what is this mathematically? 425 00:23:54,570 --> 00:23:55,570 AUDIENCE: A convolution. 426 00:23:55,570 --> 00:23:56,903 MICHALE FEE: It's a convolution. 427 00:23:56,903 --> 00:23:58,710 It's exactly what we talked-- 428 00:23:58,710 --> 00:24:03,530 it's a lot like what we talked about when we talked synapses. 429 00:24:03,530 --> 00:24:06,560 In that case, we had some delta functions here 430 00:24:06,560 --> 00:24:09,110 corresponding to spikes coming in, 431 00:24:09,110 --> 00:24:13,310 and the synaptic response was like some decaying exponential. 432 00:24:13,310 --> 00:24:18,380 And we slid that over the stimulus. 433 00:24:18,380 --> 00:24:23,340 In this case, we have this fluctuating sensory input, 434 00:24:23,340 --> 00:24:29,240 this light intensity, and we're sliding that linear response 435 00:24:29,240 --> 00:24:32,810 of the neuron over and measuring the overlap 436 00:24:32,810 --> 00:24:35,930 as a function of position, and that's a convolution. 437 00:24:41,970 --> 00:24:44,420 Mathematically, what we're doing is 438 00:24:44,420 --> 00:24:49,190 we're taking this linear kernel, this linear filter, 439 00:24:49,190 --> 00:24:54,260 sliding it over the stimulus, using this variable t, 440 00:24:54,260 --> 00:24:58,310 and we're integrating over this variable tau. 441 00:24:58,310 --> 00:25:03,480 So we have a kernel, D, multiplied by the stimulus 442 00:25:03,480 --> 00:25:05,250 at different times shifts. 443 00:25:05,250 --> 00:25:08,650 We integrate over tau, and that's 444 00:25:08,650 --> 00:25:12,860 the output of our temporal receptive field. 445 00:25:12,860 --> 00:25:18,860 That's the linear output of our receptive field. 446 00:25:18,860 --> 00:25:21,050 And we add that to spontaneous firing right, 447 00:25:21,050 --> 00:25:23,478 and that gives us a time-dependent firing 448 00:25:23,478 --> 00:25:24,270 rate of the neuron. 449 00:25:24,270 --> 00:25:24,770 Yes? 450 00:25:24,770 --> 00:25:29,743 AUDIENCE: So is tau how much we [INAUDIBLE]?? 451 00:25:29,743 --> 00:25:30,910 MICHALE FEE: Great question. 452 00:25:33,550 --> 00:25:44,060 t is the location of this kernel as we're sliding it along. 453 00:25:44,060 --> 00:25:46,910 Tau is the variable that we're integrating over 454 00:25:46,910 --> 00:25:49,470 after we multiply them. 455 00:25:49,470 --> 00:25:50,440 Does that make sense? 456 00:25:50,440 --> 00:25:55,110 So we're going to pick a t, place the kernel down 457 00:25:55,110 --> 00:25:58,065 at that time, multiply this. 458 00:25:58,065 --> 00:26:03,510 And remember, this is 0 everywhere outside of here. 459 00:26:03,510 --> 00:26:06,690 And so we're going to multiply the stimulus by this kernel. 460 00:26:06,690 --> 00:26:10,080 It's going to be-- that product is going to be 0 everywhere 461 00:26:10,080 --> 00:26:11,083 except right in here. 462 00:26:11,083 --> 00:26:12,750 You're going to get a positive bump when 463 00:26:12,750 --> 00:26:15,150 you multiply these two, a positive bump when 464 00:26:15,150 --> 00:26:16,920 you multiply those two. 465 00:26:16,920 --> 00:26:22,420 And the integral over tau gives us this positive peak here. 466 00:26:22,420 --> 00:26:24,970 If we picked a slightly different t 467 00:26:24,970 --> 00:26:27,850 so that this thing was lined up with this positive peak here, 468 00:26:27,850 --> 00:26:32,707 then you'd see that you'd have positive times negative. 469 00:26:32,707 --> 00:26:33,790 That gives you a negative. 470 00:26:33,790 --> 00:26:35,710 When you integrate over tau, that gives you 471 00:26:35,710 --> 00:26:37,560 this negative peak here. 472 00:26:37,560 --> 00:26:39,640 Does that makes sense? 473 00:26:39,640 --> 00:26:41,410 So let's just go back to the math. 474 00:26:41,410 --> 00:26:45,130 So you can see that we're integrating over tau, 475 00:26:45,130 --> 00:26:50,290 but we're sliding the relative position of D and S 476 00:26:50,290 --> 00:26:52,280 with this variable t. 477 00:26:52,280 --> 00:26:52,780 Yes? 478 00:26:52,780 --> 00:26:55,162 AUDIENCE: Is that the [INAUDIBLE]?? 479 00:26:55,162 --> 00:26:55,870 MICHALE FEE: Yes. 480 00:26:55,870 --> 00:26:57,593 S is the stimulus. 481 00:26:57,593 --> 00:26:58,135 AUDIENCE: Oh. 482 00:26:58,135 --> 00:26:59,500 And D is the kernel? 483 00:26:59,500 --> 00:27:02,538 MICHALE FEE: D is the linear kernel. 484 00:27:02,538 --> 00:27:03,038 Yes? 485 00:27:03,038 --> 00:27:06,170 AUDIENCE: [INAUDIBLE] 486 00:27:06,170 --> 00:27:10,970 MICHALE FEE: So nature chooses the shape of the kernel for us. 487 00:27:10,970 --> 00:27:15,380 So that is the receptive field of neurons. 488 00:27:15,380 --> 00:27:18,110 Now, I just made this up to demonstrate 489 00:27:18,110 --> 00:27:19,520 what this process looks like. 490 00:27:19,520 --> 00:27:25,870 But in real life, this is the property of a neuron, 491 00:27:25,870 --> 00:27:28,030 and we're going to figure out how 492 00:27:28,030 --> 00:27:33,190 to extract this property from neurons using 493 00:27:33,190 --> 00:27:35,290 a technique called spike-triggered average, which 494 00:27:35,290 --> 00:27:37,030 we'll get to later. 495 00:27:37,030 --> 00:27:39,640 But for now, what I'm trying to convey 496 00:27:39,640 --> 00:27:44,050 is, if we knew this temporal receptive field of a neuron, 497 00:27:44,050 --> 00:27:47,230 then we could predict the firing rate of the neuron 498 00:27:47,230 --> 00:27:50,050 to a time varying stimulus. 499 00:27:50,050 --> 00:27:51,560 That was a very important question. 500 00:27:51,560 --> 00:27:55,520 Does everyone understand that? 501 00:27:55,520 --> 00:27:59,240 Because it's one of those cases where once you see it 502 00:27:59,240 --> 00:28:01,040 it's pretty obvious, but sometimes I 503 00:28:01,040 --> 00:28:03,100 don't explain it well enough. 504 00:28:06,584 --> 00:28:07,084 Yes? 505 00:28:07,084 --> 00:28:12,570 AUDIENCE: [INAUDIBLE] 506 00:28:12,570 --> 00:28:14,840 MICHALE FEE: Yes. 507 00:28:14,840 --> 00:28:19,380 So I've already flipped it, and sometimes you'll see. 508 00:28:19,380 --> 00:28:22,490 So this is all going this way-- positive tau. 509 00:28:22,490 --> 00:28:24,710 I've flipped it for you already. 510 00:28:24,710 --> 00:28:26,960 Sometimes you'll see it plotted the other way with tau 511 00:28:26,960 --> 00:28:31,100 going positive to the right, but I've 512 00:28:31,100 --> 00:28:32,450 plotted it this way already. 513 00:28:41,060 --> 00:28:41,850 Any questions? 514 00:28:46,930 --> 00:28:49,990 Oh, and so that was actually the very next question. 515 00:28:49,990 --> 00:28:52,630 You might normally-- you might sometimes 516 00:28:52,630 --> 00:28:56,410 see temporal receptive fields plotted 517 00:28:56,410 --> 00:28:59,050 this way with positive tau going to the right. 518 00:28:59,050 --> 00:29:00,340 And kind of meant-- 519 00:29:00,340 --> 00:29:03,310 I always just flip it back over. 520 00:29:03,310 --> 00:29:06,760 Because in this view, you see that what the neuron responds 521 00:29:06,760 --> 00:29:12,130 to is dark followed by light, and then right there 522 00:29:12,130 --> 00:29:18,010 is when you have a peak spiking probability. 523 00:29:18,010 --> 00:29:21,870 Peak firing rate happens right here. 524 00:29:21,870 --> 00:29:22,370 Yes? 525 00:29:22,370 --> 00:29:25,228 AUDIENCE: [INAUDIBLE] 526 00:29:25,228 --> 00:29:27,020 MICHALE FEE: So we're going to get to that. 527 00:29:27,020 --> 00:29:29,680 But typically, neurons in the retina-- 528 00:29:29,680 --> 00:29:32,320 I'll show you an example in the retina. 529 00:29:32,320 --> 00:29:38,080 A typical time scale here might be tens to 100 milliseconds, 530 00:29:38,080 --> 00:29:39,460 so pretty fast. 531 00:29:47,920 --> 00:29:49,710 So that's called the temporal kernel 532 00:29:49,710 --> 00:29:51,360 or the temporal receptive field. 533 00:29:57,280 --> 00:29:59,410 And again, it's linear in the sense 534 00:29:59,410 --> 00:30:04,120 that if you, for example, had a stimulus intensity that just 535 00:30:04,120 --> 00:30:08,380 had this positive bump without the negative bump, then 536 00:30:08,380 --> 00:30:13,820 the response would be lower just by the ratio of areas. 537 00:30:13,820 --> 00:30:16,390 So if you got rid of this big negative bump here, 538 00:30:16,390 --> 00:30:19,150 then the response would be, I don't know, a third as big. 539 00:30:19,150 --> 00:30:20,980 It would be linear in the area. 540 00:30:27,632 --> 00:30:28,760 Let's push on. 541 00:30:31,810 --> 00:30:34,160 So now, let's extend this. 542 00:30:34,160 --> 00:30:36,920 So we've been talking about spatial receptive fields 543 00:30:36,920 --> 00:30:38,590 and temporal receptive fields. 544 00:30:38,590 --> 00:30:41,770 But in reality, you can combine those things together 545 00:30:41,770 --> 00:30:45,490 into a single concept, called a "spatial temporal receptive 546 00:30:45,490 --> 00:30:52,780 field," and that's usually referred to as an STRF. 547 00:30:52,780 --> 00:30:56,200 If you're working in the auditory system, STRF, 548 00:30:56,200 --> 00:30:58,570 it's the same acronym, but it just 549 00:30:58,570 --> 00:31:00,550 means spectral temporal receptive field, 550 00:31:00,550 --> 00:31:03,220 because it's sensitive to the spectral content 551 00:31:03,220 --> 00:31:05,650 of the sounds, not the spatial structure 552 00:31:05,650 --> 00:31:09,100 of the visual stimulus. 553 00:31:09,100 --> 00:31:13,570 So in general, when you have a visual stimulus, 554 00:31:13,570 --> 00:31:17,680 it actually depends on x- and y-coordinates 555 00:31:17,680 --> 00:31:19,830 in the retina and time. 556 00:31:19,830 --> 00:31:24,490 So just I of x and y, which would be like a still image 557 00:31:24,490 --> 00:31:26,320 presented to you. 558 00:31:26,320 --> 00:31:28,190 I of x, y, and t is-- 559 00:31:28,190 --> 00:31:31,840 any movie can be written like that. 560 00:31:31,840 --> 00:31:37,720 Your favorite movie is just some function of I of x, y, and t. 561 00:31:37,720 --> 00:31:44,840 And so we're going to now present to our retina, 562 00:31:44,840 --> 00:31:49,760 and we're going to simplify this by considering 563 00:31:49,760 --> 00:31:51,920 just one spatial dimension. 564 00:31:51,920 --> 00:31:54,080 So we're going to take your favorite movie 565 00:31:54,080 --> 00:31:56,720 and just collapse it into intensity 566 00:31:56,720 --> 00:31:58,413 as a function of position. 567 00:31:58,413 --> 00:32:00,080 It's probably not nearly as interesting, 568 00:32:00,080 --> 00:32:02,970 but it's much easier to analyze. 569 00:32:02,970 --> 00:32:05,750 So we're going to write the firing rate 570 00:32:05,750 --> 00:32:08,300 as a function of time as a spontaneous firing 571 00:32:08,300 --> 00:32:14,570 rate plus a filter, D, which is a spatial temporal receptive 572 00:32:14,570 --> 00:32:18,200 field acting on that intensity. 573 00:32:18,200 --> 00:32:20,900 And you can see that we're doing stuff in here that 574 00:32:20,900 --> 00:32:26,190 looks like a convolution integrating over tau, 575 00:32:26,190 --> 00:32:28,770 and we're also doing stuff that looks like a correlation when 576 00:32:28,770 --> 00:32:30,225 we integrate over x. 577 00:32:33,310 --> 00:32:37,990 So there's the convolution integrating over tau. 578 00:32:37,990 --> 00:32:41,410 What I've done is I've pulled out the D tau, 579 00:32:41,410 --> 00:32:44,640 because we can consider-- 580 00:32:44,640 --> 00:32:48,980 I've just written this as two separate integrals. 581 00:32:48,980 --> 00:32:52,250 So we have an integral over tau that looks like a convolution. 582 00:32:52,250 --> 00:32:55,502 And we have an integral over x that looks like a correlation. 583 00:33:00,130 --> 00:33:03,630 So what is separability mean? 584 00:33:03,630 --> 00:33:07,320 So separability is just a particularly-- 585 00:33:07,320 --> 00:33:09,810 if a receptive field is separable, 586 00:33:09,810 --> 00:33:15,030 it means that you can write down a spatial receptive field 587 00:33:15,030 --> 00:33:18,600 and a temporal receptive field separately. 588 00:33:18,600 --> 00:33:19,690 And that looks like this. 589 00:33:19,690 --> 00:33:23,700 So I imagine that if you have a spatial temporal receptive 590 00:33:23,700 --> 00:33:29,760 field, D, that's a function of position and time. 591 00:33:29,760 --> 00:33:31,170 But you can see that you can just 592 00:33:31,170 --> 00:33:35,550 write it as a product of the spatial part 593 00:33:35,550 --> 00:33:37,890 and the temporal part. 594 00:33:37,890 --> 00:33:41,370 So here, you have a temporal receptive field 595 00:33:41,370 --> 00:33:44,250 that looks like this, a positive lobe here and a negative lobe 596 00:33:44,250 --> 00:33:48,240 there, a spatial receptive field that looks like this, just 597 00:33:48,240 --> 00:33:49,650 a positive lobe. 598 00:33:49,650 --> 00:33:51,990 And if you multiply this function 599 00:33:51,990 --> 00:33:55,560 of x by this function of t, you can 600 00:33:55,560 --> 00:33:58,470 see that you get a function of x and t 601 00:33:58,470 --> 00:34:05,760 that looks like this, where at any position 602 00:34:05,760 --> 00:34:09,380 the function of time just looks like this-- scaled. 603 00:34:09,380 --> 00:34:13,900 And at any time, the spatial receptive field 604 00:34:13,900 --> 00:34:15,960 just looks like this. 605 00:34:15,960 --> 00:34:18,239 Does that make sense? 606 00:34:18,239 --> 00:34:20,130 Other receptive fields are not separable. 607 00:34:20,130 --> 00:34:23,610 You can see that you can't write this receptive 608 00:34:23,610 --> 00:34:28,320 field as a product of a temporal receptive field 609 00:34:28,320 --> 00:34:31,318 and a spatial receptive field. 610 00:34:31,318 --> 00:34:32,429 Does that make sense? 611 00:34:32,429 --> 00:34:35,860 Is that clear why that is? 612 00:34:35,860 --> 00:34:41,449 So basically, you can see that if you take a slice here 613 00:34:41,449 --> 00:34:44,900 at a particular position, you can 614 00:34:44,900 --> 00:34:49,320 see that the temporal pattern here 615 00:34:49,320 --> 00:34:53,350 looks very different than the temporal pattern here. 616 00:34:53,350 --> 00:34:54,870 And so you can't write this simply 617 00:34:54,870 --> 00:34:58,350 as a product of a spatial and a temporal receptive field-- 618 00:34:58,350 --> 00:35:02,030 separable, inseparable. 619 00:35:02,030 --> 00:35:04,130 So let's take a look at what happens when you have 620 00:35:04,130 --> 00:35:06,290 a separable receptive field. 621 00:35:06,290 --> 00:35:07,940 Things kind of become very simple. 622 00:35:07,940 --> 00:35:11,360 We can now write our spatial temporal receptive field 623 00:35:11,360 --> 00:35:14,500 as a spatial receptive field, which 624 00:35:14,500 --> 00:35:17,630 is a function of position, times a temporal receptive field 625 00:35:17,630 --> 00:35:20,190 that's a function of time. 626 00:35:20,190 --> 00:35:23,540 And when you put that into this integral, what you find 627 00:35:23,540 --> 00:35:27,710 is that you can pull that spatial part of the receptive 628 00:35:27,710 --> 00:35:32,050 field out of the temporal integral. 629 00:35:32,050 --> 00:35:33,800 So basically, the way you think about this 630 00:35:33,800 --> 00:35:38,420 is that you find the correlation of the spatial receptive field 631 00:35:38,420 --> 00:35:40,070 with the stimulus, and that gives you 632 00:35:40,070 --> 00:35:43,250 a time-dependent stimulus, a stimulus that's 633 00:35:43,250 --> 00:35:45,110 just a function of time. 634 00:35:45,110 --> 00:35:48,650 Then you can convolve the temporal receptive field 635 00:35:48,650 --> 00:35:51,080 with that time-dependent stimulus. 636 00:35:51,080 --> 00:35:57,620 So you can really just treat it as two separate processes, 637 00:35:57,620 --> 00:35:59,510 which can be kind of convenient just 638 00:35:59,510 --> 00:36:02,030 for thinking about how a neuron will 639 00:36:02,030 --> 00:36:04,060 respond to different stimuli. 640 00:36:06,940 --> 00:36:16,430 So let's just think about, develop some intuition about, 641 00:36:16,430 --> 00:36:18,790 how neurons with a particular receptive field 642 00:36:18,790 --> 00:36:22,660 will respond to a particular stimulus. 643 00:36:22,660 --> 00:36:24,355 So here's what I've done. 644 00:36:24,355 --> 00:36:31,720 I've taken a spatial temporal receptive field here. 645 00:36:31,720 --> 00:36:37,060 This is a function of position and time, 646 00:36:37,060 --> 00:36:40,210 and we're going to figure out how that neuron responds 647 00:36:40,210 --> 00:36:42,040 to this stimulus. 648 00:36:42,040 --> 00:36:46,150 So this stimulus is also a function of space and time. 649 00:36:46,150 --> 00:36:48,940 It's one-dimensional in space. 650 00:36:48,940 --> 00:36:50,810 So what does this look like? 651 00:36:50,810 --> 00:36:56,480 This looks like a bar of light that extends from position 2, 652 00:36:56,480 --> 00:37:00,440 let's say, 2 millimeters to 4 millimeters on our screen. 653 00:37:00,440 --> 00:37:04,220 And it turns on at time point 1, stays on, 654 00:37:04,220 --> 00:37:07,660 and turns off at time point 6. 655 00:37:07,660 --> 00:37:12,136 Let's say 1 second to 6 seconds. 656 00:37:12,136 --> 00:37:14,060 Does that make sense? 657 00:37:14,060 --> 00:37:19,690 So just imagine we have a 1D screen, just a bar, 658 00:37:19,690 --> 00:37:26,930 and we turn on light that's a bar between 2 and 4. 659 00:37:26,930 --> 00:37:28,490 So we turn on a bar of light. 660 00:37:28,490 --> 00:37:32,250 We turn it on at time 1, and we turn it off at time 6. 661 00:37:32,250 --> 00:37:35,340 It's just a very simple case. 662 00:37:35,340 --> 00:37:39,010 We flash of light on at a particular position, 663 00:37:39,010 --> 00:37:41,660 and then we turn it off. 664 00:37:41,660 --> 00:37:44,130 So let's see how this neuron responds. 665 00:37:44,130 --> 00:37:46,520 So what we're going to do is we're going to slide-- 666 00:37:46,520 --> 00:37:50,480 remember, in the 1D case where we had the temporal receptive 667 00:37:50,480 --> 00:37:52,802 field, we just slid it across the stimulus. 668 00:37:52,802 --> 00:37:54,510 So we're going to do the same thing here. 669 00:37:54,510 --> 00:37:58,570 We're going to take that spatial temporal receptive field, 670 00:37:58,570 --> 00:38:02,020 and we're going to slide it across the stimulus. 671 00:38:02,020 --> 00:38:04,090 And we're going to integrate, we're 672 00:38:04,090 --> 00:38:08,670 going to take the product, and we're going to integrate. 673 00:38:08,670 --> 00:38:12,860 And the integral plus the spontaneous rate 674 00:38:12,860 --> 00:38:16,790 is going to be the firing rate of our neuron. 675 00:38:16,790 --> 00:38:18,890 So what is the integral right there? 676 00:38:21,820 --> 00:38:23,380 The product is-- 677 00:38:23,380 --> 00:38:24,082 AUDIENCE: 0. 678 00:38:24,082 --> 00:38:24,790 MICHALE FEE: --0. 679 00:38:24,790 --> 00:38:27,610 where? integrate, it's 0. 680 00:38:27,610 --> 00:38:30,310 So we're going to add a spontaneous firing rate, which 681 00:38:30,310 --> 00:38:32,130 will be right there. 682 00:38:32,130 --> 00:38:33,870 So that will be our firing rate. 683 00:38:33,870 --> 00:38:36,460 Now, let's slide the stimulus a little further. 684 00:38:36,460 --> 00:38:38,785 That means that this time we're asking, 685 00:38:38,785 --> 00:38:40,410 what is the firing rate of that neuron? 686 00:38:40,410 --> 00:38:42,306 So what is it going to look like? 687 00:38:42,306 --> 00:38:43,560 AUDIENCE: Go up a bit. 688 00:38:43,560 --> 00:38:44,910 MICHALE FEE: It's going to go up a little bit, 689 00:38:44,910 --> 00:38:47,430 because we have a positive part of the receptive field. 690 00:38:47,430 --> 00:38:50,730 Green is positive in our pictures here. 691 00:38:50,730 --> 00:38:52,680 It's going to overlap with this bar of light, 692 00:38:52,680 --> 00:38:56,850 because that neuron is sensitive to light between, let's say, 693 00:38:56,850 --> 00:39:02,560 1 and 4, positions 1 and 4 on the screen. 694 00:39:02,560 --> 00:39:06,870 And so the light is falling within the positive part 695 00:39:06,870 --> 00:39:09,390 of that receptive field, and so the neuron's 696 00:39:09,390 --> 00:39:14,260 going to increase its firing rate. 697 00:39:14,260 --> 00:39:17,140 So now what's going to happen? 698 00:39:17,140 --> 00:39:19,263 AUDIENCE: [INAUDIBLE] 699 00:39:19,263 --> 00:39:20,680 MICHALE FEE: It's going to cancel. 700 00:39:20,680 --> 00:39:24,100 You're going to get a positive contribution to the firing rate 701 00:39:24,100 --> 00:39:25,320 here-- 702 00:39:25,320 --> 00:39:28,400 whoops-- and a negative contribution here. 703 00:39:28,400 --> 00:39:30,790 And those two are going to add up. 704 00:39:30,790 --> 00:39:33,430 You're going to multiply that times that. 705 00:39:33,430 --> 00:39:34,360 That gives you a plus. 706 00:39:34,360 --> 00:39:36,040 That times that gives you-- sorry. 707 00:39:36,040 --> 00:39:39,820 That times the light that's shining on it is negative. 708 00:39:39,820 --> 00:39:41,990 Add those up, and it's going to cancel. 709 00:39:41,990 --> 00:39:44,325 And the firing rate's going to go back to baseline. 710 00:39:49,650 --> 00:39:53,340 Now, the light in this receptive field, 711 00:39:53,340 --> 00:39:57,060 we're continuing to slide it in time over our stimulus. 712 00:39:57,060 --> 00:39:58,222 What happens here? 713 00:39:58,222 --> 00:39:59,870 AUDIENCE: Same thing. 714 00:39:59,870 --> 00:40:01,700 MICHALE FEE: Same, good. 715 00:40:01,700 --> 00:40:03,320 How about here? 716 00:40:06,870 --> 00:40:08,247 It's going to go-- 717 00:40:08,247 --> 00:40:08,997 AUDIENCE: Down. 718 00:40:08,997 --> 00:40:09,830 MICHALE FEE: --down. 719 00:40:09,830 --> 00:40:11,660 It's going to dip down, that's right. 720 00:40:11,660 --> 00:40:13,242 And then? 721 00:40:13,242 --> 00:40:17,050 AUDIENCE: [INAUDIBLE] 722 00:40:17,050 --> 00:40:19,460 MICHALE FEE: Yeah. 723 00:40:19,460 --> 00:40:21,550 By 0, you mean the spontaneous firing. 724 00:40:21,550 --> 00:40:24,646 Yeah, exactly. 725 00:40:24,646 --> 00:40:26,280 Cool. 726 00:40:26,280 --> 00:40:26,780 Yes? 727 00:40:26,780 --> 00:40:29,990 AUDIENCE: [INAUDIBLE] the rate of response because the slope 728 00:40:29,990 --> 00:40:32,890 of the line [INAUDIBLE]? 729 00:40:32,890 --> 00:40:37,840 MICHALE FEE: So you should think about this thing sliding 730 00:40:37,840 --> 00:40:40,300 over the stimulus in real time. 731 00:40:40,300 --> 00:40:43,060 So if these are units of seconds, 732 00:40:43,060 --> 00:40:47,410 then this thing is sliding across the stimulus 733 00:40:47,410 --> 00:40:52,100 at 1 second per second sliding across. 734 00:40:52,100 --> 00:40:54,070 And so that is firing rate as a function 735 00:40:54,070 --> 00:40:56,380 of time in those units. 736 00:40:56,380 --> 00:40:58,825 Does that make sense? 737 00:40:58,825 --> 00:40:59,450 AUDIENCE: Yeah. 738 00:40:59,450 --> 00:41:01,800 But why [INAUDIBLE]? 739 00:41:05,137 --> 00:41:05,970 MICHALE FEE: Oh, OK. 740 00:41:05,970 --> 00:41:09,270 Like why doesn't this go up to here? 741 00:41:09,270 --> 00:41:11,402 So what's the answer to that? 742 00:41:11,402 --> 00:41:13,540 AUDIENCE: [INAUDIBLE] 743 00:41:13,540 --> 00:41:14,290 MICHALE FEE: Yeah. 744 00:41:14,290 --> 00:41:17,910 So how would I make that steeper? 745 00:41:17,910 --> 00:41:19,410 How would I make that go up to here? 746 00:41:22,170 --> 00:41:24,010 AUDIENCE: [INAUDIBLE] the light. 747 00:41:24,010 --> 00:41:24,360 MICHALE FEE: What's that? 748 00:41:24,360 --> 00:41:25,550 AUDIENCE: You'd turn the light up. 749 00:41:25,550 --> 00:41:28,110 MICHALE FEE: Yeah, you'd turn the light up, that's right. 750 00:41:28,110 --> 00:41:30,610 This is the receptive field of a neuron, 751 00:41:30,610 --> 00:41:32,940 so we generally can't control that. 752 00:41:32,940 --> 00:41:35,760 So if we wanted to make this neuron respond more, 753 00:41:35,760 --> 00:41:38,271 we'd turn the light up to a higher intensity. 754 00:41:41,640 --> 00:41:43,001 Any other questions? 755 00:41:46,940 --> 00:41:51,600 So neurons often have more complex receptive fields. 756 00:41:51,600 --> 00:41:52,910 So here's an example. 757 00:41:52,910 --> 00:41:55,870 What is this going to do as we slide this across the stimulus? 758 00:42:01,090 --> 00:42:01,590 What's that? 759 00:42:01,590 --> 00:42:03,600 AUDIENCE: [INAUDIBLE] 760 00:42:03,600 --> 00:42:04,410 MICHALE FEE: Yeah. 761 00:42:04,410 --> 00:42:05,240 It's not going to-- 762 00:42:05,240 --> 00:42:06,948 the neuron isn't going to respond at all. 763 00:42:06,948 --> 00:42:14,540 Because as soon as it overlaps, it has a positive contribution. 764 00:42:14,540 --> 00:42:17,840 The light activates these lobes of the receptive field, 765 00:42:17,840 --> 00:42:20,460 but inhibits these lobes of the receptive field. 766 00:42:20,460 --> 00:42:24,543 And the net result is that when you integrate over the product, 767 00:42:24,543 --> 00:42:25,460 you're going to get 0. 768 00:42:28,412 --> 00:42:30,370 Does anyone have any idea what kind of stimulus 769 00:42:30,370 --> 00:42:31,810 might make this neuron respond? 770 00:42:35,340 --> 00:42:39,748 This is a very special kind of receptive field. 771 00:42:39,748 --> 00:42:40,415 Yes, [INAUDIBLE] 772 00:42:40,415 --> 00:42:42,650 AUDIENCE: The light goes from [INAUDIBLE] 773 00:42:42,650 --> 00:42:43,400 MICHALE FEE: Yeah. 774 00:42:43,400 --> 00:42:45,156 What is that called? 775 00:42:45,156 --> 00:42:46,440 AUDIENCE: I'm not sure. 776 00:42:46,440 --> 00:42:49,750 MICHALE FEE: It's called a stimulus that moves. 777 00:42:49,750 --> 00:42:50,880 AUDIENCE: [INAUDIBLE] 778 00:42:50,880 --> 00:42:52,580 MICHALE FEE: Moves-- a moving stimulus. 779 00:42:52,580 --> 00:42:53,090 Good. 780 00:42:53,090 --> 00:42:56,690 So that's a receptive field that response to a moving stimulus. 781 00:42:56,690 --> 00:42:59,807 So let's take a look at that. 782 00:42:59,807 --> 00:43:00,390 So here we go. 783 00:43:03,570 --> 00:43:06,390 Anybody want to take a guess at what this stimulus will 784 00:43:06,390 --> 00:43:07,560 do to this neuron? 785 00:43:07,560 --> 00:43:10,270 Can you visualize sliding it across? 786 00:43:13,406 --> 00:43:17,945 AUDIENCE: [INAUDIBLE] 787 00:43:17,945 --> 00:43:19,070 MICHALE FEE: And then what? 788 00:43:19,070 --> 00:43:19,945 AUDIENCE: [INAUDIBLE] 789 00:43:19,945 --> 00:43:20,700 MICHALE FEE: Yeah. 790 00:43:20,700 --> 00:43:22,970 You can see that this-- 791 00:43:22,970 --> 00:43:27,080 so let's describe what this [AUDIO OUT].. 792 00:43:27,080 --> 00:43:33,450 So we've turned a bar of light on here between 0 and 2, 793 00:43:33,450 --> 00:43:39,460 and then we slide it up over the course of a few seconds. 794 00:43:39,460 --> 00:43:42,540 So we've turned a spot of light on, and then we move it up-- 795 00:43:45,760 --> 00:43:47,420 off. 796 00:43:47,420 --> 00:43:49,670 So it's a spot of light that turns on, moves, and then 797 00:43:49,670 --> 00:43:52,880 disappears. 798 00:43:52,880 --> 00:43:54,290 So let's walk through it. 799 00:43:54,290 --> 00:43:55,720 So there's a little bit of overlap 800 00:43:55,720 --> 00:43:58,120 there, so the neuron's firing rate 801 00:43:58,120 --> 00:44:00,460 is going to start going up. 802 00:44:00,460 --> 00:44:04,940 But then as it goes further, this light 803 00:44:04,940 --> 00:44:08,850 is now activating those inhibitory lobes, 804 00:44:08,850 --> 00:44:14,510 which is going to have a negative contribution. 805 00:44:14,510 --> 00:44:16,100 So when you take the product, you're 806 00:44:16,100 --> 00:44:23,430 going to get lots of negatives there, very little contribution 807 00:44:23,430 --> 00:44:25,950 from the positive lobes, and so the firing rate's 808 00:44:25,950 --> 00:44:26,980 going to go down. 809 00:44:26,980 --> 00:44:30,240 And what happens is it goes down, 810 00:44:30,240 --> 00:44:35,150 and once the firing rate hits 0, it can't go any more negative. 811 00:44:35,150 --> 00:44:38,150 So the firing rate is just going to sit at 0 812 00:44:38,150 --> 00:44:43,200 until this stimulus moves out of the temporal receptor 813 00:44:43,200 --> 00:44:45,390 of this neuron. 814 00:44:45,390 --> 00:44:46,567 And then what? 815 00:44:46,567 --> 00:44:47,418 AUDIENCE: Back up. 816 00:44:47,418 --> 00:44:48,960 MICHALE FEE: It's going to go back up 817 00:44:48,960 --> 00:44:52,110 to the spontaneous rate. 818 00:44:52,110 --> 00:44:57,970 So what kind of stimulus will activate this neuron? 819 00:44:57,970 --> 00:45:06,470 A stimulus that moves from top down. 820 00:45:06,470 --> 00:45:08,910 So let's take a look at that. 821 00:45:08,910 --> 00:45:11,940 So here's our stimulus. 822 00:45:11,940 --> 00:45:15,730 You see that it's going to just hit that inhibitory lobe, 823 00:45:15,730 --> 00:45:16,870 go down a little bit. 824 00:45:16,870 --> 00:45:22,240 And then the excitatory lobes of the receptive field 825 00:45:22,240 --> 00:45:24,760 are going to overlap with the stimulus. 826 00:45:24,760 --> 00:45:28,720 You're going to get a big positive peak, 827 00:45:28,720 --> 00:45:32,860 and then the stimulus will move out of the receptive field, 828 00:45:32,860 --> 00:45:35,410 and the firing rate will go back down to baseline. 829 00:45:51,190 --> 00:45:53,420 Any questions about that? 830 00:45:53,420 --> 00:45:57,520 So that's very common, in both the visual system 831 00:45:57,520 --> 00:46:00,040 and in the auditory system, to have 832 00:46:00,040 --> 00:46:03,340 neurons that are responsive to moving stimuli. 833 00:46:03,340 --> 00:46:07,020 What does moving stimulus mean in the auditory system? 834 00:46:07,020 --> 00:46:08,160 AUDIENCE: Changing pitch. 835 00:46:08,160 --> 00:46:09,618 MICHALE FEE: Right, changing pitch. 836 00:46:09,618 --> 00:46:11,220 So [WHISTLE], like that. 837 00:46:11,220 --> 00:46:13,680 That activated a gazillion neurons 838 00:46:13,680 --> 00:46:18,450 in your brain that are sensitive to upward-going pitches 839 00:46:18,450 --> 00:46:21,540 that you have structure like this. 840 00:46:21,540 --> 00:46:22,440 Isn't that crazy? 841 00:46:22,440 --> 00:46:22,940 [WHISTLE] 842 00:46:22,940 --> 00:46:24,815 I can control all the neurons in your brain-- 843 00:46:24,815 --> 00:46:25,380 [WHISTLE] 844 00:46:26,838 --> 00:46:29,910 [LAUGHS] 845 00:46:29,910 --> 00:46:33,960 --at least the ones that respond to whistles. 846 00:46:33,960 --> 00:46:38,960 So now that we've seen mathematically 847 00:46:38,960 --> 00:46:43,700 how to think about what a receptive field is 848 00:46:43,700 --> 00:46:49,750 and how it interacts with a sensory stimulus, 849 00:46:49,750 --> 00:46:52,170 how do you actually discover what the receptive 850 00:46:52,170 --> 00:46:55,500 field of a neuron is? 851 00:46:55,500 --> 00:47:01,030 That turns out to actually be a very challenging problem. 852 00:47:01,030 --> 00:47:04,350 So in very early parts of the visual 853 00:47:04,350 --> 00:47:08,010 and the auditory system, like in the retina and the LGN, 854 00:47:08,010 --> 00:47:12,630 and as far as, let's say, V1 in visual cortex, 855 00:47:12,630 --> 00:47:15,570 it's been possible to find receptive fields of neurons 856 00:47:15,570 --> 00:47:21,420 by basically just randomly flashing bars and dots of light 857 00:47:21,420 --> 00:47:26,435 and just hoping to get lucky and find what the response is. 858 00:47:26,435 --> 00:47:30,570 It turns out that that's generally a very-- 859 00:47:30,570 --> 00:47:33,080 it can be a very time-consuming process. 860 00:47:33,080 --> 00:47:35,040 And so people have worked out ways 861 00:47:35,040 --> 00:47:37,560 of discovering the receptive fields of neurons 862 00:47:37,560 --> 00:47:39,265 in a much more systematic way. 863 00:47:39,265 --> 00:47:41,640 And that's what we're going to talk about next-- the idea 864 00:47:41,640 --> 00:47:44,328 of a spike-triggered average. 865 00:47:44,328 --> 00:47:45,120 So here's the idea. 866 00:47:45,120 --> 00:47:51,410 We're going to take a stimulus, and we're going to basically-- 867 00:47:51,410 --> 00:47:54,140 we're basically just going to make noise, 868 00:47:54,140 --> 00:47:57,410 just a very noisy stimulus. 869 00:47:57,410 --> 00:48:00,950 So we're going to take, let's say, an intensity, a light, 870 00:48:00,950 --> 00:48:04,145 a spot of light, and we're going to fluctuate the intensity 871 00:48:04,145 --> 00:48:06,620 of that light very rapidly. 872 00:48:06,620 --> 00:48:10,050 And we're going to do that basically with a computer. 873 00:48:10,050 --> 00:48:12,910 We just take a computer, make a random number generator, 874 00:48:12,910 --> 00:48:17,240 hook that up to, let's say, a light source 875 00:48:17,240 --> 00:48:20,570 that we can control the brightness of with a voltage. 876 00:48:20,570 --> 00:48:22,790 And then have the computer generate-- 877 00:48:22,790 --> 00:48:25,550 put out that random number sequence, 878 00:48:25,550 --> 00:48:30,005 control the light level, and then play that to our, 879 00:48:30,005 --> 00:48:32,663 let's say, our visual neuron. 880 00:48:32,663 --> 00:48:34,080 And that neuron is going to spike. 881 00:48:38,340 --> 00:48:43,530 And now, what we can do is take the times of those spikes 882 00:48:43,530 --> 00:48:45,840 and go back figure out basically what 883 00:48:45,840 --> 00:48:50,910 made the neuron fire post-hoc. 884 00:48:50,910 --> 00:48:53,630 So if we do that here, you can basically take the spike times. 885 00:48:53,630 --> 00:48:57,290 Now, you know that whatever made the neuron spike happened 886 00:48:57,290 --> 00:48:59,120 before the spike. 887 00:48:59,120 --> 00:49:01,310 It didn't happen after the spike. 888 00:49:01,310 --> 00:49:03,650 So you can basically ignore whatever 889 00:49:03,650 --> 00:49:07,730 happened after the spike and just consider the stimulus that 890 00:49:07,730 --> 00:49:09,770 came in prior to the spike. 891 00:49:09,770 --> 00:49:13,910 So we're just going to take a little block of the stimulus 892 00:49:13,910 --> 00:49:15,590 prior to the spike, and we're going 893 00:49:15,590 --> 00:49:18,110 to do that for every spike that the neuron generates, 894 00:49:18,110 --> 00:49:21,710 and we're going to pile those up and take an average-- 895 00:49:21,710 --> 00:49:24,260 spike-triggered. 896 00:49:24,260 --> 00:49:25,250 That's it. 897 00:49:33,180 --> 00:49:34,730 And that is going to be-- 898 00:49:34,730 --> 00:49:37,850 what's really cool is that you can show that 899 00:49:37,850 --> 00:49:41,990 under some conditions that spike-triggered average is 900 00:49:41,990 --> 00:49:46,550 actually just the receptive field of the neuron. 901 00:49:46,550 --> 00:49:48,920 It's the linear receptive field of the neuron. 902 00:49:58,240 --> 00:50:01,720 So let's think-- and you can write that down as follows. 903 00:50:01,720 --> 00:50:05,930 We're going to add a stimulus. 904 00:50:05,930 --> 00:50:08,930 We're going to write down the times at which all these spikes 905 00:50:08,930 --> 00:50:12,440 occur, t sub i, or the times in the stimulus 906 00:50:12,440 --> 00:50:14,150 at which the spikes occur. 907 00:50:14,150 --> 00:50:22,290 We're going to take the stimulus at those times minus some tau, 908 00:50:22,290 --> 00:50:26,230 and we're going to average them over all the spikes, all the n 909 00:50:26,230 --> 00:50:27,580 spikes, that we've measured. 910 00:50:31,320 --> 00:50:36,610 And that K of tau is going to be the spike-triggered average, 911 00:50:36,610 --> 00:50:42,120 and in many cases, it's actually the linear kernel. 912 00:50:42,120 --> 00:50:45,300 Now, let's think for a moment about what the conditions are. 913 00:50:45,300 --> 00:50:47,100 What kind of stimulus do you have 914 00:50:47,100 --> 00:50:51,630 to use in order to get the spike-triggered average 915 00:50:51,630 --> 00:50:55,560 to actually be the linear kernel of the neuron, that receptive 916 00:50:55,560 --> 00:50:56,740 field of the neuron? 917 00:50:56,740 --> 00:50:59,160 Any guesses? 918 00:50:59,160 --> 00:51:00,320 Let me give you a hint. 919 00:51:00,320 --> 00:51:02,840 What happens if I take a stimulus that 920 00:51:02,840 --> 00:51:05,660 varies very slowly? 921 00:51:05,660 --> 00:51:07,790 So instead of having these wiggles, 922 00:51:07,790 --> 00:51:10,280 it just goes like this. 923 00:51:10,280 --> 00:51:13,700 It has very slow, random wiggles. 924 00:51:13,700 --> 00:51:18,380 Will that be a good stimulus for extracting the receptive field? 925 00:51:18,380 --> 00:51:18,930 Why is that? 926 00:51:21,450 --> 00:51:21,950 Yes? 927 00:51:21,950 --> 00:51:34,640 AUDIENCE: [INAUDIBLE] 928 00:51:34,640 --> 00:51:35,390 MICHALE FEE: Yeah. 929 00:51:35,390 --> 00:51:41,060 So I think what you're saying is that that stimulus is very 930 00:51:41,060 --> 00:51:44,450 slow, and it doesn't actually have 931 00:51:44,450 --> 00:51:50,270 the fast fluctuations in it that makes the neuron spike. 932 00:51:50,270 --> 00:51:55,250 If the stimulus varies very slowly, then it-- 933 00:51:55,250 --> 00:51:57,780 see, this neuron likes to have this very fast wiggle, 934 00:51:57,780 --> 00:52:00,950 this negative followed by a positive. 935 00:52:00,950 --> 00:52:05,090 But if the stimulus you put in just varies slowly, 936 00:52:05,090 --> 00:52:07,070 then that stimulus doesn't actually 937 00:52:07,070 --> 00:52:11,250 have the kind of signal that's needed to activate this neuron. 938 00:52:11,250 --> 00:52:11,750 Yes? 939 00:52:11,750 --> 00:52:13,738 AUDIENCE: [INAUDIBLE] the stimulus [INAUDIBLE] 940 00:52:13,738 --> 00:52:16,590 smaller than tau? 941 00:52:16,590 --> 00:52:20,850 MICHALE FEE: Well, tau is just the variable that describes 942 00:52:20,850 --> 00:52:25,140 the temporal receptive field. 943 00:52:25,140 --> 00:52:28,620 But I think what you're saying is that the stimulus varies 944 00:52:28,620 --> 00:52:31,530 more slowly than the temporal structure 945 00:52:31,530 --> 00:52:34,740 in the receptive field, in the temporal receptive field. 946 00:52:34,740 --> 00:52:36,390 That's right. 947 00:52:36,390 --> 00:52:40,990 Tau is just this variable that we define the receptive field 948 00:52:40,990 --> 00:52:41,490 on. 949 00:52:41,490 --> 00:52:42,450 Yes, [INAUDIBLE] 950 00:52:42,450 --> 00:52:44,075 AUDIENCE: So when we add up an average, 951 00:52:44,075 --> 00:52:47,495 are we actually adding up from everything 952 00:52:47,495 --> 00:52:50,370 before the spike, everything before the spike? 953 00:52:50,370 --> 00:52:51,780 MICHALE FEE: So great question. 954 00:52:51,780 --> 00:52:55,965 So how far back do you think you would need to average? 955 00:52:55,965 --> 00:52:59,118 AUDIENCE: [INAUDIBLE] 956 00:52:59,118 --> 00:52:59,910 MICHALE FEE: Maybe. 957 00:52:59,910 --> 00:53:02,880 I mean, in principle, you could have spikes happening 958 00:53:02,880 --> 00:53:06,510 very fast, and you could have signal 959 00:53:06,510 --> 00:53:08,940 that affects the response of a neuron from even 960 00:53:08,940 --> 00:53:10,380 before the last spike. 961 00:53:10,380 --> 00:53:13,980 But in general, what do you think the answer 962 00:53:13,980 --> 00:53:15,810 to that question is? 963 00:53:15,810 --> 00:53:17,170 Brainstorm some more ideas. 964 00:53:21,040 --> 00:53:23,700 So let's say that you were recording in the retina, 965 00:53:23,700 --> 00:53:28,800 and you knew that neurons tend to respond to visual stimuli 966 00:53:28,800 --> 00:53:31,290 only for-- 967 00:53:31,290 --> 00:53:34,890 that temporal receptive fields in the retina 968 00:53:34,890 --> 00:53:38,410 never extend back more than 100 milliseconds. 969 00:53:38,410 --> 00:53:41,770 Then how would you choose that window? 970 00:53:41,770 --> 00:53:44,290 You would just choose this window to be 100 milliseconds, 971 00:53:44,290 --> 00:53:45,570 and that would be it. 972 00:53:45,570 --> 00:53:47,610 If you're recording in a brain area 973 00:53:47,610 --> 00:53:50,370 that you really have no idea, then you 974 00:53:50,370 --> 00:53:52,180 have to actually try different things. 975 00:53:52,180 --> 00:53:55,882 So you can try a window that goes back 100 milliseconds. 976 00:53:55,882 --> 00:53:57,840 And if when you do the spike-triggered average, 977 00:53:57,840 --> 00:54:02,858 it hasn't gone to 0 yet, then you need to take more window. 978 00:54:02,858 --> 00:54:03,900 So you can figure it out. 979 00:54:03,900 --> 00:54:06,390 You can create a short window, and it only 980 00:54:06,390 --> 00:54:08,730 takes-- like you change one number in your Matlab code, 981 00:54:08,730 --> 00:54:11,670 and hit Run again, and do it again. 982 00:54:11,670 --> 00:54:14,060 It's pretty simple. 983 00:54:14,060 --> 00:54:15,170 Yes? 984 00:54:15,170 --> 00:54:19,202 AUDIENCE: So when you've got like [INAUDIBLE].. 985 00:54:19,202 --> 00:54:19,910 MICHALE FEE: Yes. 986 00:54:19,910 --> 00:54:23,000 AUDIENCE: Wouldn't that depend [INAUDIBLE] 987 00:54:23,000 --> 00:54:33,490 what kind of filter [INAUDIBLE] 988 00:54:33,490 --> 00:54:35,600 MICHALE FEE: Yeah. 989 00:54:35,600 --> 00:54:38,650 So you're saying that the stimulus that you choose 990 00:54:38,650 --> 00:54:43,260 actually depends on the kinds of filters that the neurons are, 991 00:54:43,260 --> 00:54:44,320 right? 992 00:54:44,320 --> 00:54:48,054 Actually, the right answer, it depends. 993 00:54:48,054 --> 00:54:49,137 AUDIENCE: And so we have-- 994 00:54:49,137 --> 00:54:51,880 [INAUDIBLE] 995 00:54:51,880 --> 00:54:53,500 MICHALE FEE: Yeah. 996 00:54:53,500 --> 00:54:57,550 So generally, the statement is that the stimulus you use 997 00:54:57,550 --> 00:55:01,240 has to have fluctuations in it that are faster 998 00:55:01,240 --> 00:55:03,220 than the fluctuations in the kernel 999 00:55:03,220 --> 00:55:06,060 that you're trying to measure. 1000 00:55:06,060 --> 00:55:10,710 And so most people choose what's called a "white noise 1001 00:55:10,710 --> 00:55:12,390 stimulus." 1002 00:55:12,390 --> 00:55:15,360 And white noise stimulus comes from the idea 1003 00:55:15,360 --> 00:55:19,350 that when you take the spectrum-- 1004 00:55:19,350 --> 00:55:22,630 and we're going to get into spectra next week. 1005 00:55:22,630 --> 00:55:25,170 But when you look at the spectrum of the stimulus, 1006 00:55:25,170 --> 00:55:26,940 and you take the Fourier transform of it 1007 00:55:26,940 --> 00:55:30,560 and look at how much power there is as a function of frequency, 1008 00:55:30,560 --> 00:55:33,780 the spectrum is flat. 1009 00:55:33,780 --> 00:55:41,110 And just like in colors, white light has a flat spectrum. 1010 00:55:41,110 --> 00:55:47,330 And so the term evolved to a noise that has a flat spectrum 1011 00:55:47,330 --> 00:55:48,890 white noise. 1012 00:55:48,890 --> 00:55:52,400 And that's what people generally refer to when they 1013 00:55:52,400 --> 00:55:53,990 do spike-triggered averages. 1014 00:55:53,990 --> 00:55:58,610 They use noise that has a flat spectrum. 1015 00:55:58,610 --> 00:56:02,210 And so you'll often refer to people 1016 00:56:02,210 --> 00:56:05,360 saying that they've used a white noise stimulus to extract 1017 00:56:05,360 --> 00:56:06,970 a spike-triggered average. 1018 00:56:13,090 --> 00:56:18,260 Now, of course, you can't ever make a noise 1019 00:56:18,260 --> 00:56:20,990 that truly has a flat spectrum. 1020 00:56:20,990 --> 00:56:23,960 You have to-- you can only make things 1021 00:56:23,960 --> 00:56:26,720 fluctuate as fast as your experimental setup can 1022 00:56:26,720 --> 00:56:27,620 make them fluctuate. 1023 00:56:27,620 --> 00:56:30,020 So things eventually fall off. 1024 00:56:30,020 --> 00:56:33,140 Fortunately, neurons tend to have receptive fields that 1025 00:56:33,140 --> 00:56:36,620 only have fluctuations that are on the scale of 10 1026 00:56:36,620 --> 00:56:41,287 milliseconds, or maybe a millisecond in extreme cases. 1027 00:56:41,287 --> 00:56:43,912 Maybe in the auditory system you might get a little fluctuation 1028 00:56:43,912 --> 00:56:46,190 for a millisecond, and that's generally pretty 1029 00:56:46,190 --> 00:56:48,140 slow for an experimental setup. 1030 00:56:52,840 --> 00:56:55,240 So you choose a white noise stimulus, 1031 00:56:55,240 --> 00:56:58,240 where white noise means it's got fluctuations 1032 00:56:58,240 --> 00:57:00,760 faster than the fastest fluctuations 1033 00:57:00,760 --> 00:57:08,710 in the temporal receptive field, and that for real neurons tends 1034 00:57:08,710 --> 00:57:12,250 to be, even in early sensory areas, 1035 00:57:12,250 --> 00:57:15,880 tends to be on the scale of millisecond fluctuations. 1036 00:57:15,880 --> 00:57:22,470 And in higher brain areas, they would 1037 00:57:22,470 --> 00:57:24,420 have even slower fluctuations. 1038 00:57:24,420 --> 00:57:27,790 Now, let me just say one word about spike-triggered average. 1039 00:57:27,790 --> 00:57:31,090 It works really well in lower sensor areas. 1040 00:57:31,090 --> 00:57:35,950 But once you get up out of primary sensory areas, 1041 00:57:35,950 --> 00:57:37,990 this method doesn't work any more. 1042 00:57:37,990 --> 00:57:42,220 Neurons don't actually have simple receptive fields. 1043 00:57:42,220 --> 00:57:49,480 And this method starts not working so well outside 1044 00:57:49,480 --> 00:57:52,450 of primary sensory areas. 1045 00:57:52,450 --> 00:57:57,990 But before I get too much into the limitations of this, 1046 00:57:57,990 --> 00:58:00,120 let me just show you some examples where it 1047 00:58:00,120 --> 00:58:01,500 works really beautifully well. 1048 00:58:05,970 --> 00:58:07,440 So this is-- 1049 00:58:07,440 --> 00:58:10,770 I'll show you some slides that I got from Marcus Meister, who 1050 00:58:10,770 --> 00:58:11,460 studies-- 1051 00:58:11,460 --> 00:58:13,320 he's at Caltech. 1052 00:58:13,320 --> 00:58:16,000 He used to be at Harvard, and he studies the retina. 1053 00:58:16,000 --> 00:58:19,410 And so he developed this setup for extracting receptive fields 1054 00:58:19,410 --> 00:58:20,887 of retinal neurons. 1055 00:58:20,887 --> 00:58:21,720 And here's the idea. 1056 00:58:21,720 --> 00:58:24,810 So here's a piece of retina, thats a representation 1057 00:58:24,810 --> 00:58:27,613 of the circuitry in the cells within a piece of retina. 1058 00:58:27,613 --> 00:58:29,280 You extract the retina, and you place it 1059 00:58:29,280 --> 00:58:34,380 on a dish, a special dish, that has electrodes embedded, 1060 00:58:34,380 --> 00:58:36,720 metal electrodes embedded, in the glass, 1061 00:58:36,720 --> 00:58:38,730 sort of on the surface of the glass. 1062 00:58:38,730 --> 00:58:39,990 You take the retina out. 1063 00:58:39,990 --> 00:58:42,430 You press it down onto the glass. 1064 00:58:42,430 --> 00:58:46,050 So now the electrodes are sensing the spiking activity 1065 00:58:46,050 --> 00:58:50,940 of these neurons down here in the retinal ganglion cell 1066 00:58:50,940 --> 00:58:51,990 layer. 1067 00:58:51,990 --> 00:58:54,450 These are the photoreceptors up here. 1068 00:58:54,450 --> 00:58:58,440 And then what he does is he has a computer monitor that's 1069 00:58:58,440 --> 00:59:02,400 generating random patterns of visual stimuli, 1070 00:59:02,400 --> 00:59:04,560 and you project that using a lens down 1071 00:59:04,560 --> 00:59:09,600 onto the photoreceptors of the retina. 1072 00:59:09,600 --> 00:59:12,450 And those neurons now make lots of spikes, 1073 00:59:12,450 --> 00:59:16,920 and you can extract those spikes using the methods that you 1074 00:59:16,920 --> 00:59:21,480 saw in the video from Tuesday. 1075 00:59:21,480 --> 00:59:23,730 So here's what those signals look like. 1076 00:59:23,730 --> 00:59:26,880 This just shows the signal on four different electrodes 1077 00:59:26,880 --> 00:59:29,350 that happen to be right near each other, 1078 00:59:29,350 --> 00:59:33,450 like four adjacent electrodes on this electrode array. 1079 00:59:33,450 --> 00:59:37,260 And you can see that you get spike wave forms on all 1080 00:59:37,260 --> 00:59:39,820 these different electrodes. 1081 00:59:39,820 --> 00:59:44,250 You can see that you get-- 1082 00:59:44,250 --> 00:59:47,790 that you see what looks like lots of cells on these four 1083 00:59:47,790 --> 00:59:48,720 electrodes. 1084 00:59:48,720 --> 00:59:50,310 One really interesting thing to note 1085 00:59:50,310 --> 00:59:52,920 is that these electrodes are actually placed close enough 1086 00:59:52,920 --> 00:59:55,980 together that multiple electrodes detect 1087 00:59:55,980 --> 00:59:58,980 the spike signal from a single cell. 1088 00:59:58,980 --> 01:00:02,085 So you can see right here, here's a spike. 1089 01:00:02,085 --> 01:00:05,740 It exactly lines up with the spike on this other electrode. 1090 01:00:05,740 --> 01:00:07,950 So here is a spike on one electrode that lines up 1091 01:00:07,950 --> 01:00:09,625 with a spike on another electrode. 1092 01:00:09,625 --> 01:00:11,250 You can see there's a little blip there 1093 01:00:11,250 --> 01:00:12,480 and a little blip there. 1094 01:00:12,480 --> 01:00:17,100 All of those spikes are actually from a single cell 1095 01:00:17,100 --> 01:00:19,230 whose electrical activity is picked up 1096 01:00:19,230 --> 01:00:22,520 on four adjacent electrodes. 1097 01:00:22,520 --> 01:00:23,020 David? 1098 01:00:23,020 --> 01:00:24,820 AUDIENCE: Is this the raw data? 1099 01:00:24,820 --> 01:00:25,570 MICHALE FEE: Yeah. 1100 01:00:25,570 --> 01:00:28,888 This is the raw voltage data coming out of those electrodes. 1101 01:00:32,560 --> 01:00:36,060 And you can see here's at a different cell right here. 1102 01:00:36,060 --> 01:00:39,430 You can see that this cell has a peak of voltage fluctuation 1103 01:00:39,430 --> 01:00:40,810 on electrode two. 1104 01:00:40,810 --> 01:00:43,120 You see a little blip there, and a little blip there, 1105 01:00:43,120 --> 01:00:45,080 and nothing there. 1106 01:00:45,080 --> 01:00:48,910 And here is yet another cell that 1107 01:00:48,910 --> 01:00:52,210 has a big peak on electrode three, essentially nothing, 1108 01:00:52,210 --> 01:00:54,460 maybe a small bump there. 1109 01:00:54,460 --> 01:01:02,630 So you can actually extract many different cells 1110 01:01:02,630 --> 01:01:04,490 looking at the patterns of activity that 1111 01:01:04,490 --> 01:01:06,950 appear on nearby electrodes. 1112 01:01:06,950 --> 01:01:10,100 And it turns out that this multi-electrode array system 1113 01:01:10,100 --> 01:01:14,210 is actually very powerful for extracting 1114 01:01:14,210 --> 01:01:17,210 many different cells, the spiking 1115 01:01:17,210 --> 01:01:18,680 activity of many different cells, 1116 01:01:18,680 --> 01:01:21,530 out of a piece of tissue. 1117 01:01:24,660 --> 01:01:27,330 So what you can do is you put this through what's called 1118 01:01:27,330 --> 01:01:30,900 a "spike-sorting algorithm," which uses these different 1119 01:01:30,900 --> 01:01:33,600 spike wave forms on these different electrodes to pull 1120 01:01:33,600 --> 01:01:35,640 out a spike train. 1121 01:01:35,640 --> 01:01:38,760 And the spike train is now going to be a delta function 1122 01:01:38,760 --> 01:01:40,560 for each different neuron that you've 1123 01:01:40,560 --> 01:01:44,130 identified in this data set. 1124 01:01:44,130 --> 01:01:47,778 So even though different neurons appear 1125 01:01:47,778 --> 01:01:49,320 on these different electrodes, you're 1126 01:01:49,320 --> 01:01:51,630 eventually going to extract this now so that you 1127 01:01:51,630 --> 01:01:54,750 have a spike train for one neuron, 1128 01:01:54,750 --> 01:01:56,630 a spike train for another neuron, 1129 01:01:56,630 --> 01:01:58,720 a spike train for a third neuron, and so on. 1130 01:01:58,720 --> 01:02:02,940 And then you can plot the firing rate 1131 01:02:02,940 --> 01:02:04,600 of those different neurons. 1132 01:02:04,600 --> 01:02:09,890 This is actually a histogram, a peristimulus histogram, 1133 01:02:09,890 --> 01:02:15,260 of the activity of a bunch of different neurons 1134 01:02:15,260 --> 01:02:22,900 to a movie being played to this piece of retina in the dish. 1135 01:02:22,900 --> 01:02:27,602 And that's just literally a movie from a forest with trees 1136 01:02:27,602 --> 01:02:28,810 swaying around in the breeze. 1137 01:02:33,403 --> 01:02:35,070 So you have all these different neurons. 1138 01:02:35,070 --> 01:02:40,920 And you can see that each neuron responds to a different feature 1139 01:02:40,920 --> 01:02:42,480 of that movie. 1140 01:02:42,480 --> 01:02:45,900 And that's because each neuron has a receptive field that's 1141 01:02:45,900 --> 01:02:47,820 in a slightly different location, 1142 01:02:47,820 --> 01:02:52,020 has a slightly different spatial and temporal receptive field, 1143 01:02:52,020 --> 01:02:56,010 that it allows it to pick out different features 1144 01:02:56,010 --> 01:02:58,530 of the visual stimulus. 1145 01:02:58,530 --> 01:03:02,460 And there are about a million of these neurons that project 1146 01:03:02,460 --> 01:03:06,480 on the back of the retina. 1147 01:03:06,480 --> 01:03:08,110 Actually, I should be careful. 1148 01:03:08,110 --> 01:03:12,868 It's actually the-- it's the back of the retina. 1149 01:03:12,868 --> 01:03:14,910 Because the light goes through the ganglion cells 1150 01:03:14,910 --> 01:03:17,215 through the photo receptors, which are actually-- 1151 01:03:21,020 --> 01:03:21,650 sorry. 1152 01:03:21,650 --> 01:03:24,870 Photoreceptors are actually on the backside of the retina. 1153 01:03:24,870 --> 01:03:26,700 Ganglion cells are on the front, and light 1154 01:03:26,700 --> 01:03:30,660 goes through the ganglion cells to get to the photoreceptors. 1155 01:03:30,660 --> 01:03:32,790 And there are million of those retinal ganglion 1156 01:03:32,790 --> 01:03:35,010 cells that then project up through the optic nerve 1157 01:03:35,010 --> 01:03:36,585 to the thalamus. 1158 01:03:41,200 --> 01:03:43,810 So how do we figure out what each of those neurons 1159 01:03:43,810 --> 01:03:48,160 is actually responding to in this movie? 1160 01:03:48,160 --> 01:03:50,950 So what we can do is-- 1161 01:03:50,950 --> 01:03:56,100 you could imagine doing a spike-triggered average 1162 01:03:56,100 --> 01:04:00,800 of these neurons to the movie that's 1163 01:04:00,800 --> 01:04:03,690 playing the trees swaying in the breeze. 1164 01:04:03,690 --> 01:04:06,090 But why would you not want to do that? 1165 01:04:06,090 --> 01:04:07,400 Why would that be a bad idea? 1166 01:04:12,840 --> 01:04:15,150 What is it that we just decided is 1167 01:04:15,150 --> 01:04:20,480 the best kind of stimulus to use to extract receptive fields? 1168 01:04:23,620 --> 01:04:26,260 This is a highly structured stimulus 1169 01:04:26,260 --> 01:04:34,310 that's got particular patterns in the stimulus 1170 01:04:34,310 --> 01:04:37,290 and both in space and in time. 1171 01:04:37,290 --> 01:04:39,290 So it's really not an optimal stimulus 1172 01:04:39,290 --> 01:04:41,780 for finding the receptive fields in neurons. 1173 01:04:41,780 --> 01:04:45,740 What we want to do is to make a very noisy stimulus 1174 01:04:45,740 --> 01:04:49,800 that we can play, and that's what they did. 1175 01:04:49,800 --> 01:04:52,370 So then they make this, what they 1176 01:04:52,370 --> 01:04:55,850 call in the visual system, a "random flicker stimulus." 1177 01:04:55,850 --> 01:05:00,140 So it's basically a movie where you randomly choose that 1178 01:05:00,140 --> 01:05:06,530 the pixel values, both in R, G, and B-- red, blue, and green-- 1179 01:05:06,530 --> 01:05:09,127 for each stimulus at each time step. 1180 01:05:09,127 --> 01:05:10,460 And here's what that looks like. 1181 01:05:16,010 --> 01:05:20,250 So now, you play that movie to the retina, 1182 01:05:20,250 --> 01:05:21,810 and you record the spike trains. 1183 01:05:21,810 --> 01:05:24,470 So there's the neurons spiking. 1184 01:05:24,470 --> 01:05:26,670 And now what you do is you-- 1185 01:05:26,670 --> 01:05:29,490 because this is now a two-dimensional stimulus, what 1186 01:05:29,490 --> 01:05:34,650 you do is you have to collect the samples of the movie 1187 01:05:34,650 --> 01:05:40,262 at a bunch of time steps prior to the neurons spiking. 1188 01:05:40,262 --> 01:05:42,690 Does that make sense? 1189 01:05:42,690 --> 01:05:45,960 Now, you do that for each spike that occurs. 1190 01:05:45,960 --> 01:05:48,210 And now, you average those all together 1191 01:05:48,210 --> 01:05:51,270 to produce a little movie of what 1192 01:05:51,270 --> 01:05:55,920 happened on average before each spike of the neuron. 1193 01:05:58,450 --> 01:06:01,038 And here's what that looks like. 1194 01:06:01,038 --> 01:06:02,580 So this is for two different neurons. 1195 01:06:10,050 --> 01:06:12,420 So what is that? 1196 01:06:12,420 --> 01:06:14,970 So this is time across the top. 1197 01:06:14,970 --> 01:06:19,680 So it starts at minus half a second. 1198 01:06:24,170 --> 01:06:25,540 So what did that look like? 1199 01:06:25,540 --> 01:06:28,680 What was it that made that neurons spike? 1200 01:06:28,680 --> 01:06:32,910 What was it that happened right before that neuron spiked? 1201 01:06:32,910 --> 01:06:43,010 AUDIENCE: [INAUDIBLE] 1202 01:06:43,010 --> 01:06:44,390 MICHALE FEE: Yeah, a dark spot. 1203 01:06:44,390 --> 01:06:49,990 So that neuron was excited by a spot of light, 1204 01:06:49,990 --> 01:06:55,170 sorry, by a stimulus that looked like a darkness right 1205 01:06:55,170 --> 01:06:59,120 in that location right there. 1206 01:06:59,120 --> 01:07:04,640 So that neuron is essentially being inhibited by light 1207 01:07:04,640 --> 01:07:05,600 at that location. 1208 01:07:05,600 --> 01:07:09,090 And when the light at that location goes away, 1209 01:07:09,090 --> 01:07:13,997 boom, the neuron is released from inhibition and spikes. 1210 01:07:13,997 --> 01:07:14,830 Here's another cell. 1211 01:07:21,120 --> 01:07:25,150 So that neuron responded to a spot of light 1212 01:07:25,150 --> 01:07:27,340 right there in that location. 1213 01:07:27,340 --> 01:07:30,970 And that's because that neuron gets excitatory input 1214 01:07:30,970 --> 01:07:37,540 from bipolar cells that are located 1215 01:07:37,540 --> 01:07:41,263 in the retina at that location. 1216 01:07:41,263 --> 01:07:42,930 And those bipolar cells respond to input 1217 01:07:42,930 --> 01:07:45,930 from the photoreceptors at that location. 1218 01:07:45,930 --> 01:07:47,610 That's called an on cell. 1219 01:07:47,610 --> 01:07:49,900 That's called an off cell. 1220 01:07:49,900 --> 01:07:54,610 So there are many different kinds of neurons in the retina. 1221 01:07:54,610 --> 01:07:56,490 There's something like-- 1222 01:07:56,490 --> 01:07:58,220 I forget the latest count-- 1223 01:07:58,220 --> 01:08:02,710 40 or 50 different types of retinal ganglion cells that 1224 01:08:02,710 --> 01:08:08,410 have very specific responses to visual stimuli. 1225 01:08:08,410 --> 01:08:11,140 So now let's break that down into a spatial and temporal 1226 01:08:11,140 --> 01:08:12,290 receptive field. 1227 01:08:12,290 --> 01:08:15,280 Most-- I probably shouldn't say most-- 1228 01:08:15,280 --> 01:08:19,689 but many retinal ganglion cells are separable in the sense 1229 01:08:19,689 --> 01:08:22,180 that they have a spatial receptive field 1230 01:08:22,180 --> 01:08:24,100 and a temporal receptive field that are just 1231 01:08:24,100 --> 01:08:26,220 a product of each other. 1232 01:08:26,220 --> 01:08:30,200 The STRF is a product of a spatial and temporal component. 1233 01:08:30,200 --> 01:08:34,210 So here you can see as a function of time 1234 01:08:34,210 --> 01:08:35,560 before the spike. 1235 01:08:35,560 --> 01:08:39,760 So this is the stimulus at the time of the spike. 1236 01:08:39,760 --> 01:08:42,050 This neuron responds with a spike 1237 01:08:42,050 --> 01:08:46,029 to a spot of light that happened about 150 milliseconds earlier. 1238 01:08:50,370 --> 01:08:53,520 And here's what that stimulus looked like as a function 1239 01:08:53,520 --> 01:08:57,450 of space on the retina. 1240 01:08:57,450 --> 01:08:59,609 So that's the spatial receptive field. 1241 01:08:59,609 --> 01:09:02,850 Sorry, that's the spatial temporal receptive field-- 1242 01:09:02,850 --> 01:09:06,100 a spatial stimulus as a function of time. 1243 01:09:06,100 --> 01:09:10,170 You can write that as a product of a spatial receptive field 1244 01:09:10,170 --> 01:09:11,700 and a temporal receptive field. 1245 01:09:16,149 --> 01:09:19,112 So here's what the spatial receptive field 1246 01:09:19,112 --> 01:09:21,279 looks like, and here's the temporal receptive field. 1247 01:09:21,279 --> 01:09:23,950 You can see that this neuron, just like the example that we 1248 01:09:23,950 --> 01:09:26,569 talked about earlier, this neuron 1249 01:09:26,569 --> 01:09:33,910 likes to respond when there's a darkening in the central area, 1250 01:09:33,910 --> 01:09:36,290 followed by a bright spot. 1251 01:09:43,770 --> 01:09:47,620 You can see that little bit of darkening right here. 1252 01:09:47,620 --> 01:09:50,890 So the response when this goes dark and then bright. 1253 01:09:59,720 --> 01:10:01,520 So that's the visual system. 1254 01:10:01,520 --> 01:10:03,910 Let's take a look at auditory - you use this same method 1255 01:10:03,910 --> 01:10:06,250 for finding receptive fields in the auditory system. 1256 01:10:08,800 --> 01:10:10,190 So we're going to talk briefly. 1257 01:10:10,190 --> 01:10:14,530 We're going to come back to spectral analysis 1258 01:10:14,530 --> 01:10:18,910 and spectral processing of signals in a couple 1259 01:10:18,910 --> 01:10:20,920 lectures, but let me just introduce 1260 01:10:20,920 --> 01:10:22,780 some of the basic ideas. 1261 01:10:22,780 --> 01:10:24,670 So we're going to talk about the idea 1262 01:10:24,670 --> 01:10:28,600 of a spectral representation of a sound. 1263 01:10:28,600 --> 01:10:32,371 So this is a microphone signal of-- 1264 01:10:32,371 --> 01:10:35,080 let me see if you can guess what it is-- 1265 01:10:35,080 --> 01:10:37,630 of a creature. 1266 01:10:37,630 --> 01:10:41,060 There are parts of this stimulus that have high frequency. 1267 01:10:41,060 --> 01:10:42,910 So this is a microphone signal. 1268 01:10:42,910 --> 01:10:47,290 It fluctuates due to fluctuations in air pressure 1269 01:10:47,290 --> 01:10:48,790 when you hear something. 1270 01:10:48,790 --> 01:10:52,000 Parts of that signal have high-frequency fluctuations. 1271 01:10:52,000 --> 01:10:54,970 Parts of that signal have low-frequency fluctuations. 1272 01:10:54,970 --> 01:10:57,770 You can compute a Fourier transform-- 1273 01:10:57,770 --> 01:11:00,520 which we'll talk about more later-- 1274 01:11:00,520 --> 01:11:04,750 as a function of time stimulus and see what 1275 01:11:04,750 --> 01:11:07,500 the spectral components are. 1276 01:11:07,500 --> 01:11:11,120 So this is a spectrogram of the sound that I just-- right now. 1277 01:11:11,120 --> 01:11:14,190 But you can see frequency as a function of time, 1278 01:11:14,190 --> 01:11:18,110 and the intensity, or in this case the darkness on the plot, 1279 01:11:18,110 --> 01:11:20,470 shows you how much energy there is 1280 01:11:20,470 --> 01:11:23,390 at a particular frequency at a particular time, 1281 01:11:23,390 --> 01:11:25,830 so frequency as a function of time. 1282 01:11:25,830 --> 01:11:30,200 And now, neurons respond to stimuli like this. 1283 01:11:30,200 --> 01:11:34,010 It's a canary song. 1284 01:11:34,010 --> 01:11:38,630 And neurons respond to different sounds. 1285 01:11:38,630 --> 01:11:42,890 And so you can discover what sounds activate neurons 1286 01:11:42,890 --> 01:11:44,300 by doing the same trick. 1287 01:11:44,300 --> 01:11:45,260 So I'll show you. 1288 01:11:45,260 --> 01:11:50,000 This is from a paper from Michael Merzenich's lab. 1289 01:11:50,000 --> 01:11:52,310 This was worked on by Christophe deCharms 1290 01:11:52,310 --> 01:11:55,560 who was a post-doc in the Merzenich lab. 1291 01:11:55,560 --> 01:11:57,720 And basically, what you can do is-- 1292 01:11:57,720 --> 01:11:58,220 OK. 1293 01:11:58,220 --> 01:12:01,460 So this is for calculating a visual receptive field. 1294 01:12:01,460 --> 01:12:04,310 For calculating an auditory receptive field, what 1295 01:12:04,310 --> 01:12:08,570 you can do is you can basically play noisy stimuli 1296 01:12:08,570 --> 01:12:10,580 in auditory space. 1297 01:12:10,580 --> 01:12:15,230 So what you can do is present random patterns of tones. 1298 01:12:15,230 --> 01:12:20,265 So this is frequency, and this is time. 1299 01:12:20,265 --> 01:12:21,640 And so what you can do is you can 1300 01:12:21,640 --> 01:12:27,850 make a little chords of tone, tone, tone that last, 1301 01:12:27,850 --> 01:12:30,080 let's say, 20 milliseconds. 1302 01:12:30,080 --> 01:12:33,160 And then you make a different random combination of tones, 1303 01:12:33,160 --> 01:12:35,650 and then a different random combination of tones. 1304 01:12:35,650 --> 01:12:41,880 And this sounds like a very scrambled, noisy stimulus. 1305 01:12:41,880 --> 01:12:45,760 And you play this to the animal while you're recording 1306 01:12:45,760 --> 01:12:49,040 a neuron in auditory cortex. 1307 01:12:49,040 --> 01:12:50,800 The neuron spikes. 1308 01:12:50,800 --> 01:12:55,810 And then what you can do is just do exactly the same trick. 1309 01:12:55,810 --> 01:12:57,700 You can look at the stimulus that 1310 01:12:57,700 --> 01:13:02,590 occurred before each spike, pile up those columns. 1311 01:13:02,590 --> 01:13:06,730 There's a little spectrum temporal pattern 1312 01:13:06,730 --> 01:13:09,610 of stimuli that the bird-- 1313 01:13:09,610 --> 01:13:12,460 in this case, a monkey heard right 1314 01:13:12,460 --> 01:13:13,990 before that neuron spiked. 1315 01:13:13,990 --> 01:13:15,270 And you can do the same thing. 1316 01:13:15,270 --> 01:13:19,150 You can take that little snapshot of that sound 1317 01:13:19,150 --> 01:13:22,992 and average them together. 1318 01:13:22,992 --> 01:13:24,700 And here are the kinds of things you see. 1319 01:13:24,700 --> 01:13:28,090 So that's a spectro-temporal receptive field. 1320 01:13:28,090 --> 01:13:29,950 You can see I plotted it. 1321 01:13:29,950 --> 01:13:33,340 It's plotted in a way that this is the stimulus that 1322 01:13:33,340 --> 01:13:35,800 occurs with the spike. 1323 01:13:35,800 --> 01:13:40,510 So this is like the D plotted already flipped. 1324 01:13:43,470 --> 01:13:45,430 And you can see that-- how would you describe 1325 01:13:45,430 --> 01:13:49,653 what this neuron responds to? 1326 01:13:49,653 --> 01:13:50,820 How would you describe that? 1327 01:13:53,500 --> 01:13:54,910 So this is frequently. 1328 01:13:54,910 --> 01:13:58,990 Sorry, this is frequency in kilohertz. 1329 01:13:58,990 --> 01:14:02,490 And that's time in milliseconds. 1330 01:14:02,490 --> 01:14:05,914 So what do you think this neuron responds to? 1331 01:14:05,914 --> 01:14:08,218 AUDIENCE: [INAUDIBLE] 1332 01:14:08,218 --> 01:14:09,260 MICHALE FEE: What's that? 1333 01:14:09,260 --> 01:14:10,908 AUDIENCE: It responds [INAUDIBLE].. 1334 01:14:10,908 --> 01:14:12,850 MICHALE FEE: It responds maximally, actually, 1335 01:14:12,850 --> 01:14:17,680 to a very short tone at 4 kilohertz. 1336 01:14:17,680 --> 01:14:19,970 You see how it kind of has some inhibition there? 1337 01:14:19,970 --> 01:14:22,300 See how it's kind of darker right there? 1338 01:14:22,300 --> 01:14:24,930 So this neuron actually will respond better 1339 01:14:24,930 --> 01:14:28,350 to a tone that only lasts 20 milliseconds than it will 1340 01:14:28,350 --> 01:14:32,380 to a tone that lasts a long time. 1341 01:14:32,380 --> 01:14:36,220 So this is a neuron that responds to a short tone pulse. 1342 01:14:36,220 --> 01:14:38,140 What happens if we play a stimulus 1343 01:14:38,140 --> 01:14:41,650 to this neuron that's broad? 1344 01:14:41,650 --> 01:14:44,170 That instead of just being a tone, [WHISTLE],, 1345 01:14:44,170 --> 01:14:45,460 is broad, like [STATIC]? 1346 01:14:50,120 --> 01:14:54,210 The noise that's at 4 kilohertz here 1347 01:14:54,210 --> 01:14:56,310 will tend to excite the neuron, but the noise 1348 01:14:56,310 --> 01:14:58,770 that's over here at 5 kilohertz or 3 kilohertz 1349 01:14:58,770 --> 01:15:00,640 will tend to inhibit the neuron. 1350 01:15:00,640 --> 01:15:04,080 So the best response, the best stimulus 1351 01:15:04,080 --> 01:15:07,170 to make this neuron respond, is a pure tone 1352 01:15:07,170 --> 01:15:12,440 at 4 kilohertz that lasts about 20-ish milliseconds. 1353 01:15:12,440 --> 01:15:14,380 How about this? 1354 01:15:17,307 --> 01:15:19,140 Let's take a look at this neuron right here. 1355 01:15:19,140 --> 01:15:21,180 What about that neuron? 1356 01:15:21,180 --> 01:15:23,580 What does that neuron want to respond to? 1357 01:15:23,580 --> 01:15:25,810 What does it like to hear? 1358 01:15:25,810 --> 01:15:28,318 I'm anthropomorphizing shamelessly. 1359 01:15:28,318 --> 01:15:29,610 You're not supposed to do that. 1360 01:15:33,140 --> 01:15:36,102 What kind of stimulus drives this neuron? 1361 01:15:36,102 --> 01:15:37,590 AUDIENCE: [INAUDIBLE] 1362 01:15:37,590 --> 01:15:40,900 MICHALE FEE: Good, a downward sweep, tone sweep, 1363 01:15:40,900 --> 01:15:45,840 that goes from about 4 kilohertz to 3 kilohertz. 1364 01:15:45,840 --> 01:15:47,706 In how long? 1365 01:15:47,706 --> 01:15:49,020 AUDIENCE: 100 [INAUDIBLE]. 1366 01:15:49,020 --> 01:15:53,140 MICHALE FEE: In about 100 milliseconds, that's right. 1367 01:15:53,140 --> 01:15:56,860 Maybe 50 would do, [WHISTLE],, like that. 1368 01:16:01,072 --> 01:16:02,065 How about this? 1369 01:16:04,830 --> 01:16:06,820 It's kind of messy, right? 1370 01:16:06,820 --> 01:16:10,410 So you can see that neurons have receptive fields that 1371 01:16:10,410 --> 01:16:14,310 can be very complex in space, or in this case, 1372 01:16:14,310 --> 01:16:17,070 in frequency and time. 1373 01:16:17,070 --> 01:16:21,843 They are very selective to particular patterns 1374 01:16:21,843 --> 01:16:22,510 in the stimulus. 1375 01:16:28,270 --> 01:16:31,550 So we talked about a mathematical version 1376 01:16:31,550 --> 01:16:34,880 of receptive fields, which are essentially 1377 01:16:34,880 --> 01:16:37,250 describing patterns of sensory inputs 1378 01:16:37,250 --> 01:16:38,970 that make a neuron spike. 1379 01:16:38,970 --> 01:16:42,000 And we've talked about a very specific model, called 1380 01:16:42,000 --> 01:16:44,300 a linear/nonlinear model, that describes 1381 01:16:44,300 --> 01:16:48,230 how neurons respond to stimuli or become selective to stimuli. 1382 01:16:48,230 --> 01:16:51,300 We've talked about a spatial receptive field, described 1383 01:16:51,300 --> 01:16:54,560 the response of a neuron as a correlation 1384 01:16:54,560 --> 01:16:56,990 between the spatial receptive field and the stimulus. 1385 01:16:56,990 --> 01:17:00,900 Temporal receptive fields, where we've used convolution 1386 01:17:00,900 --> 01:17:04,370 to predict the response of a neuron to a temporal-- 1387 01:17:04,370 --> 01:17:05,900 to a stimulus. 1388 01:17:05,900 --> 01:17:07,910 We've talked about the idea of a spatio- 1389 01:17:07,910 --> 01:17:11,210 or spectro-temporal receptive field, 1390 01:17:11,210 --> 01:17:14,840 and we've talked about how to use a spike-triggered average 1391 01:17:14,840 --> 01:17:22,410 to extract the spectro-temporal or spatio-temporal receptive 1392 01:17:22,410 --> 01:17:27,240 field of a neuron using white noise or random stimuli.