1 00:00:01,550 --> 00:00:03,920 The following content is provided under a Creative 2 00:00:03,920 --> 00:00:05,310 Commons license. 3 00:00:05,310 --> 00:00:07,520 Your support will help MIT OpenCourseWare 4 00:00:07,520 --> 00:00:11,610 continue to offer high quality educational resources for free. 5 00:00:11,610 --> 00:00:14,180 To make a donation or to view additional materials 6 00:00:14,180 --> 00:00:18,140 from hundreds of MIT courses, visit MIT OpenCourseWare 7 00:00:18,140 --> 00:00:19,026 at ocw.mit.edu. 8 00:00:22,470 --> 00:00:24,450 GILBERT STRANG: OK, let me start again. 9 00:00:24,450 --> 00:00:29,430 Thanks for helping with Professor Rao 10 00:00:29,430 --> 00:00:34,470 and working on the labs the last two times. 11 00:00:34,470 --> 00:00:36,420 I was saying the labs were really 12 00:00:36,420 --> 00:00:39,000 for his class, which is 90 minutes, 13 00:00:39,000 --> 00:00:42,930 so that we were kind of running out of time just answering 14 00:00:42,930 --> 00:00:48,660 those questions, getting the system to say, OK, keep going. 15 00:00:48,660 --> 00:00:53,700 Anyway, he's going to develop those. 16 00:00:53,700 --> 00:00:56,740 And I think they'll be a good thing. 17 00:00:56,740 --> 00:01:04,140 So actually, the second lab, as I was watching it, 18 00:01:04,140 --> 00:01:06,000 I was thinking of a math question. 19 00:01:06,000 --> 00:01:09,940 And may I report to you on that math question? 20 00:01:09,940 --> 00:01:15,810 So you remember the lab was completing a matrix. 21 00:01:15,810 --> 00:01:17,710 You got some entries. 22 00:01:17,710 --> 00:01:22,860 And you can put in any other entries in the other positions. 23 00:01:22,860 --> 00:01:26,775 And the question was can you complete it to a rank 1 matrix? 24 00:01:30,690 --> 00:01:32,110 Here's my question. 25 00:01:32,110 --> 00:01:34,380 What positions are OK? 26 00:01:34,380 --> 00:01:36,870 And what positions could you maybe not 27 00:01:36,870 --> 00:01:41,160 be able to complete to a rank 1 matrix? 28 00:01:41,160 --> 00:01:44,310 Examples are the best. 29 00:01:44,310 --> 00:01:49,370 So I'm looking at this question here. 30 00:01:49,370 --> 00:01:58,890 OK, I'm looking for a rank 1 matrix of the form x column y 31 00:01:58,890 --> 00:02:04,530 transpose, or uv transpose, I guess. 32 00:02:04,530 --> 00:02:11,039 Those are the letters where you used to, uv transpose. 33 00:02:11,039 --> 00:02:17,955 And that means of course that aij is ui times vj. 34 00:02:23,020 --> 00:02:27,140 So we're able to choose-- 35 00:02:27,140 --> 00:02:28,915 we have m u's. 36 00:02:32,740 --> 00:02:34,680 Let me ask a question. 37 00:02:34,680 --> 00:02:40,330 Could I complete-- so let me take m equal n equal 3. 38 00:02:40,330 --> 00:02:42,740 So a 3 by 3 matrix. 39 00:02:42,740 --> 00:02:46,810 I'm going to give you m plus n minus 1, which is 3 40 00:02:46,810 --> 00:02:49,931 plus 3 minus 1, five positions. 41 00:02:55,910 --> 00:03:10,600 And suppose I give non-zeros in which five positions? 42 00:03:10,600 --> 00:03:17,230 So where is that number m plus n minus 1 coming from? 43 00:03:17,230 --> 00:03:20,260 Well, because I have m u's and I have 44 00:03:20,260 --> 00:03:24,490 n v's for a rank 1 matrix-- 45 00:03:24,490 --> 00:03:28,990 i goes from 1 to m, j goes from 1 to n. 46 00:03:28,990 --> 00:03:30,940 So I have m plus n. 47 00:03:30,940 --> 00:03:38,210 But obviously I could require the first u to be a 1. 48 00:03:38,210 --> 00:03:40,300 In other words, when I'm multiplying those, 49 00:03:40,300 --> 00:03:45,700 there is one degree of freedom that is sort of repeated here, 50 00:03:45,700 --> 00:03:50,530 because if I'm just given-- 51 00:03:55,480 --> 00:04:02,710 anyway, you see it, that I can rescale the u 52 00:04:02,710 --> 00:04:05,470 so that the first u is a 1. 53 00:04:05,470 --> 00:04:10,580 And that leave me m plus n minus 1. 54 00:04:10,580 --> 00:04:15,700 So could I give you these five numbers 55 00:04:15,700 --> 00:04:20,490 and could you complete that to a rank 1 matrix? 56 00:04:20,490 --> 00:04:21,899 So those are non-zeros. 57 00:04:21,899 --> 00:04:24,300 Why do I say non-zeros? 58 00:04:24,300 --> 00:04:27,690 Because if that number happened to be a zero, 59 00:04:27,690 --> 00:04:29,370 I'd have bad news there. 60 00:04:29,370 --> 00:04:33,510 If I prescribed that one and I prescribed it to be zero, 61 00:04:33,510 --> 00:04:38,640 then if every column has to be a multiple of that column, 62 00:04:38,640 --> 00:04:40,710 these would all have to be 0. 63 00:04:40,710 --> 00:04:41,970 So I don't want that. 64 00:04:41,970 --> 00:04:47,790 I want to be able to prescribe five numbers. 65 00:04:47,790 --> 00:04:53,970 And therefore, they better not be zero. 66 00:04:53,970 --> 00:04:56,850 Could you complete the job with that choice 67 00:04:56,850 --> 00:05:00,980 of a five positions? 68 00:05:00,980 --> 00:05:02,270 I think you could. 69 00:05:02,270 --> 00:05:06,800 Because how would you decide on that number? 70 00:05:06,800 --> 00:05:08,540 Well, I mean at least one way to look 71 00:05:08,540 --> 00:05:11,630 at it is, what would that number be there? 72 00:05:15,320 --> 00:05:17,740 That number would be chosen. 73 00:05:17,740 --> 00:05:21,860 There would be one and only one choice for that that 74 00:05:21,860 --> 00:05:23,360 would give rank 1. 75 00:05:23,360 --> 00:05:28,730 In fact, that little 2 by 2 determinant would have to be? 76 00:05:28,730 --> 00:05:32,710 0, right? 77 00:05:32,710 --> 00:05:37,690 All the columns are supposed to be multiples of one column. 78 00:05:37,690 --> 00:05:40,360 All the rows are supposed to be multiples of one row. 79 00:05:40,360 --> 00:05:46,830 All the 2 by 2s are rank 1. 80 00:05:46,830 --> 00:05:51,210 And rank 1 for a 2 by 2 means determinant 0. 81 00:05:51,210 --> 00:05:53,910 So I would know what that one had to be. 82 00:05:53,910 --> 00:06:01,320 That number would have to be a 1 2 times a 2 1 over a 1 1. 83 00:06:08,450 --> 00:06:11,090 Every 2 by determinant has to be 0 here. 84 00:06:11,090 --> 00:06:13,940 If I want rank 1, I can't stand any 2 85 00:06:13,940 --> 00:06:18,230 by 2s that are invertible matrices. 86 00:06:18,230 --> 00:06:22,250 So all the determinants have to be zero. 87 00:06:22,250 --> 00:06:25,160 And, of course, then these three numbers 88 00:06:25,160 --> 00:06:27,560 would allow me to fill in there. 89 00:06:27,560 --> 00:06:31,160 Those three numbers would allow me to fill that. 90 00:06:31,160 --> 00:06:33,580 Those three numbers will allow me to fill that. 91 00:06:33,580 --> 00:06:36,770 It would be easy, easy. 92 00:06:36,770 --> 00:06:42,290 So that I can see that those are five positions that work. 93 00:06:42,290 --> 00:06:45,500 Now, give me five positions that won't work. 94 00:06:45,500 --> 00:06:47,000 So this is good. 95 00:06:47,000 --> 00:06:48,410 This is check. 96 00:06:48,410 --> 00:06:52,370 Now, I want one where I can't make it work. 97 00:06:52,370 --> 00:06:56,315 So let me give that number, that number. 98 00:06:59,570 --> 00:07:02,240 So I'm looking for something that fails, 99 00:07:02,240 --> 00:07:04,010 a set of positions that fails. 100 00:07:04,010 --> 00:07:08,090 And then I want to be able to test, does the set of positions 101 00:07:08,090 --> 00:07:13,210 work for any non-zeros or does it not work? 102 00:07:16,235 --> 00:07:17,360 What am I looking for here? 103 00:07:17,360 --> 00:07:20,970 A different one that works? 104 00:07:20,970 --> 00:07:25,320 OK, so what else would work, apart from that? 105 00:07:25,320 --> 00:07:27,870 Let's see, I suppose I take that one. 106 00:07:27,870 --> 00:07:31,290 Now, what about that position there? 107 00:07:31,290 --> 00:07:33,600 I must not choose that one, right? 108 00:07:33,600 --> 00:07:37,260 I must not choose that one, because that would complete a 2 109 00:07:37,260 --> 00:07:37,940 by 2. 110 00:07:37,940 --> 00:07:41,460 And if I gave the wrong numbers, the determinant 111 00:07:41,460 --> 00:07:42,330 wouldn't be zero. 112 00:07:42,330 --> 00:07:43,740 And I would be failed. 113 00:07:43,740 --> 00:07:45,330 So I must not give that position. 114 00:07:45,330 --> 00:07:47,370 Maybe I could give that position. 115 00:07:47,370 --> 00:07:49,550 Maybe I could give this one. 116 00:07:49,550 --> 00:07:51,930 How's that? 117 00:07:51,930 --> 00:07:55,470 If I had those five, would I be able to complete? 118 00:07:55,470 --> 00:07:59,160 It looks good anyway. 119 00:07:59,160 --> 00:08:01,250 It looks good. 120 00:08:01,250 --> 00:08:04,770 Let's see, how would I go about completing that? 121 00:08:04,770 --> 00:08:07,530 Let's see, I would know what that number 122 00:08:07,530 --> 00:08:10,380 had to be from this 2 by 2. 123 00:08:10,380 --> 00:08:14,280 From this 2 by 2, I would know that number. 124 00:08:14,280 --> 00:08:16,980 From this-- where now? 125 00:08:16,980 --> 00:08:21,900 This and this and this, no, that's not any good. 126 00:08:21,900 --> 00:08:24,840 What do I do now? 127 00:08:24,840 --> 00:08:26,420 How would I get this number? 128 00:08:30,380 --> 00:08:33,760 Oh, OK, I guess I have to use-- 129 00:08:33,760 --> 00:08:36,010 this would complete that. 130 00:08:36,010 --> 00:08:39,000 Oh, this 2 by 2, oh, yeah, OK, no problem. 131 00:08:39,000 --> 00:08:41,950 That would complete that one, right? 132 00:08:41,950 --> 00:08:47,710 But this one, I don't think I can get immediately. 133 00:08:47,710 --> 00:08:49,000 But I can get it. 134 00:08:49,000 --> 00:08:53,170 Once I know these, then obviously I can get that, 135 00:08:53,170 --> 00:08:53,950 right? 136 00:08:53,950 --> 00:08:58,840 So it's sort of nice combinatorial problem. 137 00:08:58,840 --> 00:09:01,300 Which five positions will work? 138 00:09:01,300 --> 00:09:03,980 Which five will not work? 139 00:09:03,980 --> 00:09:04,710 How do I tell? 140 00:09:07,510 --> 00:09:10,210 Of course, that's for a 3 by 3. 141 00:09:10,210 --> 00:09:11,530 Yeah, so let me-- 142 00:09:11,530 --> 00:09:15,580 I had like a bigger example. 143 00:09:15,580 --> 00:09:18,390 Let me move up to 4 by 4. 144 00:09:18,390 --> 00:09:24,190 So here comes the idea that I learned last night 145 00:09:24,190 --> 00:09:28,360 from a math professor who does combinatorics. 146 00:09:28,360 --> 00:09:30,350 So these people who do combinatorics, 147 00:09:30,350 --> 00:09:34,570 they know stuff that the rest of us don't. 148 00:09:34,570 --> 00:09:45,850 So his view is these would be rows, one, two, three. 149 00:09:45,850 --> 00:09:48,070 I may add another row. 150 00:09:48,070 --> 00:09:49,860 Let me do a 4 by 4. 151 00:09:49,860 --> 00:09:56,440 And these would be columns, one, two, three, four. 152 00:09:56,440 --> 00:09:58,810 Now, I'm looking for seven-- 153 00:09:58,810 --> 00:10:04,330 so if I prescribe an entry, I'll put in an edge between them. 154 00:10:04,330 --> 00:10:07,630 So let me let me hype this one up to 4 by 4. 155 00:10:11,410 --> 00:10:12,360 Here, a good one. 156 00:10:12,360 --> 00:10:16,980 Actually, I'll just draw this picture. 157 00:10:16,980 --> 00:10:20,100 So suppose I prescribe the 1, 1-- oh, yeah, 158 00:10:20,100 --> 00:10:22,530 and over here I'll show you what I've done. 159 00:10:22,530 --> 00:10:26,160 So that I'm putting in the 1, 1 position. 160 00:10:26,160 --> 00:10:28,620 I'm looking for 7, right? 161 00:10:28,620 --> 00:10:31,860 4 plus 4 minus 1, seven numbers. 162 00:10:31,860 --> 00:10:32,670 OK. 163 00:10:32,670 --> 00:10:37,320 Then, I'll put in, this is row 2 column 1. 164 00:10:37,320 --> 00:10:40,110 Row 2 column 1, that would be there. 165 00:10:40,110 --> 00:10:43,930 Then I'll put in a 2 to 4. 166 00:10:43,930 --> 00:10:49,140 So that would be row 2, column 4. 167 00:10:49,140 --> 00:10:52,530 Then I'll put in a 3 to 4. 168 00:10:52,530 --> 00:10:55,920 So that would be row 3, column 4. 169 00:10:55,920 --> 00:10:57,700 And finally, I'll put in-- 170 00:11:00,290 --> 00:11:03,100 let's see, so how many have I done? 171 00:11:03,100 --> 00:11:05,280 One, two, three, four, and I've got to get up 172 00:11:05,280 --> 00:11:08,970 to seven, prescribed seven. 173 00:11:08,970 --> 00:11:15,060 Well, I'll put more in. 174 00:11:15,060 --> 00:11:19,140 So on row 4, I gave 2, 3, and 4. 175 00:11:22,590 --> 00:11:25,080 And did I put in this one? 176 00:11:25,080 --> 00:11:25,940 Yes, I did. 177 00:11:33,010 --> 00:11:37,090 So I've now prescribed seven numbers. 178 00:11:37,090 --> 00:11:41,230 Could I complete the matrix to a rank 1 matrix? 179 00:11:44,790 --> 00:11:46,340 You see the math question. 180 00:11:46,340 --> 00:11:49,850 If I'm given seven positions there-- 181 00:11:49,850 --> 00:11:52,850 this is another way to look at the same picture. 182 00:11:52,850 --> 00:11:56,810 This is I put in an edge for every x. 183 00:11:56,810 --> 00:12:02,070 So if I have seven x's, I've got seven edges. 184 00:12:02,070 --> 00:12:05,060 And this is called a bipartite graph. 185 00:12:05,060 --> 00:12:07,050 So it's a graph. 186 00:12:07,050 --> 00:12:09,680 I've got four nodes here, four nodes here. 187 00:12:09,680 --> 00:12:11,960 So I have eight nodes. 188 00:12:11,960 --> 00:12:15,800 It's a bipartite graph because I have 189 00:12:15,800 --> 00:12:19,370 one part of nodes over there, one part of nodes here. 190 00:12:19,370 --> 00:12:21,980 Bipartite means two parts. 191 00:12:21,980 --> 00:12:25,460 So the rows give me one part of the edges. 192 00:12:25,460 --> 00:12:28,550 The columns give me the other part of the edges. 193 00:12:28,550 --> 00:12:33,500 And all the connections go between the parts. 194 00:12:33,500 --> 00:12:38,420 I don't have any lines from a row to a row, 195 00:12:38,420 --> 00:12:49,100 because the whole code is that this tells me where those seven 196 00:12:49,100 --> 00:12:50,930 positions are. 197 00:12:50,930 --> 00:12:56,360 Now, I'm just going ask you, can I complete the graph? 198 00:12:56,360 --> 00:12:58,340 Can I complete the matrix? 199 00:12:58,340 --> 00:13:01,460 Can I complete rank 1 matrix there? 200 00:13:04,300 --> 00:13:06,060 What do you think? 201 00:13:06,060 --> 00:13:12,380 And the real question is, what's the rule? 202 00:13:12,380 --> 00:13:16,520 How can I see when I can't complete the matrix, 203 00:13:16,520 --> 00:13:18,290 something gets in the way? 204 00:13:18,290 --> 00:13:23,490 Here, what got in the way was this 2 by 2. 205 00:13:23,490 --> 00:13:27,830 And actually, I asked the question on email, 206 00:13:27,830 --> 00:13:30,620 can I always complete it-- yeah, so that would be 207 00:13:30,620 --> 00:13:32,390 the question I could ask you. 208 00:13:32,390 --> 00:13:35,420 Can I always complete the graph-- 209 00:13:35,420 --> 00:13:43,430 complete the matrix to have rank 1 when I don't run into this? 210 00:13:43,430 --> 00:13:48,050 Is that all I have to avoid, a 2 by 2, 211 00:13:48,050 --> 00:13:52,430 where I'm given all four entries-- oh, yeah, 212 00:13:52,430 --> 00:13:56,600 this I was able to complete. 213 00:13:56,600 --> 00:14:00,200 Sorry to-- let's do one. 214 00:14:00,200 --> 00:14:08,720 If I'm given those entries and maybe some others, 215 00:14:08,720 --> 00:14:11,180 I guess seven altogether, that's a failure. 216 00:14:14,540 --> 00:14:18,440 I can't prescribe any non-zeros in those seven positions, 217 00:14:18,440 --> 00:14:21,380 because if I prescribe any non-zeros, 218 00:14:21,380 --> 00:14:25,790 I probably won't have a zero determinant here. 219 00:14:25,790 --> 00:14:28,260 I won't have rank 1. 220 00:14:28,260 --> 00:14:32,030 That column will not be a multiple of that column. 221 00:14:32,030 --> 00:14:35,390 Whatever I do here, I've screwed up already. 222 00:14:35,390 --> 00:14:36,500 So that's a fail. 223 00:14:43,910 --> 00:14:48,530 Let me take that picture and turn it into this picture, so 224 00:14:48,530 --> 00:14:51,570 rows, columns. 225 00:14:51,570 --> 00:14:55,040 So if I take these seven, that's a row 1-- 226 00:14:55,040 --> 00:15:00,050 row 1, 2, 3, 4, so row 1 goes to 1 and 2. 227 00:15:00,050 --> 00:15:03,470 Row 2 goes to 1 and 2. 228 00:15:03,470 --> 00:15:06,530 Row 3 goes to 4. 229 00:15:06,530 --> 00:15:09,520 And row 4 goes to 3 and 4. 230 00:15:13,290 --> 00:15:15,360 That's a failure. 231 00:15:15,360 --> 00:15:17,040 And how do I know it's a failure? 232 00:15:17,040 --> 00:15:20,190 I want to now come up with the answer. 233 00:15:22,750 --> 00:15:27,180 So if I give any seven positions or any m plus n 234 00:15:27,180 --> 00:15:33,270 minus 1 positions like that, I can create a graph like that, 235 00:15:33,270 --> 00:15:35,730 just following the rule that you saw. 236 00:15:35,730 --> 00:15:41,270 Those seven positions gave me seven edges in my graph. 237 00:15:41,270 --> 00:15:43,980 And it's a bipartite graph because every edge 238 00:15:43,980 --> 00:15:49,230 goes from this part over to this park. 239 00:15:49,230 --> 00:15:51,000 And that's a failure. 240 00:15:51,000 --> 00:15:59,790 And the reason it's a failure is that I have here a cycle. 241 00:15:59,790 --> 00:16:03,390 If I go across, down, across, down, I 242 00:16:03,390 --> 00:16:04,560 come back where I started. 243 00:16:04,560 --> 00:16:10,740 That would be a cycle equals failure. 244 00:16:16,420 --> 00:16:22,480 Everybody see I can't give these four numbers generally. 245 00:16:22,480 --> 00:16:25,120 Once I've given you three, I don't have any freedom 246 00:16:25,120 --> 00:16:27,170 left with that fourth one. 247 00:16:27,170 --> 00:16:30,220 And the way to see that in this picture 248 00:16:30,220 --> 00:16:33,700 is that there's a cycle. 249 00:16:33,700 --> 00:16:35,890 So here is the combinatorics thing 250 00:16:35,890 --> 00:16:42,340 that Professor Postnikov told me about last night. 251 00:16:46,120 --> 00:16:49,630 You can complete to a rank 1 matrix if and only 252 00:16:49,630 --> 00:16:52,750 if no cycles. 253 00:16:52,750 --> 00:16:59,710 So that just answered my question perfectly. 254 00:16:59,710 --> 00:17:04,030 This one is one where I can't complete, and it's got a cycle. 255 00:17:04,030 --> 00:17:08,660 A cycle meaning you come back to where you started. 256 00:17:13,190 --> 00:17:19,560 So my question to him was, can you always complete it 257 00:17:19,560 --> 00:17:21,760 if there are no 2 by 2s in the way? 258 00:17:24,599 --> 00:17:29,760 And his answers told me that maybe the 2 by 2s could be OK, 259 00:17:29,760 --> 00:17:32,160 but there could be a bigger cycle, a longer 260 00:17:32,160 --> 00:17:34,200 cycle that would screw you up. 261 00:17:34,200 --> 00:17:40,590 So let me close with an example of that sort. 262 00:17:40,590 --> 00:17:46,230 So this is going to be fail again. 263 00:17:46,230 --> 00:17:52,470 But no 2 by 2 is responsible. 264 00:17:52,470 --> 00:17:55,950 In other words, it's going to fail. 265 00:17:55,950 --> 00:17:58,680 I won't be able to complete this matrix, 266 00:17:58,680 --> 00:18:03,810 even though there aren't any completed 2 by 2s. 267 00:18:03,810 --> 00:18:07,440 The 2 by 2, I knew immediately was failure. 268 00:18:07,440 --> 00:18:09,000 So let me see if I can do that. 269 00:18:11,850 --> 00:18:14,880 So here's a failure. 270 00:18:14,880 --> 00:18:19,440 This is rows 1, 2, 3, 4. 271 00:18:19,440 --> 00:18:22,000 And this is columns 1, 2, 3, 4. 272 00:18:22,000 --> 00:18:26,790 And I think I'm going to have 1 given there. 273 00:18:26,790 --> 00:18:29,550 And I sort of started this before. 274 00:18:29,550 --> 00:18:34,290 And 2 goes to 1 and 4. 275 00:18:34,290 --> 00:18:37,605 And 3 goes to 4. 276 00:18:40,260 --> 00:18:45,760 And 4 goes to-- 277 00:18:45,760 --> 00:18:46,300 let's see. 278 00:18:49,360 --> 00:18:52,150 OK, now, I've only put it in 2, 4, 5. 279 00:18:52,150 --> 00:18:54,680 And I'm allowed seven. 280 00:18:54,680 --> 00:18:57,760 But I think I'm already in trouble here. 281 00:18:57,760 --> 00:19:01,570 So let me draw this picture for that. 282 00:19:01,570 --> 00:19:06,020 So this is rows 1, 2, 3, 4. 283 00:19:06,020 --> 00:19:10,330 And this is column 1, 2, 3, 4. 284 00:19:10,330 --> 00:19:13,300 And now, I've prescribed that one. 285 00:19:16,930 --> 00:19:19,300 Oh, no, that goes from-- 286 00:19:19,300 --> 00:19:22,000 oh, I've forgotten the right way to do the picture. 287 00:19:26,340 --> 00:19:30,740 My bipartite graph wasn't what it should have been. 288 00:19:30,740 --> 00:19:36,660 So the 1, 2, 3, 4. 289 00:19:36,660 --> 00:19:40,820 OK, now I'm going to do the bipartite graph picture that 290 00:19:40,820 --> 00:19:42,170 goes with this picture. 291 00:19:42,170 --> 00:19:44,870 So 1 to 1. 292 00:19:44,870 --> 00:19:45,545 2 to 1. 293 00:19:48,100 --> 00:19:50,190 2 to 4. 294 00:19:50,190 --> 00:19:53,500 But what I'm going to do-- 295 00:19:53,500 --> 00:19:55,660 let me just say what I'm going to do. 296 00:19:55,660 --> 00:20:01,370 I'm going to create a cycle over here of length 6. 297 00:20:01,370 --> 00:20:06,340 A cycle of length 4 is what I got from a 2 by 2. 298 00:20:06,340 --> 00:20:10,860 From my 2 by 2, that took me one way, back 299 00:20:10,860 --> 00:20:12,560 another, back another, back another. 300 00:20:12,560 --> 00:20:13,920 I completed a cycle. 301 00:20:13,920 --> 00:20:18,390 I came back to where I started with just four edges. 302 00:20:18,390 --> 00:20:23,740 Now, I want to complete a cycle with six edges. 303 00:20:23,740 --> 00:20:27,000 So let me draw it in the picture here. 304 00:20:27,000 --> 00:20:31,410 Now, I'm going to put an edge from 4 back to 1. 305 00:20:31,410 --> 00:20:44,550 So 1 to 1, 1 to 4, 2 to 1, 2 to 4. 306 00:20:44,550 --> 00:20:45,930 Have I got any-- 307 00:20:45,930 --> 00:20:49,060 I think 1, 2, ah, damn. 308 00:20:51,660 --> 00:20:53,280 I didn't want a short cycle. 309 00:20:53,280 --> 00:20:55,740 I want a bigger, longer cycle. 310 00:20:55,740 --> 00:20:57,690 Let me get the darn thing here. 311 00:20:57,690 --> 00:21:01,140 So 2 to 4 is not what I want. 312 00:21:01,140 --> 00:21:02,630 So I start a cycle. 313 00:21:02,630 --> 00:21:04,500 OK, go somewhere. 314 00:21:04,500 --> 00:21:05,460 Go back. 315 00:21:05,460 --> 00:21:06,270 Go somewhere. 316 00:21:06,270 --> 00:21:07,380 Go back. 317 00:21:07,380 --> 00:21:08,610 Go somewhere. 318 00:21:08,610 --> 00:21:09,300 Go back. 319 00:21:09,300 --> 00:21:10,970 Now, there I got it. 320 00:21:10,970 --> 00:21:12,340 Length six. 321 00:21:12,340 --> 00:21:13,030 Length six. 322 00:21:13,030 --> 00:21:16,180 So let me put the x's in where they belong. 323 00:21:16,180 --> 00:21:18,240 So 1 is connected to 1 and 4. 324 00:21:18,240 --> 00:21:19,200 That's right. 325 00:21:19,200 --> 00:21:21,255 2 is connected to 2 and 4. 326 00:21:25,810 --> 00:21:28,920 3 is not connected to anybody. 327 00:21:28,920 --> 00:21:31,690 Let's put that one in. 328 00:21:31,690 --> 00:21:34,910 And 4 is connected to 1 and 2. 329 00:21:34,910 --> 00:21:37,090 4 is connected to 1 and 2. 330 00:21:37,090 --> 00:21:45,280 OK, I believe that picture goes with that picture. 331 00:21:45,280 --> 00:21:49,780 Now, my claim is that there are no 332 00:21:49,780 --> 00:21:57,260 there is no 2 by 2 in here that shows me immediately failure. 333 00:21:57,260 --> 00:21:59,360 But I will fail-- 334 00:21:59,360 --> 00:22:02,420 I can't live with those-- 335 00:22:02,420 --> 00:22:04,520 that looks like eight. 336 00:22:04,520 --> 00:22:08,195 Sorry, I only want seven. 337 00:22:10,980 --> 00:22:12,990 So shall I just take-- 338 00:22:12,990 --> 00:22:14,264 sorry? 339 00:22:14,264 --> 00:22:18,120 AUDIENCE: You've got 3, 4 in the wrong spot. 340 00:22:18,120 --> 00:22:20,120 GILBERT STRANG: 3, 4, and it shouldn't be there. 341 00:22:20,120 --> 00:22:23,130 All right, OK, thanks. 342 00:22:23,130 --> 00:22:24,270 Right. 343 00:22:24,270 --> 00:22:26,400 You see I'm not a combinatorics person. 344 00:22:26,400 --> 00:22:30,450 But it's so beautiful to have the inspiration 345 00:22:30,450 --> 00:22:33,930 to convert that picture to this picture 346 00:22:33,930 --> 00:22:38,410 and then realize that a problem in this picture 347 00:22:38,410 --> 00:22:40,650 is a cycle in this picture. 348 00:22:40,650 --> 00:22:42,360 That's the whole message. 349 00:22:42,360 --> 00:22:46,230 That's the whole message, that when I'm looking here 350 00:22:46,230 --> 00:22:52,320 a cycle means that I've built in a requirement, which 351 00:22:52,320 --> 00:22:54,780 random non-zeros won't satisfy. 352 00:22:54,780 --> 00:22:56,130 So you see that anyway. 353 00:22:56,130 --> 00:23:05,580 You see the cycle, 1, 2, 3, 4, 5, 6, yeah. 354 00:23:05,580 --> 00:23:07,710 So there's a cycle there. 355 00:23:07,710 --> 00:23:13,320 So somehow those six x's, whichever they are-- 356 00:23:13,320 --> 00:23:16,140 I guess all but the 3 by 3. 357 00:23:16,140 --> 00:23:21,030 So I could take away this part of the graph 358 00:23:21,030 --> 00:23:26,820 and just have 3 and 3, and there would be 6 numbers in there 359 00:23:26,820 --> 00:23:29,540 and that's too many. 360 00:23:29,540 --> 00:23:33,573 OK, I'm going to stop there exactly half way 361 00:23:33,573 --> 00:23:34,365 through that class. 362 00:23:37,050 --> 00:23:39,780 Well, I'm going to stop there with a question, which 363 00:23:39,780 --> 00:23:44,100 I don't know if I dare send another question to Professor 364 00:23:44,100 --> 00:23:46,830 Postnikov, who answered this one perfectly. 365 00:23:46,830 --> 00:23:49,950 But my question would be, when could you 366 00:23:49,950 --> 00:23:52,920 complete a rank 2 matrix? 367 00:23:52,920 --> 00:23:58,920 What positions could you fill in and be able to reach rank 2? 368 00:23:58,920 --> 00:24:01,860 That would be trouble, right? 369 00:24:01,860 --> 00:24:05,850 I don't know where we would go with that. 370 00:24:05,850 --> 00:24:11,670 But for us, for 18.065, that's the natural question. 371 00:24:11,670 --> 00:24:14,460 Rank 1 is super special for us. 372 00:24:14,460 --> 00:24:21,330 And rank 2 or rank R would be the general case. 373 00:24:21,330 --> 00:24:23,510 So they're good math questions. 374 00:24:23,510 --> 00:24:29,250 Now, topic 2, convolution. 375 00:24:32,500 --> 00:24:38,450 So the first lab in the course just 376 00:24:38,450 --> 00:24:42,410 happened to have a convolution and used that word and so on. 377 00:24:49,590 --> 00:24:53,030 But we didn't connect that with the lectures. 378 00:24:53,030 --> 00:24:56,150 So now, I'd like to talk about convolutions, 379 00:24:56,150 --> 00:24:59,360 which are extremely important. 380 00:24:59,360 --> 00:25:01,040 And they are important in machine 381 00:25:01,040 --> 00:25:12,470 learning because they give you a set of weights connecting-- 382 00:25:16,060 --> 00:25:18,800 they're an efficient way to-- 383 00:25:18,800 --> 00:25:19,760 let me show you. 384 00:25:19,760 --> 00:25:21,600 Let me show you. 385 00:25:21,600 --> 00:25:23,495 So a convolution matrix-- 386 00:25:30,880 --> 00:25:36,760 and this will be a cyclic convolution matrix. 387 00:25:36,760 --> 00:25:44,030 And the shorthand for cyclic convolution matrix 388 00:25:44,030 --> 00:25:44,822 is circulant. 389 00:25:44,822 --> 00:25:46,280 So what does a circulant look like? 390 00:25:46,280 --> 00:25:48,990 I'll call it C. 391 00:25:48,990 --> 00:25:54,140 A circulant has constant diagonals. 392 00:25:54,140 --> 00:25:57,080 That's what convolution means matrix wise. 393 00:25:57,080 --> 00:26:02,925 Convolution means constant down each diagonal. 394 00:26:13,550 --> 00:26:19,640 And cyclic means complete, circle around again. 395 00:26:19,640 --> 00:26:27,560 The diagonals circle around. 396 00:26:27,560 --> 00:26:29,300 So I'll just show you what I mean. 397 00:26:34,380 --> 00:26:36,500 So 4 by 4, let's say. 398 00:26:36,500 --> 00:26:38,480 So it has some entry to-- 399 00:26:38,480 --> 00:26:41,010 there's one diagonal. 400 00:26:41,010 --> 00:26:42,140 Here is another diagonal. 401 00:26:45,370 --> 00:26:48,130 So now I have constant diagonals. 402 00:26:48,130 --> 00:26:50,440 But if it's going to be a circulant matrix-- 403 00:26:50,440 --> 00:26:55,810 and this is the best family of matrices in the world. 404 00:26:55,810 --> 00:26:59,500 The algebra is just terrific for these matrices. 405 00:26:59,500 --> 00:27:04,130 And that's why they're the heart of signal processing. 406 00:27:04,130 --> 00:27:06,820 So this is a constant diagonal matrix. 407 00:27:06,820 --> 00:27:08,980 But it's not yet a circulant. 408 00:27:08,980 --> 00:27:15,040 That diagonal circles around to be completed. 409 00:27:15,040 --> 00:27:18,050 Every diagonal has four entries here. 410 00:27:18,050 --> 00:27:22,840 So let me take another one, say 1, 1. 411 00:27:22,840 --> 00:27:25,660 Then that would circle around to 1, 1. 412 00:27:25,660 --> 00:27:27,300 And this guy could be 0. 413 00:27:30,190 --> 00:27:31,420 There's a circulant matrix. 414 00:27:34,540 --> 00:27:39,760 Do you understand then what the entry-- 415 00:27:39,760 --> 00:27:44,350 you only need four numbers, say the first column. 416 00:27:44,350 --> 00:27:48,190 If you prescribe the first column of a circulant matrix, 417 00:27:48,190 --> 00:27:52,420 then you've told Matlab, for example, all it needs to know. 418 00:27:52,420 --> 00:27:56,050 If you tell it one column, it can get all the other columns 419 00:27:56,050 --> 00:27:58,320 just by cyclic shift. 420 00:27:58,320 --> 00:28:01,720 There's the first column, 2, 0, 1, 5. 421 00:28:01,720 --> 00:28:06,880 The second column is 2, 0, 1, 5 shifted down by one. 422 00:28:06,880 --> 00:28:11,590 The next column is again shifted down by one, 2, 0, 1, 5. 423 00:28:11,590 --> 00:28:14,680 And the last column is 2, 0, 1, 5. 424 00:28:14,680 --> 00:28:21,790 So they're all the same columns after a cyclic shift. 425 00:28:21,790 --> 00:28:27,610 So the key matrix in this is really a cyclic shift matrix. 426 00:28:27,610 --> 00:28:37,540 Say 0, 1, 0, 0, 0, 0-- so it just has one non-zero diagonal. 427 00:28:37,540 --> 00:28:39,340 And then it's cyclic. 428 00:28:43,370 --> 00:28:48,050 Do you see that in fact this matrix C, 429 00:28:48,050 --> 00:28:54,380 I can produce this matrix C from P and P squared? 430 00:28:54,380 --> 00:28:56,070 What would P squared be? 431 00:28:58,860 --> 00:29:05,030 And I'll write it here without while keeping our eye on P. 432 00:29:05,030 --> 00:29:10,160 So if I square that matrix, what matrix would I get? 433 00:29:10,160 --> 00:29:13,820 So this is a shift by one. 434 00:29:13,820 --> 00:29:17,540 If I shift by one again, that's multiplying it again by P. 435 00:29:17,540 --> 00:29:19,790 So that would give me P squared. 436 00:29:19,790 --> 00:29:21,410 What what's in the-- 437 00:29:25,260 --> 00:29:30,540 OK, so what happens now? 438 00:29:30,540 --> 00:29:34,390 It's a shift by two I guess. 439 00:29:34,390 --> 00:29:37,744 So this will be 0, 0, 1, 0. 440 00:29:37,744 --> 00:29:40,740 This will be 0, 0, 0, 1. 441 00:29:40,740 --> 00:29:43,090 Then it's cyclic still. 442 00:29:43,090 --> 00:29:46,870 So I'll put in the 1s there and then just fill in the 0s. 443 00:29:52,660 --> 00:29:55,700 So P squared, shift by two. 444 00:29:55,700 --> 00:29:57,400 You start out with this one. 445 00:30:01,590 --> 00:30:04,000 Is that right? 446 00:30:04,000 --> 00:30:04,780 Yes. 447 00:30:04,780 --> 00:30:05,290 Yes. 448 00:30:05,290 --> 00:30:06,280 OK. 449 00:30:06,280 --> 00:30:06,850 OK. 450 00:30:06,850 --> 00:30:10,690 Yeah, let's just make it multiply a vector, 451 00:30:10,690 --> 00:30:14,760 x0, x1, x2, x3. 452 00:30:14,760 --> 00:30:17,815 What it does is it puts-- 453 00:30:21,170 --> 00:30:24,160 well, I've started the numbering with 2 here. 454 00:30:24,160 --> 00:30:31,860 x2, x3, x0, and x1. 455 00:30:31,860 --> 00:30:37,710 So it's shifted it by two and cyclically. 456 00:30:37,710 --> 00:30:41,040 So the x2, x3 that got shifted off the bottom 457 00:30:41,040 --> 00:30:42,100 come back to the top. 458 00:30:46,770 --> 00:30:51,690 So the first property is, of circulants-- 459 00:30:51,690 --> 00:30:55,837 so I suppose I have matrices C and D circulants. 460 00:30:59,740 --> 00:31:07,158 So fact 1, C times D is also a circulant. 461 00:31:13,720 --> 00:31:16,360 So if I multiply circulant matrices, 462 00:31:16,360 --> 00:31:18,630 I get more circulant matrices. 463 00:31:18,630 --> 00:31:21,230 And the identity is a circulant matrix. 464 00:31:21,230 --> 00:31:24,300 I have a little group of matrices, the best little group 465 00:31:24,300 --> 00:31:26,910 of matrices there is. 466 00:31:26,910 --> 00:31:30,420 I can multiply two guys in the group, 467 00:31:30,420 --> 00:31:32,470 and I get another one in the group. 468 00:31:32,470 --> 00:31:36,960 Why is CD also a circulant? 469 00:31:36,960 --> 00:31:39,690 Let's see, can we see why that fact is true? 470 00:31:42,230 --> 00:31:45,560 I guess here here's my way to look at it. 471 00:31:45,560 --> 00:31:49,040 A circulant matrix-- let me put it here-- 472 00:31:49,040 --> 00:31:59,230 a circulant matrix, every circulant matrix is C0 times 473 00:31:59,230 --> 00:32:05,860 the identity circulant plus C1 times the single shift 474 00:32:05,860 --> 00:32:12,580 plus C2 times the double shift plus C3 times the triple shift. 475 00:32:12,580 --> 00:32:17,500 That's what it takes to put C0, C1, C2, 476 00:32:17,500 --> 00:32:19,980 and C3 on those diagonals. 477 00:32:19,980 --> 00:32:23,590 C0I is obviously on the main diagonal. 478 00:32:23,590 --> 00:32:27,100 C1P-- remember what P is-- 479 00:32:27,100 --> 00:32:32,470 that puts C1 on the next diagonal. 480 00:32:32,470 --> 00:32:39,580 Then C2P squared, when I square this, I've shifted by one more. 481 00:32:39,580 --> 00:32:42,100 So it will put C2 on that diagonal. 482 00:32:42,100 --> 00:32:45,410 And finally, C3 would go in those positions. 483 00:32:48,020 --> 00:32:55,920 So every circulant matrix is a polynomial in P, 484 00:32:55,920 --> 00:32:57,840 in a single shift. 485 00:32:57,840 --> 00:33:01,290 This is any combination of shifts. 486 00:33:01,290 --> 00:33:03,160 This is a single shift. 487 00:33:03,160 --> 00:33:04,270 That's a double shift. 488 00:33:04,270 --> 00:33:05,410 That's a triple shift. 489 00:33:05,410 --> 00:33:07,140 That's a zero shift. 490 00:33:07,140 --> 00:33:14,870 And if I take combinations I get a circulant matrix. 491 00:33:14,870 --> 00:33:18,720 So now, I can see why CD is also a circulant, 492 00:33:18,720 --> 00:33:25,710 because this is a polynomial in P. 493 00:33:25,710 --> 00:33:31,350 And this is a polynomial in P. And when 494 00:33:31,350 --> 00:33:37,170 I multiply those together, I get a polynomial in P. 495 00:33:37,170 --> 00:33:44,340 But usually, if I multiply a third degree polynomial 496 00:33:44,340 --> 00:33:47,380 by another third degree polynomial-- 497 00:33:47,380 --> 00:33:51,660 so say I'm looking at 4 by 4 circulant matrices-- 498 00:33:51,660 --> 00:33:57,000 so this would be a polynomial of P in P third degree 3. 499 00:34:00,540 --> 00:34:02,760 And this would be a polynomial of degree 3. 500 00:34:05,670 --> 00:34:09,440 So that would give me a polynomial of degree 6. 501 00:34:09,440 --> 00:34:10,745 But I don't want that. 502 00:34:10,745 --> 00:34:14,370 That somehow I have to define multiplication 503 00:34:14,370 --> 00:34:18,230 so that I want this to be degree-- 504 00:34:18,230 --> 00:34:25,010 these are all supposed to be 4 by 4 matrices, 505 00:34:25,010 --> 00:34:26,420 just like that one. 506 00:34:26,420 --> 00:34:31,159 That's twice the identity plus five P plus 1P squared 507 00:34:31,159 --> 00:34:33,020 plus 0P cubed. 508 00:34:33,020 --> 00:34:35,300 And suppose I square that. 509 00:34:35,300 --> 00:34:38,810 Then, again, I'm going to get up a circulant matrix. 510 00:34:38,810 --> 00:34:42,920 And I don't want it to go up to degree 7. 511 00:34:42,920 --> 00:34:47,540 I just want four terms in my polynomial. 512 00:34:47,540 --> 00:34:51,949 I just want the main diagonal and the next three diagonals 513 00:34:51,949 --> 00:34:56,000 and then cycling around completes the matrix. 514 00:34:56,000 --> 00:35:04,020 So how to get degrees 3? 515 00:35:11,610 --> 00:35:13,280 That's the question then. 516 00:35:13,280 --> 00:35:17,130 Can you just tell me the answer? 517 00:35:17,130 --> 00:35:22,260 So I'm multiplying-- yeah, let's just do an example. 518 00:35:22,260 --> 00:35:35,100 So I've C0, C1P, C2P squared, C3P cubed times D0 plus D1P 519 00:35:35,100 --> 00:35:39,730 plus D2P squared and D3P cubed. 520 00:35:39,730 --> 00:35:45,885 So P is 4 by 4 circular shift. 521 00:35:53,920 --> 00:35:57,380 And I'm writing the 4 by 4 matrix that way. 522 00:35:57,380 --> 00:36:01,020 This should be the identity, of course. 523 00:36:01,020 --> 00:36:02,450 You knew that. 524 00:36:02,450 --> 00:36:04,550 OK, so when I multiply those guys, 525 00:36:04,550 --> 00:36:09,650 why do I not get degree six? 526 00:36:09,650 --> 00:36:11,990 Why is that product-- 527 00:36:11,990 --> 00:36:16,475 like P cubed times P cubed C3P cubed times 528 00:36:16,475 --> 00:36:21,810 DP3 cubed, that gives me C3D3 P to the sixth power. 529 00:36:21,810 --> 00:36:23,478 So what's up? 530 00:36:23,478 --> 00:36:24,020 What do I do? 531 00:36:24,020 --> 00:36:24,680 Yeah. 532 00:36:24,680 --> 00:36:26,820 AUDIENCE: Does P to the 4 equal to identity? 533 00:36:26,820 --> 00:36:28,790 GILBERT STRANG: Yes, that's the key. 534 00:36:28,790 --> 00:36:30,670 That's the periodic part. 535 00:36:30,670 --> 00:36:32,900 P to the 4 equal the identity. 536 00:36:32,900 --> 00:36:34,350 Thank you. 537 00:36:34,350 --> 00:36:35,480 Yeah. 538 00:36:35,480 --> 00:36:39,740 So the P to the sixth term is really a P squared term. 539 00:36:39,740 --> 00:36:43,440 The P to the fifth term is really a P term. 540 00:36:43,440 --> 00:36:48,800 P the fourth term is really a P to the zero term. 541 00:36:48,800 --> 00:36:56,570 So I'm just doing cyclic convolution actually. 542 00:36:56,570 --> 00:36:59,870 Let me just say when I'm multiplying-- 543 00:36:59,870 --> 00:37:04,450 yeah, so now, I'm telling you what-- 544 00:37:04,450 --> 00:37:07,100 I can first tell you what convolution is 545 00:37:07,100 --> 00:37:09,170 and then what cyclic convolution is. 546 00:37:09,170 --> 00:37:15,200 And listen up to this, because it's a good thing to know. 547 00:37:15,200 --> 00:37:17,340 So, first of all, convolution. 548 00:37:17,340 --> 00:37:23,480 So suppose I want to take the convolution of 3, 1, 2-- 549 00:37:23,480 --> 00:37:30,620 that's often the symbol for a convolution-- with 4, 6, 1. 550 00:37:33,410 --> 00:37:35,870 So what does that mean? 551 00:37:35,870 --> 00:37:37,400 I've got a vector. 552 00:37:37,400 --> 00:37:45,020 So for me, what's hiding there is polynomial 3 553 00:37:45,020 --> 00:37:53,380 plus x plus 2x squared times 4 plus 6x plus 1x squared. 554 00:37:53,380 --> 00:37:55,340 And I just multiply them out. 555 00:37:55,340 --> 00:37:57,235 And I get 5-- 556 00:38:01,570 --> 00:38:05,830 so I get 3 times 4 gives me a 12. 557 00:38:05,830 --> 00:38:08,650 3 times 6x is 18x. 558 00:38:08,650 --> 00:38:11,060 But I've also got 4x from here. 559 00:38:11,060 --> 00:38:13,780 So I've got 22x. 560 00:38:13,780 --> 00:38:19,150 So I just do that multiplication as of polynomials. 561 00:38:19,150 --> 00:38:23,410 And what I'm doing is convolution with the vectors. 562 00:38:23,410 --> 00:38:30,400 And here's the way you wrote it in first grade. 563 00:38:30,400 --> 00:38:33,070 So here's first grade. 564 00:38:33,070 --> 00:38:36,700 2, 12, 8. 565 00:38:36,700 --> 00:38:39,160 You haven't learned to carry yet. 566 00:38:39,160 --> 00:38:41,950 So 12 goes in there as 12. 567 00:38:41,950 --> 00:38:45,070 Then 1 is 4, 6, 1. 568 00:38:45,070 --> 00:38:49,900 And 3 is 12, 18, 3. 569 00:38:49,900 --> 00:38:51,490 And then you add up. 570 00:38:51,490 --> 00:39:01,090 So you have 2, 13, 17, 22, and 12. 571 00:39:01,090 --> 00:39:03,160 So that's the answer there. 572 00:39:03,160 --> 00:39:04,690 That's the convolution-- 573 00:39:04,690 --> 00:39:06,990 I guess I have to take it from-- 574 00:39:06,990 --> 00:39:09,130 yeah-- 12. 575 00:39:09,130 --> 00:39:11,080 3 times four gave 12. 576 00:39:11,080 --> 00:39:15,100 Now, 3 times 6 plus 1 times 4 that's the 22. 577 00:39:15,100 --> 00:39:17,140 Oh, yeah, we already started here. 578 00:39:17,140 --> 00:39:20,510 But now, what's after that 17? 579 00:39:20,510 --> 00:39:23,432 Why did I say 17? 580 00:39:23,432 --> 00:39:25,692 AUDIENCE: 8 plus 6 plus 3. 581 00:39:25,692 --> 00:39:27,390 GILBERT STRANG: Oh, 8 plus 6 plus 3. 582 00:39:27,390 --> 00:39:28,210 Thanks. 583 00:39:28,210 --> 00:39:30,560 17. 584 00:39:30,560 --> 00:39:32,900 And then 12 plus 1, 13. 585 00:39:32,900 --> 00:39:34,020 And then 2. 586 00:39:34,020 --> 00:39:35,070 That's non-cyclic. 587 00:39:39,950 --> 00:39:41,010 So that's convolution. 588 00:39:41,010 --> 00:39:43,730 Oh, if I want it to be non-cyclic, 589 00:39:43,730 --> 00:39:44,990 I don't put a circle in. 590 00:39:49,040 --> 00:39:51,740 So that's a symbol for ordinary convolution. 591 00:39:51,740 --> 00:39:56,600 If you do the Matlab command conv for convolution, 592 00:39:56,600 --> 00:39:59,390 if you gave it those vectors, it would give you 593 00:39:59,390 --> 00:40:04,200 a vector with five components as a result. 594 00:40:04,200 --> 00:40:09,920 But now, let me make it cyclic. 595 00:40:09,920 --> 00:40:13,360 So what's going to happen now when I make it cyclic? 596 00:40:13,360 --> 00:40:20,540 So this represents 12, 22P, 17P squared, 13P cubed. 597 00:40:20,540 --> 00:40:25,370 And what's 13P cubed? 598 00:40:25,370 --> 00:40:29,450 If n is 3 and I'm doing 3 by 3 matrices, 599 00:40:29,450 --> 00:40:33,940 then 13P cubed is the same as? 600 00:40:33,940 --> 00:40:35,860 13, right? 601 00:40:35,860 --> 00:40:37,360 P cube is I. 602 00:40:37,360 --> 00:40:40,090 So the 13 will go back there. 603 00:40:40,090 --> 00:40:42,670 And the 2 will be P fourth. 604 00:40:42,670 --> 00:40:49,540 And it will go back as P. So now with convolution, 605 00:40:49,540 --> 00:40:59,080 cyclic convolution gives 12 and 13, 25, 22 and 2, 24, 606 00:40:59,080 --> 00:40:59,855 and the 17. 607 00:41:03,590 --> 00:41:09,010 So I'm getting back a vector of length 3 just as I wanted to. 608 00:41:09,010 --> 00:41:15,430 If I have a 3 by 3 matrix, convolution cyclic circulant 609 00:41:15,430 --> 00:41:18,730 matrix, with those three diagonals and these three 610 00:41:18,730 --> 00:41:20,980 diagonals, then I want the answer to be a 3 611 00:41:20,980 --> 00:41:25,220 by 3 matrix with these diagonals. 612 00:41:25,220 --> 00:41:30,830 So again, matrix multiplication of circulant matrices 613 00:41:30,830 --> 00:41:35,750 corresponds exactly to multiplying polynomials 614 00:41:35,750 --> 00:41:37,610 cyclically. 615 00:41:37,610 --> 00:41:41,420 And that's cyclic convolution. 616 00:41:41,420 --> 00:41:43,520 And then there's just one little trick 617 00:41:43,520 --> 00:41:46,340 to see if you've got the numbers. 618 00:41:46,340 --> 00:41:52,790 I believe that it should be true that if 3 plus 1 plus 2 is 6, 619 00:41:52,790 --> 00:41:56,720 4 plus 6 plus 1 is 11, and I believe 620 00:41:56,720 --> 00:42:01,950 that those should add to what? 621 00:42:01,950 --> 00:42:04,026 What do I hope that they add to? 622 00:42:04,026 --> 00:42:04,526 66. 623 00:42:07,600 --> 00:42:12,920 I'm a little nervous about doing it, but I think we can. 624 00:42:12,920 --> 00:42:18,430 Yeah, so 49, 59, 66, check. 625 00:42:18,430 --> 00:42:24,760 Yeah, so the digits, the sum of the digits of one factor 626 00:42:24,760 --> 00:42:27,280 multiplies the sum of the digits of the other factor 627 00:42:27,280 --> 00:42:31,390 to give the sum of the digits in the convolution. 628 00:42:31,390 --> 00:42:32,890 And that would be true whether we're 629 00:42:32,890 --> 00:42:39,230 doing cyclic, which is bringing the 13 and the 2 back in or not 630 00:42:39,230 --> 00:42:41,975 cyclic, where we have 5 numbers. 631 00:42:44,680 --> 00:42:50,680 That's actually a check that I don't know if you ever 632 00:42:50,680 --> 00:42:53,980 thought about that in second grade, 633 00:42:53,980 --> 00:42:57,730 multiplying these numbers, multiplying 634 00:42:57,730 --> 00:43:01,360 that number by that number and getting 635 00:43:01,360 --> 00:43:07,960 some answer, which probably half the class did not get right. 636 00:43:07,960 --> 00:43:12,370 Then fifth grade, they're kind of getting it. 637 00:43:12,370 --> 00:43:16,450 But if they knew a check, so they could have just added 638 00:43:16,450 --> 00:43:19,390 these numbers to get 11, added these numbers 639 00:43:19,390 --> 00:43:22,630 to get 6, multiplied those to get 66, 640 00:43:22,630 --> 00:43:26,180 and check that those added to 66. 641 00:43:28,840 --> 00:43:34,780 That never occurred to me I admit in second grade either. 642 00:43:34,780 --> 00:43:36,450 But now you know. 643 00:43:36,450 --> 00:43:40,830 Now you can pass second grade. 644 00:43:40,830 --> 00:43:46,200 So this is the picture for cyclic convolution. 645 00:43:46,200 --> 00:43:47,850 And I just have a few minutes left 646 00:43:47,850 --> 00:43:51,510 to tell you about the eigenvalues and eigenvectors 647 00:43:51,510 --> 00:43:54,700 of these matrices. 648 00:43:54,700 --> 00:43:57,350 So what can you tell me about the eigenvalues-- 649 00:43:57,350 --> 00:44:01,450 or say, the eigenvectors? 650 00:44:01,450 --> 00:44:06,150 I have a matrix P. It happens to be a permutation matrix. 651 00:44:06,150 --> 00:44:08,310 But it's a matrix. 652 00:44:08,310 --> 00:44:13,650 Then I take this polynomial in the matrix. 653 00:44:13,650 --> 00:44:20,290 What can you tell me about eigenvectors of this matrix C? 654 00:44:20,290 --> 00:44:25,120 So my C looks like that. 655 00:44:25,120 --> 00:44:28,240 And I'm asking you for the eigenvectors of that. 656 00:44:28,240 --> 00:44:37,730 And I'm saying that that is a polynomial, sum of powers of P, 657 00:44:37,730 --> 00:44:39,570 like so. 658 00:44:39,570 --> 00:44:44,460 In fact, the numbers here just come off the diagonals. 659 00:44:44,460 --> 00:44:47,610 So what about the eigenvectors? 660 00:44:47,610 --> 00:44:50,000 AUDIENCE: All 1s. 661 00:44:50,000 --> 00:44:52,070 GILBERT STRANG: Well, that is an eigenvector. 662 00:44:52,070 --> 00:44:52,700 That's true. 663 00:44:52,700 --> 00:44:56,000 The vector of all 1s will be an eigenvector. 664 00:44:56,000 --> 00:44:58,560 With what eigenvalue actually? 665 00:44:58,560 --> 00:45:00,320 AUDIENCE: C2 plus C1. 666 00:45:00,320 --> 00:45:01,760 GILBERT STRANG: Yeah. 667 00:45:01,760 --> 00:45:08,420 So what I want you to see is the eigenvectors of C 668 00:45:08,420 --> 00:45:11,300 are the same as the eigenvectors of P. 669 00:45:11,300 --> 00:45:13,370 If I have an eigenvector of P, that's 670 00:45:13,370 --> 00:45:18,050 also an eigenvector of P squared and P cubed in a combination. 671 00:45:18,050 --> 00:45:32,270 So eigenvectors of P are eigenvectors of C. 672 00:45:32,270 --> 00:45:36,260 So then the question is, what are the eigenvectors of P? 673 00:45:36,260 --> 00:45:38,710 You see what I mean? 674 00:45:38,710 --> 00:45:43,280 It didn't really matter if I'm looking for eigenvectors 675 00:45:43,280 --> 00:45:45,670 what those numbers were. 676 00:45:45,670 --> 00:45:50,620 The point is that those numbers were constant on diagonals 677 00:45:50,620 --> 00:45:56,830 and that therefore it's built out of 1 matrix P, built out 678 00:45:56,830 --> 00:46:02,840 of 1 matrix P. So all I have to know is the eigenvectors of P. 679 00:46:02,840 --> 00:46:05,390 So that's the final step here. 680 00:46:05,390 --> 00:46:08,810 What are the eigenvectors and eigenvalues of P? 681 00:46:13,710 --> 00:46:27,660 So eigenvectors, eigenvalues for P equal 1, 1, 1, and 1, say. 682 00:46:27,660 --> 00:46:28,980 And otherwise zeros. 683 00:46:34,140 --> 00:46:40,170 So somebody mentioned that the vector of 1, 1, 1, 1, that 684 00:46:40,170 --> 00:46:44,340 is an eigenvector, because if I do that multiplication, 685 00:46:44,340 --> 00:46:47,380 it shifts this vector down and cyclically around. 686 00:46:47,380 --> 00:46:51,400 But that just brings back the same vector. 687 00:46:51,400 --> 00:46:53,750 So lambda is 1. 688 00:46:53,750 --> 00:47:03,560 And that's an eigenvector with eigenvalue 1. 689 00:47:03,560 --> 00:47:05,930 But this is a 4 by 4 matrix. 690 00:47:05,930 --> 00:47:08,280 AUDIENCE: 1 minus 1, 1 minus 1. 691 00:47:08,280 --> 00:47:11,320 GILBERT STRANG: Ah, now you're guessing 1 minus 1, 1 minus 1. 692 00:47:13,830 --> 00:47:16,220 Interesting. 693 00:47:16,220 --> 00:47:18,800 Well, tell me an eigenvector for minus 1. 694 00:47:23,102 --> 00:47:27,780 Do you want to just alternate signs there? 695 00:47:27,780 --> 00:47:34,970 So let me let me do that multiplication times 1 minus 1, 696 00:47:34,970 --> 00:47:36,860 1 minus 1. 697 00:47:36,860 --> 00:47:39,350 Of course, I don't have to do that multiplication. 698 00:47:39,350 --> 00:47:41,540 I'm just going to shift this. 699 00:47:41,540 --> 00:47:45,060 So I'm going to shift that down, 1 minus 1 700 00:47:45,060 --> 00:47:48,180 and cyclically around. 701 00:47:48,180 --> 00:47:51,150 And now, what's the eigenvalue? 702 00:47:51,150 --> 00:47:53,540 It is an eigenvector. 703 00:47:53,540 --> 00:48:00,770 And what multiple of that vector gives that vector? 704 00:48:00,770 --> 00:48:01,860 AUDIENCE: Minus 1. 705 00:48:01,860 --> 00:48:02,860 GILBERT STRANG: Minus 1. 706 00:48:07,740 --> 00:48:10,980 So that was a good idea. 707 00:48:10,980 --> 00:48:15,520 But the rest of what you said was a bad idea, 708 00:48:15,520 --> 00:48:24,500 because the next eigenvectors got to be somewhere else. 709 00:48:24,500 --> 00:48:27,200 We've just got one eigenvalue there and one there. 710 00:48:27,200 --> 00:48:29,850 And we haven't got the other two. 711 00:48:29,850 --> 00:48:32,450 So what are the other two eigenvalues 712 00:48:32,450 --> 00:48:36,230 of this permutation matrix? 713 00:48:36,230 --> 00:48:38,580 AUDIENCE: 1. 714 00:48:38,580 --> 00:48:40,140 GILBERT STRANG: 1, that's right. 715 00:48:40,140 --> 00:48:42,750 We're in a circulant world. 716 00:48:42,750 --> 00:48:44,490 Draw a circle. 717 00:48:44,490 --> 00:48:49,530 So the eigenvectors are the four roots of 1. 718 00:48:49,530 --> 00:48:50,660 Complex. 719 00:48:50,660 --> 00:48:53,280 So 1 and minus 1 you've got. 720 00:48:53,280 --> 00:48:55,230 These are the lambdas. 721 00:48:55,230 --> 00:48:58,970 But the other ones are I and minus I. Yeah. 722 00:48:58,970 --> 00:49:01,050 Yeah. 723 00:49:01,050 --> 00:49:04,890 Why don't I, since time's up, let me 724 00:49:04,890 --> 00:49:08,250 leave until Friday the eigenvalues 725 00:49:08,250 --> 00:49:16,710 and eigenvectors for in general, for arbitrary size. 726 00:49:16,710 --> 00:49:19,380 But that's a picture of the eigen-- 727 00:49:19,380 --> 00:49:24,690 we're close to the eigenvectors and eigenvalues of every-- 728 00:49:24,690 --> 00:49:29,070 all circulants have the same eigenvectors. 729 00:49:29,070 --> 00:49:33,900 All circulants have that eigenvector, 4 by 4. 730 00:49:33,900 --> 00:49:36,900 All circulants have that eigenvector 4 by 4. 731 00:49:36,900 --> 00:49:38,610 And we'll find the other two. 732 00:49:38,610 --> 00:49:44,400 OK, so I'll see you Friday for those and the convolution 733 00:49:44,400 --> 00:49:48,810 rule, which is the most important rule in signal 734 00:49:48,810 --> 00:49:49,610 processing. 735 00:49:49,610 --> 00:49:50,110 OK. 736 00:49:50,110 --> 00:49:51,660 Thanks.