1 00:00:01,161 --> 00:00:03,920 ANNOUNCER: The following content is provided under a Creative 2 00:00:03,920 --> 00:00:05,310 Commons license. 3 00:00:05,310 --> 00:00:07,520 Your support will help MIT Open Courseware 4 00:00:07,520 --> 00:00:11,610 continue to offer high quality educational resources for free. 5 00:00:11,610 --> 00:00:14,180 To make a donation or to view additional materials 6 00:00:14,180 --> 00:00:16,670 from hundreds of MIT courses, visit 7 00:00:16,670 --> 00:00:18,540 MITopencourseware@ocw.MIT.edu. 8 00:00:21,746 --> 00:00:23,160 PROFESSOR: So we're really moving 9 00:00:23,160 --> 00:00:27,550 along this review of the highlights of linear algebra. 10 00:00:27,550 --> 00:00:32,460 And today it's matrices Q. They get that name there. 11 00:00:32,460 --> 00:00:35,500 They have orthonormal columns. 12 00:00:35,500 --> 00:00:37,020 So that's what one looks like. 13 00:00:37,020 --> 00:00:40,920 And then the key fact, orthonormal columns 14 00:00:40,920 --> 00:00:47,100 translates directly into that simple fact that you just 15 00:00:47,100 --> 00:00:50,400 keep remembering every time you see Q transpose Q, 16 00:00:50,400 --> 00:00:52,390 you've got the identity matrix. 17 00:00:52,390 --> 00:00:53,820 Let's just see why. 18 00:00:53,820 --> 00:00:57,410 So Q transpose would be-- 19 00:00:57,410 --> 00:01:00,135 I'll take those columns and make them into rows. 20 00:01:02,970 --> 00:01:05,250 And then I multiply by Q with the columns. 21 00:01:08,760 --> 00:01:10,830 And what do I get? 22 00:01:10,830 --> 00:01:13,050 Well hopefully, I get the identity matrix. 23 00:01:19,170 --> 00:01:19,830 Why? 24 00:01:19,830 --> 00:01:24,510 Because-- oh yeah, the normal part tells me that the length 25 00:01:24,510 --> 00:01:25,800 of each vector-- 26 00:01:25,800 --> 00:01:29,190 that's the length squared Q transpose Q-- 27 00:01:29,190 --> 00:01:31,650 the length squared is one. 28 00:01:31,650 --> 00:01:35,190 So that gives me the one in the identity matrix 29 00:01:35,190 --> 00:01:36,900 all along the diagonal. 30 00:01:36,900 --> 00:01:41,040 And then Q transpose times a different Q is zero. 31 00:01:41,040 --> 00:01:43,050 That's the ortho part. 32 00:01:43,050 --> 00:01:45,510 So that gives me the zeros. 33 00:01:45,510 --> 00:01:48,290 So that's a simple identity, but it 34 00:01:48,290 --> 00:01:55,700 translates from a lot of words into a simple expression. 35 00:01:55,700 --> 00:02:02,130 Now does that mean that in the other order, Q, Q transpose, 36 00:02:02,130 --> 00:02:03,810 is that the identity? 37 00:02:03,810 --> 00:02:06,810 So that's a question to think about. 38 00:02:06,810 --> 00:02:10,680 Is Q, Q transpose equal the identity? 39 00:02:10,680 --> 00:02:15,420 Question, sometimes yes, sometimes no-- 40 00:02:15,420 --> 00:02:17,025 easy to tell which. 41 00:02:17,025 --> 00:02:28,830 If the answer is yes, yes when Q is square, the answer is yes. 42 00:02:28,830 --> 00:02:32,640 If Q is a square m a square matrix-- 43 00:02:32,640 --> 00:02:35,550 this is saying that a square matrix 44 00:02:35,550 --> 00:02:39,750 Q has that inverse on its left. 45 00:02:39,750 --> 00:02:42,450 But for square matrices, a left inverse, 46 00:02:42,450 --> 00:02:45,800 Q transpose is also a right inverse. 47 00:02:45,800 --> 00:02:47,490 So for a square matrix, if you have 48 00:02:47,490 --> 00:02:49,210 an inverse that works on one side, 49 00:02:49,210 --> 00:02:51,480 it will work on the other side. 50 00:02:51,480 --> 00:02:54,180 So the answer is yes in that case. 51 00:02:54,180 --> 00:03:00,720 And then in that case, that's the case when we call Q is-- 52 00:03:00,720 --> 00:03:01,740 well, we really-- 53 00:03:01,740 --> 00:03:03,330 I don't know what the right name would 54 00:03:03,330 --> 00:03:06,300 be, but here is the name everybody uses, 55 00:03:06,300 --> 00:03:07,695 an orthogonal matrix. 56 00:03:14,320 --> 00:03:17,900 And that's only in their square case, square. 57 00:03:22,820 --> 00:03:26,330 Q is an orthogonal matrix. 58 00:03:26,330 --> 00:03:29,060 Do you want to just see an example of how that works? 59 00:03:31,590 --> 00:03:34,670 So if Q is rectangular-- 60 00:03:34,670 --> 00:03:39,500 let me do a rectangular Q and a square Q. 61 00:03:39,500 --> 00:03:43,530 So I think there must be a board up there somewhere. 62 00:03:43,530 --> 00:03:44,030 Here. 63 00:03:47,000 --> 00:03:48,770 OK, square. 64 00:03:52,740 --> 00:03:54,730 All right. 65 00:03:54,730 --> 00:03:56,782 Good to see some orthogonal matrices, 66 00:03:56,782 --> 00:03:58,240 because my message is that they are 67 00:03:58,240 --> 00:04:01,340 really important in all kinds of applications. 68 00:04:01,340 --> 00:04:03,220 Let's start two by two. 69 00:04:03,220 --> 00:04:06,160 I can think of two different ways 70 00:04:06,160 --> 00:04:08,320 to get an orthogonal matrix. 71 00:04:08,320 --> 00:04:11,050 That's a two by two matrix. 72 00:04:11,050 --> 00:04:13,800 And one of them you will know immediately, 73 00:04:13,800 --> 00:04:16,970 cos theta sine theta. 74 00:04:16,970 --> 00:04:19,000 So that's a unit vector. 75 00:04:19,000 --> 00:04:19,990 It's normalized. 76 00:04:19,990 --> 00:04:22,360 Cos squared plus sine squared is one. 77 00:04:22,360 --> 00:04:25,250 And this guy has to be orthogonal to it. 78 00:04:25,250 --> 00:04:28,590 So I'll make that minus sine theta and cos theta. 79 00:04:32,020 --> 00:04:35,530 Those are both length one, they're orthogonal, 80 00:04:35,530 --> 00:04:40,960 then this is my Q. And the inverse of Q 81 00:04:40,960 --> 00:04:43,630 will be the transpose. 82 00:04:43,630 --> 00:04:46,900 The transpose would put the minus sign down here 83 00:04:46,900 --> 00:04:49,180 and would produce the inverse matrix. 84 00:04:49,180 --> 00:04:54,970 And what is that particular matrix represent? 85 00:04:54,970 --> 00:05:00,110 Geometrically, where do we see that matrix? 86 00:05:00,110 --> 00:05:01,880 It's a rotation, thank you. 87 00:05:01,880 --> 00:05:05,240 It's a rotation of the whole plane by theta. 88 00:05:05,240 --> 00:05:05,930 Yeah. 89 00:05:05,930 --> 00:05:10,190 So if I apply that to one, zero for example, 90 00:05:10,190 --> 00:05:14,600 I get the first column, which is cos theta sine theta. 91 00:05:14,600 --> 00:05:18,710 And that's just-- let me draw a picture. 92 00:05:18,710 --> 00:05:23,270 That vector one, zero has gotten rotated up to-- 93 00:05:23,270 --> 00:05:26,810 so there's the one, zero. 94 00:05:26,810 --> 00:05:30,260 And there, it got rotated through an angle theta. 95 00:05:30,260 --> 00:05:32,990 And similarly, zero, one will get 96 00:05:32,990 --> 00:05:36,650 rotated through an angle theta to there. 97 00:05:36,650 --> 00:05:40,200 So the whole plane rotates. 98 00:05:40,200 --> 00:05:43,920 Oh, that makes me remember a highly important, 99 00:05:43,920 --> 00:05:50,370 very important property of Q. It doesn't change length. 100 00:05:50,370 --> 00:05:55,260 The length of any vector is the same after you rotate it. 101 00:05:55,260 --> 00:05:59,100 The length of any vector is the same after you multiply by Q. 102 00:05:59,100 --> 00:06:00,810 Can I just do that? 103 00:06:00,810 --> 00:06:05,640 I claim any x, any vector x, I want 104 00:06:05,640 --> 00:06:08,890 to look at the length of Qx. 105 00:06:08,890 --> 00:06:12,630 And I claim it has the same length as x. 106 00:06:12,630 --> 00:06:15,750 Actually, that's the reason in computations 107 00:06:15,750 --> 00:06:20,460 that orthogonal matrix are so much loved, 108 00:06:20,460 --> 00:06:24,330 because no overflow can happen with orthogonal matrices. 109 00:06:24,330 --> 00:06:26,130 The lengths don't change. 110 00:06:26,130 --> 00:06:30,030 I can multiply by any number of orthogonal matrices 111 00:06:30,030 --> 00:06:32,820 and the length don't change. 112 00:06:32,820 --> 00:06:36,960 Can we just see why that's true? 113 00:06:36,960 --> 00:06:38,670 So what do we have to go on? 114 00:06:38,670 --> 00:06:43,740 What we have to go on is Q transpose Q equal I. 115 00:06:43,740 --> 00:06:46,350 Whatever we're going to prove, it's got to come out of that, 116 00:06:46,350 --> 00:06:49,260 because that's all we know. 117 00:06:49,260 --> 00:06:53,852 So how do I use that to get that one? 118 00:06:53,852 --> 00:06:55,810 Well, we haven't said a whole lot about length, 119 00:06:55,810 --> 00:06:57,460 but you'll see it all now. 120 00:06:57,460 --> 00:07:01,120 It'll be easier to prove that the squares are the same. 121 00:07:01,120 --> 00:07:03,760 So what's the what's the matrix expression 122 00:07:03,760 --> 00:07:05,410 for the length squared? 123 00:07:05,410 --> 00:07:08,920 What's the right hand side of that equation? 124 00:07:08,920 --> 00:07:11,200 X transpose x, right? 125 00:07:11,200 --> 00:07:14,560 X transpose x gives me the sum of the squares. 126 00:07:14,560 --> 00:07:17,670 Pythagoras says that's the length squared. 127 00:07:17,670 --> 00:07:21,030 So that right hand side is x transpose x. 128 00:07:21,030 --> 00:07:23,530 What's the left side? 129 00:07:23,530 --> 00:07:25,810 It's the length squared of this, of Qx. 130 00:07:25,810 --> 00:07:32,290 So it must be the same as Qx transpose Qx. 131 00:07:32,290 --> 00:07:36,510 And the claim is that that equation holds. 132 00:07:36,510 --> 00:07:40,030 And do you see it? 133 00:07:40,030 --> 00:07:45,710 So any property from Q has to just come out directly 134 00:07:45,710 --> 00:07:46,660 from that. 135 00:07:46,660 --> 00:07:48,430 Where is it here? 136 00:07:48,430 --> 00:07:52,730 Do I just like, push away a little bit at that left hand 137 00:07:52,730 --> 00:07:55,070 side and see it? 138 00:07:55,070 --> 00:08:00,500 Qx transpose is the same as x transpose Q transpose. 139 00:08:00,500 --> 00:08:03,500 And Qx is Qx. 140 00:08:03,500 --> 00:08:05,560 And now I'm seeing-- 141 00:08:05,560 --> 00:08:07,130 well, you might say, wait a minute. 142 00:08:07,130 --> 00:08:09,650 The parentheses were there and there. 143 00:08:09,650 --> 00:08:13,160 But I say the most important law for matrix multiplication 144 00:08:13,160 --> 00:08:16,560 is you can move parentheses or throw them away. 145 00:08:16,560 --> 00:08:17,970 Let's throw them away. 146 00:08:17,970 --> 00:08:21,230 So in here I'm seeing Q transpose Q, 147 00:08:21,230 --> 00:08:22,460 which is the identity. 148 00:08:22,460 --> 00:08:26,500 So it's true, yeah. 149 00:08:26,500 --> 00:08:29,830 So that means that you're never under flow or overflow 150 00:08:29,830 --> 00:08:35,770 when you're multiplying by Q. Every numerical algorithm 151 00:08:35,770 --> 00:08:42,309 is written to use orthogonal matrices wherever it can. 152 00:08:42,309 --> 00:08:44,545 And here's the first example. 153 00:08:49,420 --> 00:08:53,260 I think it may be good for me to think of other examples 154 00:08:53,260 --> 00:08:55,720 or for us to think of other examples 155 00:08:55,720 --> 00:08:58,790 of orthogonal matrices. 156 00:08:58,790 --> 00:09:03,360 So I'm using that word orthogonal matrix. 157 00:09:03,360 --> 00:09:06,070 We should really be saying orthonormal. 158 00:09:06,070 --> 00:09:11,230 And I'm really thinking mostly of square ones. 159 00:09:11,230 --> 00:09:16,550 So in this square case when Q transpose is Q inverse. 160 00:09:19,320 --> 00:09:23,140 Of course, that fact makes it easy to solve 161 00:09:23,140 --> 00:09:27,610 all equations that have Q as a coefficient matrix, 162 00:09:27,610 --> 00:09:31,180 because you want the inverse and you just use the transpose. 163 00:09:34,600 --> 00:09:40,342 Let's just take some minutes to think of examples of Q's. 164 00:09:40,342 --> 00:09:41,800 If they're so important, there have 165 00:09:41,800 --> 00:09:44,560 to be interesting examples. 166 00:09:44,560 --> 00:09:46,450 And that was a first one. 167 00:09:46,450 --> 00:09:52,180 Now there's one more two by two example that you should know. 168 00:09:52,180 --> 00:09:54,890 Do you know what that would be? 169 00:09:54,890 --> 00:09:58,990 This will be an example two, and it's also going to be only two 170 00:09:58,990 --> 00:10:01,870 by two and real. 171 00:10:01,870 --> 00:10:07,310 And what possibility have I got left here? 172 00:10:07,310 --> 00:10:11,240 I'll use this same first column, cos theta sine theta, 173 00:10:11,240 --> 00:10:17,620 because that's more or less any unit vector in two dimensions, 174 00:10:17,620 --> 00:10:20,030 this has got that form. 175 00:10:20,030 --> 00:10:25,036 So what do you propose for the second column? 176 00:10:25,036 --> 00:10:25,994 Yes? 177 00:10:25,994 --> 00:10:29,347 AUDIENCE: [INAUDIBLE]. 178 00:10:29,347 --> 00:10:33,550 PROFESSOR: Yeah, put the minus sign down here. 179 00:10:33,550 --> 00:10:35,890 So you think does that make any difference? 180 00:10:35,890 --> 00:10:41,240 So sine theta and minus cos theta. 181 00:10:41,240 --> 00:10:45,100 I don't know if you've ever looked at that matrix. 182 00:10:45,100 --> 00:10:47,990 We're trying to collect together a few matrices that 183 00:10:47,990 --> 00:10:51,050 are worth knowing, are worth looking at. 184 00:10:51,050 --> 00:10:53,150 Now what's happened here? 185 00:10:53,150 --> 00:10:57,290 You may say that was a trivial change, which it kind of was. 186 00:10:57,290 --> 00:10:59,150 But it's a different matrix now. 187 00:10:59,150 --> 00:11:01,520 It's not a rotation anymore. 188 00:11:01,520 --> 00:11:03,530 That's not a rotation. 189 00:11:03,530 --> 00:11:08,630 And yeah, somehow now it's symmetric. 190 00:11:08,630 --> 00:11:15,300 And yeah, it's eigenvectors must be something or other. 191 00:11:15,300 --> 00:11:17,670 We'll get to those. 192 00:11:17,670 --> 00:11:20,970 But what does that matrix do? 193 00:11:20,970 --> 00:11:22,680 I don't know if you've seen it. 194 00:11:22,680 --> 00:11:29,190 If you haven't, it doesn't jump out, but it's a important case. 195 00:11:29,190 --> 00:11:30,960 This is a reflection matrix. 196 00:11:39,640 --> 00:11:42,290 Notice that it's determinant is minus one. 197 00:11:42,290 --> 00:11:45,500 You have minus cos square theta, minus sine squared theta. 198 00:11:45,500 --> 00:11:46,940 It's determinant is minus one. 199 00:11:46,940 --> 00:11:54,350 There's some eigenvalue coming up that's got a minus. 200 00:11:57,410 --> 00:11:59,960 So what do I mean by a reflection matrix? 201 00:11:59,960 --> 00:12:01,265 Let me draw the plane. 202 00:12:04,080 --> 00:12:07,460 So one, zero, let's follow that, follow again. 203 00:12:07,460 --> 00:12:09,470 One, zero, where does that go? 204 00:12:09,470 --> 00:12:10,950 That gives me the first column. 205 00:12:10,950 --> 00:12:15,800 So as before, it goes to cos theta sine theta. 206 00:12:20,060 --> 00:12:22,600 And when I say reflection, let me 207 00:12:22,600 --> 00:12:26,170 put the mirror into the picture so you 208 00:12:26,170 --> 00:12:28,000 see what reflection it is. 209 00:12:28,000 --> 00:12:34,330 The mirror is along here at angle theta over two, 210 00:12:34,330 --> 00:12:37,780 theta over two line. 211 00:12:37,780 --> 00:12:42,520 So sure enough, one, zero at angle zero 212 00:12:42,520 --> 00:12:48,960 got reflected into a unit vector at angle theta, 213 00:12:48,960 --> 00:12:53,730 and halfway between was theta over two line. 214 00:12:53,730 --> 00:12:55,300 That's OK. 215 00:12:55,300 --> 00:12:59,590 Now what about the other guy, zero, one? 216 00:12:59,590 --> 00:13:01,950 Here's zero, one. 217 00:13:01,950 --> 00:13:03,350 I multiply that. 218 00:13:03,350 --> 00:13:07,440 Can I put the zero, one up here, so your I 219 00:13:07,440 --> 00:13:09,480 does the multiplication? 220 00:13:09,480 --> 00:13:13,630 Where is the result of zero, one? 221 00:13:13,630 --> 00:13:19,400 What's the output from Q applied to zero, one? 222 00:13:19,400 --> 00:13:22,170 Sine theta minus cos theta, right? 223 00:13:22,170 --> 00:13:24,300 It's the second column. 224 00:13:24,300 --> 00:13:26,990 And so where is that? 225 00:13:26,990 --> 00:13:29,000 Well, it's perpendicular to that guy. 226 00:13:29,000 --> 00:13:31,100 That's what I know. 227 00:13:31,100 --> 00:13:34,970 That was the point, that the two columns are perpendicular. 228 00:13:34,970 --> 00:13:37,910 So it must go down this way, right? 229 00:13:37,910 --> 00:13:41,630 Sine theta cos theta. 230 00:13:41,630 --> 00:13:43,970 And it doesn't change the length. 231 00:13:43,970 --> 00:13:47,690 All these facts that we just learned are key. 232 00:13:47,690 --> 00:13:50,630 So there's zero, one. 233 00:13:50,630 --> 00:13:56,960 And it goes to this guy, which is whatever that second column 234 00:13:56,960 --> 00:14:00,190 is sine theta minus cos theta. 235 00:14:02,960 --> 00:14:07,730 And if you check that, actually, gosh! 236 00:14:07,730 --> 00:14:09,800 This is like plain geometry. 237 00:14:09,800 --> 00:14:14,150 I believe-- it never occurred to me before-- but I believe it, 238 00:14:14,150 --> 00:14:18,525 that this angle going down to there, 239 00:14:18,525 --> 00:14:22,850 that that goes straight through, and that the halfway one 240 00:14:22,850 --> 00:14:23,650 is that line. 241 00:14:27,200 --> 00:14:29,580 Yeah, I think that picture has got it. 242 00:14:29,580 --> 00:14:33,070 And I think it's in the note. 243 00:14:33,070 --> 00:14:35,390 So that's a reflection matrix. 244 00:14:35,390 --> 00:14:38,650 Well, that's a two by two reflection matrix. 245 00:14:38,650 --> 00:14:45,060 Would you like to see some other matrices like this one, 246 00:14:45,060 --> 00:14:47,590 but larger? 247 00:14:47,590 --> 00:14:53,940 They're named after a guy named Householder. 248 00:14:53,940 --> 00:14:57,660 So these are Householder reflections. 249 00:15:02,520 --> 00:15:04,210 What am I doing here? 250 00:15:04,210 --> 00:15:07,260 I'm collecting together some orthogonal matrices 251 00:15:07,260 --> 00:15:09,930 that are useful and important. 252 00:15:09,930 --> 00:15:15,530 And Householder found a whole bunch of them. 253 00:15:15,530 --> 00:15:18,990 And his algorithm is a much used part 254 00:15:18,990 --> 00:15:21,330 of numerical linear algebra. 255 00:15:21,330 --> 00:15:24,400 So he started with a unit vector. 256 00:15:24,400 --> 00:15:31,040 Start with a unit vector u transpose u equal one. 257 00:15:31,040 --> 00:15:33,940 So the length of the vector is one. 258 00:15:33,940 --> 00:15:34,890 And then he created-- 259 00:15:34,890 --> 00:15:36,970 let's name it after him-- 260 00:15:36,970 --> 00:15:37,470 H. 261 00:15:37,470 --> 00:15:41,865 He created this matrix, the identity minus two u, 262 00:15:41,865 --> 00:15:42,780 u transpose. 263 00:15:46,920 --> 00:15:50,325 And I believe that that's a really useful matrix. 264 00:15:53,910 --> 00:15:57,780 I think this review is like going beyond 18.06, 265 00:15:57,780 --> 00:16:04,230 into what ones are really worth knowing, 266 00:16:04,230 --> 00:16:06,960 worth knowing individually. 267 00:16:06,960 --> 00:16:10,470 Could we just check what are the properties of Householders 268 00:16:10,470 --> 00:16:13,200 reflection of that I minus two? 269 00:16:13,200 --> 00:16:17,800 You recognize here a column times a row. 270 00:16:17,800 --> 00:16:19,690 So that's a matrix. 271 00:16:19,690 --> 00:16:27,000 And what could you tell me about that matrix u, u transpose? 272 00:16:27,000 --> 00:16:29,810 It's yeah? 273 00:16:32,810 --> 00:16:39,080 What can we say about H? 274 00:16:39,080 --> 00:16:44,540 So I guess I'm believing that H is a orthogonal matrix, 275 00:16:44,540 --> 00:16:46,160 otherwise it wouldn't be here today. 276 00:16:46,160 --> 00:16:49,070 So I believe that-- and that not only is it orthogonal, 277 00:16:49,070 --> 00:16:51,580 it is also-- 278 00:16:51,580 --> 00:16:52,940 have a look at it-- 279 00:16:52,940 --> 00:16:54,130 symmetric. 280 00:16:54,130 --> 00:16:55,460 It's also symmetric. 281 00:16:55,460 --> 00:16:57,260 The identity is symmetric. 282 00:16:57,260 --> 00:16:59,290 u, u transpose is symmetric. 283 00:16:59,290 --> 00:17:03,380 So this is a family of symmetric orthogonal matrices. 284 00:17:03,380 --> 00:17:06,109 And that was one of them. 285 00:17:06,109 --> 00:17:08,450 That's a symmetric orthogonal matrix. 286 00:17:08,450 --> 00:17:13,130 These matrices are really great to have. 287 00:17:13,130 --> 00:17:17,660 In using linear algebra, you just 288 00:17:17,660 --> 00:17:22,550 get a collection of useful matrices that you can call on. 289 00:17:22,550 --> 00:17:26,780 And these are definitely one family. 290 00:17:26,780 --> 00:17:28,369 Well, it's obviously symmetric. 291 00:17:28,369 --> 00:17:31,040 Shall we check that it's orthogonal? 292 00:17:31,040 --> 00:17:33,110 So to check that it's orthogonal, 293 00:17:33,110 --> 00:17:40,540 so I'm going to check that H transpose H is the identity. 294 00:17:40,540 --> 00:17:43,380 Can I just check that? 295 00:17:43,380 --> 00:17:46,800 Well, H transpose is the same as H because it was symmetric. 296 00:17:46,800 --> 00:17:49,070 So I'm going to square this guy. 297 00:17:49,070 --> 00:17:51,650 This is really H times H. 298 00:17:51,650 --> 00:17:53,360 I'm squaring it. 299 00:17:53,360 --> 00:17:55,730 And what do I get if I square-- 300 00:17:55,730 --> 00:17:57,830 So I get-- 301 00:17:57,830 --> 00:18:01,740 I hope I get the identity, but let's see it. 302 00:18:01,740 --> 00:18:05,835 What do I get when I square this? 303 00:18:05,835 --> 00:18:09,020 I get little-- multiply it out. 304 00:18:09,020 --> 00:18:12,170 So I times I is I. 305 00:18:12,170 --> 00:18:15,590 And then I get some number of u, u transposes. 306 00:18:15,590 --> 00:18:19,162 How many do I get from that? 307 00:18:19,162 --> 00:18:22,400 So I'm squaring this thing because H transpose 308 00:18:22,400 --> 00:18:26,760 J is the same as H times H. So I'm squaring it. 309 00:18:26,760 --> 00:18:29,080 So what do I put here? 310 00:18:29,080 --> 00:18:30,740 Four, thanks. 311 00:18:30,740 --> 00:18:33,720 And now I've got this guy squared with a plus. 312 00:18:33,720 --> 00:18:38,360 So that's four, u, you, transpose u, u transpose. 313 00:18:43,690 --> 00:18:45,700 Yeah, I'm totally realizing I've practiced 314 00:18:45,700 --> 00:18:49,660 for a lifetime doing these dinky little calculations. 315 00:18:49,660 --> 00:18:51,970 But they are dinky. 316 00:18:51,970 --> 00:18:55,750 And you'll get the hang of it quickly. 317 00:18:55,750 --> 00:18:59,390 Now what am I hoping out of that bottom line? 318 00:18:59,390 --> 00:19:04,450 That it is I. We're hoping we're going to get I. Do we get I? 319 00:19:04,450 --> 00:19:05,770 Yes. 320 00:19:05,770 --> 00:19:09,760 Who sees how to get I out of that thing? 321 00:19:09,760 --> 00:19:15,190 Yeah, u transpose u in here is a number. 322 00:19:15,190 --> 00:19:18,730 That was u-- that was column times row times column times 323 00:19:18,730 --> 00:19:19,570 row. 324 00:19:19,570 --> 00:19:23,750 And I look at in the middle here is row times column. 325 00:19:23,750 --> 00:19:28,430 And that's the number one right because it's there. 326 00:19:28,430 --> 00:19:31,240 So that's one and then I have minus 4 327 00:19:31,240 --> 00:19:32,650 of that plus 4 that that. 328 00:19:32,650 --> 00:19:37,690 They cancel each other, and I get I. 329 00:19:37,690 --> 00:19:40,260 So those are good matrices. 330 00:19:43,130 --> 00:19:43,920 We'll use them. 331 00:19:46,610 --> 00:19:47,330 We'll use them. 332 00:19:47,330 --> 00:19:49,400 Actually, they're better than Gram-Schmidt. 333 00:19:49,400 --> 00:19:53,750 So we'll use them in making things orthogonal. 334 00:19:53,750 --> 00:19:57,750 So what other orthogonal matrices? 335 00:19:57,750 --> 00:19:59,430 Let's create some. 336 00:19:59,430 --> 00:20:05,410 Creating good orthogonal matrices is-- 337 00:20:05,410 --> 00:20:08,030 you know, it pays off. 338 00:20:08,030 --> 00:20:11,200 Let's think. 339 00:20:11,200 --> 00:20:22,790 So there a family named after this French guy 340 00:20:22,790 --> 00:20:23,900 who lived to 100. 341 00:20:23,900 --> 00:20:27,820 He was a real old timer. 342 00:20:27,820 --> 00:20:32,890 Well, MIT had a faculty member in math, when I came, 343 00:20:32,890 --> 00:20:36,100 Professor Struik, who lived to 106. 344 00:20:36,100 --> 00:20:40,420 And I heard him give a lecture at Brown University at age 100. 345 00:20:40,420 --> 00:20:42,100 And it was perfect. 346 00:20:42,100 --> 00:20:44,020 You could not have done it better. 347 00:20:44,020 --> 00:20:45,550 So he's my inspiration. 348 00:20:45,550 --> 00:20:46,540 I'm keep going. 349 00:20:46,540 --> 00:20:50,710 I only have I like, n more years to get there. 350 00:20:50,710 --> 00:20:53,440 And then it's-- well, it's too many anyway. 351 00:20:56,410 --> 00:20:58,240 So Hadamard, he created-- 352 00:20:58,240 --> 00:21:06,750 well, that's the simplest, the smallest. 353 00:21:06,750 --> 00:21:10,410 Now the next guy is going to be four by four. 354 00:21:10,410 --> 00:21:12,060 I'm going to put that-- 355 00:21:12,060 --> 00:21:16,940 so where I see a one, I'm going to put 356 00:21:16,940 --> 00:21:22,580 Hadamard one, one, one minus one, one, one, one minus one. 357 00:21:22,580 --> 00:21:25,610 And then when I say a minus, I'm going to put an n with a minus. 358 00:21:29,990 --> 00:21:32,300 You saw that picture? 359 00:21:32,300 --> 00:21:39,440 It was a picture of H2, H2, H2, and minus H2. 360 00:21:39,440 --> 00:21:40,640 That's what I've got there. 361 00:21:43,380 --> 00:21:46,303 And I believe those columns are orthogonal. 362 00:21:49,081 --> 00:21:51,860 Right? 363 00:21:51,860 --> 00:21:53,670 Now what could I do? 364 00:21:53,670 --> 00:21:56,090 Well it's not quite an orthogonal matrix. 365 00:21:56,090 --> 00:21:59,690 What do I have to do to make-- this isn't quite 366 00:21:59,690 --> 00:22:01,520 an orthogonal matrix either. 367 00:22:01,520 --> 00:22:05,630 What do I do to make that an orthogonal matrix? 368 00:22:05,630 --> 00:22:11,000 Divide by square root of two? 369 00:22:11,000 --> 00:22:15,120 I need unit vectors there. 370 00:22:15,120 --> 00:22:19,190 And here, those links are one squared, one squared, got four, 371 00:22:19,190 --> 00:22:20,710 square root of four is two. 372 00:22:20,710 --> 00:22:23,720 So I better divide by the two there. 373 00:22:23,720 --> 00:22:26,190 And now here I'm up to-- 374 00:22:26,190 --> 00:22:26,810 yeah. 375 00:22:26,810 --> 00:22:29,080 So that was that one. 376 00:22:29,080 --> 00:22:31,010 Tell me the next one up. 377 00:22:31,010 --> 00:22:32,140 What's that going to be? 378 00:22:32,140 --> 00:22:34,040 Eight by eight? 379 00:22:34,040 --> 00:22:34,640 What's that? 380 00:22:34,640 --> 00:22:36,290 So this is H4 here. 381 00:22:39,080 --> 00:22:41,720 Oops, four. 382 00:22:41,720 --> 00:22:45,330 So tell me, what I should do for eight by eight. 383 00:22:45,330 --> 00:22:47,240 You know, it's simple. 384 00:22:47,240 --> 00:22:51,560 But that's a good thing to say about it. 385 00:22:51,560 --> 00:22:54,380 You know, they're in coding theory, 386 00:22:54,380 --> 00:22:58,040 all sorts of places you want matrices of ones and minus 387 00:22:58,040 --> 00:22:59,450 ones. 388 00:22:59,450 --> 00:23:01,770 What's H8? 389 00:23:01,770 --> 00:23:04,850 I'm going to build it out of H4. 390 00:23:04,850 --> 00:23:07,810 So what's it going to be? 391 00:23:07,810 --> 00:23:09,620 I'm going to put an H4 there. 392 00:23:09,620 --> 00:23:12,230 What am I going to put here? 393 00:23:12,230 --> 00:23:14,190 Another H4. 394 00:23:14,190 --> 00:23:16,260 And up here? 395 00:23:16,260 --> 00:23:17,430 Another H4. 396 00:23:17,430 --> 00:23:19,530 And finally, here? 397 00:23:19,530 --> 00:23:20,370 Minus H4. 398 00:23:23,030 --> 00:23:26,190 And I think I've got orthogonal columns again. 399 00:23:29,310 --> 00:23:36,210 Because the columns within these dot products with themselves 400 00:23:36,210 --> 00:23:38,230 give zero and zero. 401 00:23:38,230 --> 00:23:42,030 The dot products from these columns and these columns 402 00:23:42,030 --> 00:23:43,980 obviously, have the minus. 403 00:23:43,980 --> 00:23:48,120 And the dot products in here are zero from that and zero 404 00:23:48,120 --> 00:23:48,690 from that. 405 00:23:48,690 --> 00:23:50,010 Yeah, it works. 406 00:23:52,750 --> 00:23:59,770 And we could keep going to 16 and 32 and 64. 407 00:23:59,770 --> 00:24:04,600 But then up comes a question. 408 00:24:04,600 --> 00:24:07,930 What about H12? 409 00:24:07,930 --> 00:24:14,920 Is there ones and minus ones matrix of size 12, 12 by 12? 410 00:24:19,020 --> 00:24:22,590 It doesn't come directly from our little pattern, which 411 00:24:22,590 --> 00:24:25,110 is doubling size every time. 412 00:24:25,110 --> 00:24:27,060 But you could still hope. 413 00:24:27,060 --> 00:24:28,890 And it works. 414 00:24:28,890 --> 00:24:30,600 I don't know what it is, but there's 415 00:24:30,600 --> 00:24:36,000 a there's a matrix of ones and minus ones 12 416 00:24:36,000 --> 00:24:37,890 orthogonal columns. 417 00:24:37,890 --> 00:24:38,925 So the answer is yes. 418 00:24:42,480 --> 00:24:47,550 So we make an 18.065 conjecture. 419 00:24:47,550 --> 00:24:52,090 Every matrix size-- well, not every matrix size, 420 00:24:52,090 --> 00:24:54,850 because three by three is not going to work. 421 00:24:54,850 --> 00:24:57,570 One by one is not going to work either. 422 00:24:57,570 --> 00:25:00,970 But H12 works, H8, H16-- 423 00:25:00,970 --> 00:25:02,860 What's our conjecture? 424 00:25:02,860 --> 00:25:05,770 We won't be the first to conjecture it, 425 00:25:05,770 --> 00:25:09,440 that there is a ones and minus ones 426 00:25:09,440 --> 00:25:13,290 orthogonal matrix with orthogonal columns 427 00:25:13,290 --> 00:25:15,570 of every size n. 428 00:25:18,490 --> 00:25:25,320 So I'll start the conjecture, always possible 429 00:25:25,320 --> 00:25:31,470 if n, which is the size of the matrix, let's say. 430 00:25:31,470 --> 00:25:34,840 What would you guess? 431 00:25:34,840 --> 00:25:36,040 Just take a shot. 432 00:25:36,040 --> 00:25:39,100 You won't be asked for a proof, because nobody has a proof. 433 00:25:42,050 --> 00:25:43,700 Even? 434 00:25:43,700 --> 00:25:46,310 Well, you could hope for even. 435 00:25:46,310 --> 00:25:49,860 You could look for try six, I guess, would be the first guy 436 00:25:49,860 --> 00:25:50,360 there. 437 00:25:50,360 --> 00:25:52,700 And I don't think it's possible. 438 00:25:52,700 --> 00:25:55,430 I think six is not possible. 439 00:25:55,430 --> 00:25:59,220 So every even size is, I think, not 440 00:25:59,220 --> 00:26:02,870 going to work, but a natural idea to try. 441 00:26:02,870 --> 00:26:05,200 What's the next thought? 442 00:26:05,200 --> 00:26:06,370 Every multiple of four. 443 00:26:06,370 --> 00:26:10,890 If n over a four is a whole number. 444 00:26:20,860 --> 00:26:25,340 But nobody has a systematic way to create these things. 445 00:26:25,340 --> 00:26:27,830 So like some of them, at this point, 446 00:26:27,830 --> 00:26:30,080 we're down to doing it one at a time. 447 00:26:30,080 --> 00:26:34,430 And we're up to 668. 448 00:26:34,430 --> 00:26:36,680 But we haven't got that one yet. 449 00:26:36,680 --> 00:26:37,520 Isn't that crazy? 450 00:26:37,520 --> 00:26:39,440 So all this is coming from Wikipedia. 451 00:26:42,020 --> 00:26:44,450 My source of all that's good in mathematics 452 00:26:44,450 --> 00:26:46,520 is there on Wikipedia. 453 00:26:46,520 --> 00:26:49,340 Anyway, this is the conjecture. 454 00:26:49,340 --> 00:26:51,800 Conjecture means you don't have any damn idea 455 00:26:51,800 --> 00:26:55,280 of whether it's true or not. 456 00:26:55,280 --> 00:26:57,710 Is that divisible by four? 457 00:26:57,710 --> 00:26:59,480 Yeah, I guess it would be. 458 00:26:59,480 --> 00:27:02,200 600 certainly is, and 68 certainly is. 459 00:27:02,200 --> 00:27:03,680 Yeah, OK. 460 00:27:03,680 --> 00:27:06,570 So I think that's the first one. 461 00:27:06,570 --> 00:27:10,910 If you find one of size 668, just skip the homework 462 00:27:10,910 --> 00:27:13,620 and tell us about that one. 463 00:27:13,620 --> 00:27:18,340 But I don't think I'll assign that. 464 00:27:18,340 --> 00:27:21,320 Yeah, I must have searched for it online. 465 00:27:21,320 --> 00:27:25,050 But yeah. 466 00:27:25,050 --> 00:27:29,865 Anyway, so those are the Hadamard matrices. 467 00:27:33,100 --> 00:27:37,430 Now where else do I remember orthogonal matrices coming 468 00:27:37,430 --> 00:27:37,930 from? 469 00:27:37,930 --> 00:27:44,760 Well, yeah really, the biggest source. 470 00:27:44,760 --> 00:27:47,400 So when I'm looking for orthogonal matrices, 471 00:27:47,400 --> 00:27:52,020 I'm looking for a basis of orthogonal vectors. 472 00:27:52,020 --> 00:27:56,320 And where in math am I going to find vectors 473 00:27:56,320 --> 00:27:59,700 that come out to be orthogonal? 474 00:27:59,700 --> 00:28:02,340 We haven't seen-- that's the next section of the notes, 475 00:28:02,340 --> 00:28:05,130 but maybe you are remembering. 476 00:28:05,130 --> 00:28:08,580 Where will we sort of like, automatically show up 477 00:28:08,580 --> 00:28:11,920 with orthogonal vectors? 478 00:28:11,920 --> 00:28:14,440 They could be the eye eigenvectors 479 00:28:14,440 --> 00:28:17,830 of symmetric matrix. 480 00:28:17,830 --> 00:28:22,270 And that's where the most important ones come from. 481 00:28:22,270 --> 00:28:24,970 Oh, I could tell you about wavelets though. 482 00:28:24,970 --> 00:28:26,910 Wavelets are more like this picture. 483 00:28:26,910 --> 00:28:30,220 They're ones and minus ones, or the simplest wavelets 484 00:28:30,220 --> 00:28:32,530 are ones and minus ones. 485 00:28:32,530 --> 00:28:36,540 Before I go on to the eigenvector business, 486 00:28:36,540 --> 00:28:40,450 can I mention the wavelets matrices? 487 00:28:40,450 --> 00:28:44,860 Yeah, these are really important simple and important 488 00:28:44,860 --> 00:28:46,540 constructions. 489 00:28:46,540 --> 00:28:55,570 So wavelets- let me draw a picture of-- 490 00:28:55,570 --> 00:28:57,250 I'm going to come up with four-- 491 00:28:57,250 --> 00:28:59,440 I'll do the four by four case. 492 00:28:59,440 --> 00:29:07,410 And these are the orthogonal guys. 493 00:29:07,410 --> 00:29:14,680 And then the next one is and down and zero. 494 00:29:14,680 --> 00:29:20,980 And the last one is zero and up and down. 495 00:29:20,980 --> 00:29:22,940 So that's four things. 496 00:29:22,940 --> 00:29:24,850 But let me show you the matrix. 497 00:29:24,850 --> 00:29:27,880 So I'll call it W for wavelets. 498 00:29:27,880 --> 00:29:31,810 So that guy, I'm thinking of was one, one, one, one. 499 00:29:31,810 --> 00:29:35,320 This guy I'm thinking of as one, one, minus one, minus one. 500 00:29:35,320 --> 00:29:38,770 It's looking sort of Hadamard's way. 501 00:29:38,770 --> 00:29:40,870 But there's a difference here. 502 00:29:40,870 --> 00:29:44,770 This guy is one minus one, zero, zero. 503 00:29:44,770 --> 00:29:48,630 So the wavelengths rescale. 504 00:29:48,630 --> 00:29:51,390 That's the difference between Hadamard and wavelets. 505 00:29:51,390 --> 00:29:56,880 Wavelets are self-scaling. 506 00:29:56,880 --> 00:29:59,010 And what's the last guy here? 507 00:29:59,010 --> 00:30:00,480 What's the fourth column? 508 00:30:00,480 --> 00:30:03,430 From that fourth wavelet? 509 00:30:03,430 --> 00:30:11,460 Zero, zero, one minus one, yeah. 510 00:30:14,460 --> 00:30:17,560 So Haar came up with that. 511 00:30:17,560 --> 00:30:23,610 This is the Haar wavelet, which was many years before the word 512 00:30:23,610 --> 00:30:26,970 wavelets was invented. 513 00:30:26,970 --> 00:30:32,690 He came up with this construction, the Haar matrix, 514 00:30:32,690 --> 00:30:34,460 the Haar functions. 515 00:30:34,460 --> 00:30:38,840 So they're very simple functions, but you know, 516 00:30:38,840 --> 00:30:42,680 that makes them usable that the fact that they're so simple. 517 00:30:42,680 --> 00:30:47,030 Now I don't know if you want to see the pattern in eight 518 00:30:47,030 --> 00:30:50,600 by eight but let me start the eight by eight so you'll 519 00:30:50,600 --> 00:30:53,240 know what wavelets are about. 520 00:30:53,240 --> 00:30:55,840 You'll know what these Haar wavelets are about, anyway. 521 00:30:55,840 --> 00:31:02,540 They're the ones that were kind of easy to visualize. 522 00:31:02,540 --> 00:31:08,330 So if that's W4, let's just take a minute. 523 00:31:08,330 --> 00:31:12,680 It won't take long for W8. 524 00:31:12,680 --> 00:31:14,815 So the first column is going to be eight, one. 525 00:31:21,150 --> 00:31:23,370 And what's the next column going to be? 526 00:31:26,590 --> 00:31:30,480 Four ones and four minus ones, like so. 527 00:31:30,480 --> 00:31:34,350 So four ones and four minus ones. 528 00:31:36,990 --> 00:31:46,420 And now the next column, two ones two minus ones and zeros. 529 00:31:46,420 --> 00:31:50,660 One, one, minus one, minus one and zero. 530 00:31:50,660 --> 00:31:58,310 And the fourth will be zeros and two ones and two minus ones. 531 00:31:58,310 --> 00:31:59,480 We got half a matrix now. 532 00:32:02,660 --> 00:32:07,290 Now if we just tell me the fifth, what do you think? 533 00:32:07,290 --> 00:32:10,410 What do I put in the fifth one? 534 00:32:10,410 --> 00:32:15,810 So again, it's going to squeeze down and rescale. 535 00:32:15,810 --> 00:32:18,740 And what's fifth column up here that's 536 00:32:18,740 --> 00:32:22,740 going to be ones and minus ones and zeros now? 537 00:32:22,740 --> 00:32:25,440 So it's not Hadamard, it's Haar. 538 00:32:25,440 --> 00:32:28,430 And what shall I put? 539 00:32:28,430 --> 00:32:29,630 One, one. 540 00:32:29,630 --> 00:32:31,757 Shall I start with one, one? 541 00:32:31,757 --> 00:32:32,590 AUDIENCE: 1 minus 1. 542 00:32:32,590 --> 00:32:35,510 PROFESSOR: Oh! 543 00:32:35,510 --> 00:32:36,330 And then all zeros? 544 00:32:39,860 --> 00:32:40,700 Oh yeah, thanks! 545 00:32:40,700 --> 00:32:41,870 Perfect! 546 00:32:41,870 --> 00:32:44,240 One minus and then all zeros. 547 00:32:44,240 --> 00:32:46,340 And then the next three columns, we'll 548 00:32:46,340 --> 00:32:50,330 have the one minus one here and the one minus one here, 549 00:32:50,330 --> 00:32:51,620 and the one minus one here. 550 00:32:51,620 --> 00:32:53,080 And otherwise all zeros. 551 00:32:53,080 --> 00:32:53,930 Yeah. 552 00:32:53,930 --> 00:32:55,130 So you see the pattern. 553 00:32:55,130 --> 00:32:57,200 It's scaling at every step. 554 00:33:01,010 --> 00:33:04,640 So that matrix has the advantage of being 555 00:33:04,640 --> 00:33:07,670 quite sparse, short of-- 556 00:33:07,670 --> 00:33:13,680 This, in my mind is a-- 557 00:33:13,680 --> 00:33:21,670 or four ones, that get involved with like, taking the average. 558 00:33:21,670 --> 00:33:26,740 Then this guy is like, taking the differences 559 00:33:26,740 --> 00:33:28,270 between those and those. 560 00:33:28,270 --> 00:33:32,920 And then this is like taking the difference at a smaller scale. 561 00:33:32,920 --> 00:33:35,240 And that also at a smaller scale. 562 00:33:35,240 --> 00:33:36,700 So that's what we keep doing. 563 00:33:36,700 --> 00:33:37,270 Yeah. 564 00:33:37,270 --> 00:33:37,600 Yeah. 565 00:33:37,600 --> 00:33:38,380 So that wavelets. 566 00:33:41,750 --> 00:33:48,876 It looks so simple right, but just 567 00:33:48,876 --> 00:33:51,030 a one minute history of wavelets, 568 00:33:51,030 --> 00:33:54,480 so Haar invented this in like, 1910. 569 00:33:54,480 --> 00:33:58,700 I mean, a long-- 570 00:33:58,700 --> 00:34:00,260 forever. 571 00:34:00,260 --> 00:34:03,560 But then you wanted wavelets said 572 00:34:03,560 --> 00:34:09,110 we're a little better, and not just 573 00:34:09,110 --> 00:34:11,510 ones and minus ones and zeros. 574 00:34:11,510 --> 00:34:16,670 And that took a lot of thinking. 575 00:34:16,670 --> 00:34:18,690 A lot of people were searching for it. 576 00:34:18,690 --> 00:34:23,449 And Ingrid Daubechies-- so I'll just put her name-- 577 00:34:23,449 --> 00:34:27,280 became famous for finding them. 578 00:34:27,280 --> 00:34:36,560 So in about 1988 she found a whole lot 579 00:34:36,560 --> 00:34:38,350 of families of wavelets. 580 00:34:38,350 --> 00:34:40,790 And when I say wavelets, she found 581 00:34:40,790 --> 00:34:43,159 a whole lot of orthogonal matrices 582 00:34:43,159 --> 00:34:45,630 that had good properties. 583 00:34:45,630 --> 00:34:46,130 Yeah. 584 00:34:46,130 --> 00:34:49,040 So that's the wavelet picture. 585 00:34:49,040 --> 00:34:50,300 OK. 586 00:34:50,300 --> 00:34:53,600 Now to close today and to connect 587 00:34:53,600 --> 00:34:58,310 with the next lecture on eigenvalues, eigenvectors, 588 00:34:58,310 --> 00:35:01,520 positive definite matrices-- we're really getting 589 00:35:01,520 --> 00:35:04,520 to the heart of things here. 590 00:35:04,520 --> 00:35:07,220 Let me follow through on that idea. 591 00:35:09,840 --> 00:35:17,950 So the eigenvectors of a symmetric matrix, 592 00:35:17,950 --> 00:35:25,743 but also of an orthogonal matrix are orthogonal. 593 00:35:33,150 --> 00:35:35,970 And that is really where, people-- 594 00:35:35,970 --> 00:35:37,680 where you can invent-- 595 00:35:37,680 --> 00:35:39,570 because you don't have to work hard. 596 00:35:39,570 --> 00:35:42,210 You just find a symmetric matrix. 597 00:35:42,210 --> 00:35:46,980 Its eigenvectors are automatically orthogonal. 598 00:35:46,980 --> 00:35:51,300 That doesn't mean they're great for use, but some of them 599 00:35:51,300 --> 00:35:55,650 are really important. 600 00:35:55,650 --> 00:35:57,750 And that maybe the most important of all 601 00:35:57,750 --> 00:35:59,670 is the Fourier. 602 00:35:59,670 --> 00:36:03,630 So you probably have seen Fourier series sines 603 00:36:03,630 --> 00:36:04,530 and cosines. 604 00:36:04,530 --> 00:36:07,830 Those guys are orthogonal functions. 605 00:36:07,830 --> 00:36:11,760 But the discrete Fourier series is what everybody computes. 606 00:36:11,760 --> 00:36:14,500 And those are orthogonal vectors. 607 00:36:14,500 --> 00:36:17,970 So the are orthogonal vectors that 608 00:36:17,970 --> 00:36:22,900 go into the discrete Fourier transform. 609 00:36:22,900 --> 00:36:24,990 And then they're done at high speed 610 00:36:24,990 --> 00:36:27,360 by the fast Fourier transform. 611 00:36:27,360 --> 00:36:32,520 Those are eigenvectors of Q, eigenvectors of the right Q. 612 00:36:32,520 --> 00:36:39,390 So let me just tell you in the right Q that get-- 613 00:36:39,390 --> 00:36:41,410 who's eigenvectors-- 614 00:36:41,410 --> 00:36:43,110 So here we go. 615 00:36:43,110 --> 00:36:50,610 Eigenvectors of Q-- you will just 616 00:36:50,610 --> 00:36:53,880 be amazed by how simple this matrix is. 617 00:36:53,880 --> 00:36:56,370 It's just that matrix. 618 00:36:56,370 --> 00:37:01,920 It's called a permutation matrix. 619 00:37:01,920 --> 00:37:04,782 It just puts the-- 620 00:37:04,782 --> 00:37:15,840 those are the four, eight, four Fourier discrete-- 621 00:37:15,840 --> 00:37:18,160 let me put that word discrete up here-- 622 00:37:18,160 --> 00:37:18,660 transform. 623 00:37:22,550 --> 00:37:26,260 So I really meant to put discrete Fourier transform. 624 00:37:26,260 --> 00:37:27,630 Yeah. 625 00:37:27,630 --> 00:37:31,650 The eigenvectors of that matrix, first of all, 626 00:37:31,650 --> 00:37:37,680 they are orthogonal, and then second, and more important, 627 00:37:37,680 --> 00:37:40,380 they're tremendously useful. 628 00:37:40,380 --> 00:37:42,630 They're the heart of signal processing. 629 00:37:45,808 --> 00:37:50,620 In signal processing, they just take a discrete Fourier 630 00:37:50,620 --> 00:37:53,270 transform of a vector before they even look at it. 631 00:37:53,270 --> 00:37:55,910 I mean, like that's the way to see it, 632 00:37:55,910 --> 00:37:58,990 is split it into its frequencies. 633 00:37:58,990 --> 00:38:01,930 And that's what the eigenvectors of this will do. 634 00:38:01,930 --> 00:38:07,570 So we're going to see the discrete Fourier transform. 635 00:38:07,570 --> 00:38:15,050 But my point here is to know that they're orthogonal just 636 00:38:15,050 --> 00:38:18,620 comes out of this fact that the eigenvectors are 637 00:38:18,620 --> 00:38:22,490 orthogonal for any Q. And that is certainly 638 00:38:22,490 --> 00:38:27,380 a Q. Everybody can see that those columns are orthogonal. 639 00:38:30,270 --> 00:38:33,840 That's a permutation matrix. 640 00:38:33,840 --> 00:38:36,030 You've taken the columns of the identity, which 641 00:38:36,030 --> 00:38:38,430 are totally orthogonal, and you just 642 00:38:38,430 --> 00:38:40,530 put them in a different order. 643 00:38:40,530 --> 00:38:43,230 So a permutation matrix is a reordering 644 00:38:43,230 --> 00:38:44,910 of the identity matrix. 645 00:38:44,910 --> 00:38:51,740 It's got to be a Q, and therefore, its eigenvectors 646 00:38:51,740 --> 00:38:53,210 are orthogonal. 647 00:38:53,210 --> 00:38:56,690 And they're just the winners. 648 00:38:56,690 --> 00:39:01,530 I mean, the matrix of the Fourier matrix 649 00:39:01,530 --> 00:39:06,930 with those four eigenvectors in it, I'll show you now. 650 00:39:06,930 --> 00:39:13,530 We're finishing today by leading into Wednesday, eigenvectors 651 00:39:13,530 --> 00:39:15,000 and eigenvalues. 652 00:39:15,000 --> 00:39:18,000 And we happen to be doing eigenvectors and eigenvalues 653 00:39:18,000 --> 00:39:24,600 of a Q, not today, of an S. Most of the time, 654 00:39:24,600 --> 00:39:27,980 it's a symmetric matrix whose eigenvectors we take, 655 00:39:27,980 --> 00:39:30,960 but here that happens to be a Q. Can I 656 00:39:30,960 --> 00:39:35,780 show you the eigenvectors, the four eigenvectors of that? 657 00:39:35,780 --> 00:39:36,810 Now, oh! 658 00:39:40,500 --> 00:39:44,470 The complex number I is going to come in. 659 00:39:44,470 --> 00:39:46,510 You have to let it in here. 660 00:39:46,510 --> 00:39:47,270 Yeah, sorry! 661 00:39:52,170 --> 00:39:57,370 If S is a real symmetric matrix, its eigenvectors are real. 662 00:39:57,370 --> 00:40:02,190 But if Q is a orthogonal matrix, its eigenvectors-- 663 00:40:02,190 --> 00:40:07,470 even though you couldn't ask for a more real matrix than that-- 664 00:40:07,470 --> 00:40:13,450 but its eigenvectors are at least, 665 00:40:13,450 --> 00:40:17,210 the good way are complex numbers. 666 00:40:17,210 --> 00:40:19,630 So can I just show you the eigenvectors of-- 667 00:40:27,150 --> 00:40:29,070 So again, overall the point today 668 00:40:29,070 --> 00:40:33,120 is to see orthogonal matrices. 669 00:40:33,120 --> 00:40:36,870 So I'll just repeat now while I can-- 670 00:40:36,870 --> 00:40:54,240 rotations here, reflections, wavelets, the Householder idea 671 00:40:54,240 --> 00:40:58,350 of reflections of big, large matrices 672 00:40:58,350 --> 00:41:02,880 that have this form I minus 2 u, u transpose are orthogonal. 673 00:41:02,880 --> 00:41:06,080 And now we're going to see the big guys, 674 00:41:06,080 --> 00:41:12,400 the eigenvectors of Q. Yes? 675 00:41:12,400 --> 00:41:14,540 AUDIENCE: [INAUDIBLE]? 676 00:41:14,540 --> 00:41:17,138 PROFESSOR: Ah yeah, we don't have orthonormal. 677 00:41:17,138 --> 00:41:17,680 That's right. 678 00:41:17,680 --> 00:41:18,940 We don't have orthonormal. 679 00:41:18,940 --> 00:41:21,820 I better divide by the square root of eight. 680 00:41:21,820 --> 00:41:24,100 AUDIENCE: [INAUDIBLE]. 681 00:41:24,100 --> 00:41:25,200 PROFESSOR: Oh yes! 682 00:41:25,200 --> 00:41:27,470 Oh, you're right! 683 00:41:27,470 --> 00:41:27,970 Sorry. 684 00:41:27,970 --> 00:41:31,390 I just thought I'd get away with that, but I didn't. 685 00:41:31,390 --> 00:41:32,620 Yeah. 686 00:41:32,620 --> 00:41:34,870 So these guys these words of eight. 687 00:41:34,870 --> 00:41:35,830 So are these. 688 00:41:35,830 --> 00:41:38,350 But these guys are square roots of four. 689 00:41:38,350 --> 00:41:39,970 These guys are square roots of two. 690 00:41:39,970 --> 00:41:40,750 Thank you. 691 00:41:40,750 --> 00:41:42,130 Absolutely right. 692 00:41:42,130 --> 00:41:43,270 Absolutely. 693 00:41:43,270 --> 00:41:45,040 Yeah. 694 00:41:45,040 --> 00:41:50,120 OK, so what are the eigenvectors of a permutation. 695 00:41:50,120 --> 00:41:54,830 This is going to be nice to see. 696 00:41:54,830 --> 00:41:58,010 And I'll use the matrix F for the eigenvector 697 00:41:58,010 --> 00:42:12,280 matrix of that Q up there, and F is for Fourier. 698 00:42:12,280 --> 00:42:14,840 And it'd be the four by four Fourier matrix. 699 00:42:14,840 --> 00:42:15,380 OK. 700 00:42:15,380 --> 00:42:19,240 What are the eigenvectors of Q? 701 00:42:19,240 --> 00:42:20,320 OK. 702 00:42:20,320 --> 00:42:23,850 So Q is a permutation. 703 00:42:23,850 --> 00:42:27,450 So like, I'm going to ask you for one eigenvector 704 00:42:27,450 --> 00:42:29,340 of every permutation matrix. 705 00:42:29,340 --> 00:42:34,170 What vector can you tell me that actually the eigenvalue 706 00:42:34,170 --> 00:42:35,610 will be one? 707 00:42:35,610 --> 00:42:39,180 What vector can you tell me where if I permute it, 708 00:42:39,180 --> 00:42:41,670 I don't change it? 709 00:42:41,670 --> 00:42:43,650 One, one, one, one. 710 00:42:43,650 --> 00:42:46,260 Like, it's everywhere here, one, one, one, one. 711 00:42:49,160 --> 00:42:52,220 So that's the zero frequency for a vector, 712 00:42:52,220 --> 00:42:55,220 the constant vector, all ones. 713 00:42:55,220 --> 00:42:59,610 Everybody sees if I'm multiply by Q, it doesn't change. 714 00:42:59,610 --> 00:43:00,110 OK. 715 00:43:00,110 --> 00:43:01,880 Now the next one-- 716 00:43:01,880 --> 00:43:04,430 I'll show you the four now. 717 00:43:04,430 --> 00:43:08,720 The next one will be 1 I, I squared and I cubed. 718 00:43:12,610 --> 00:43:15,070 Of course, I squared-- 719 00:43:15,070 --> 00:43:19,720 I don't know how many course 6J people are in this audience, 720 00:43:19,720 --> 00:43:21,600 but this is a math building. 721 00:43:21,600 --> 00:43:22,560 We paid for it. 722 00:43:22,560 --> 00:43:23,820 It's 2-190. 723 00:43:23,820 --> 00:43:25,380 And it's I in this room. 724 00:43:30,840 --> 00:43:36,280 Anyway, I is the first letter in what? 725 00:43:36,280 --> 00:43:36,970 Imaginary. 726 00:43:36,970 --> 00:43:39,270 Thank you, the first letter in imaginary. 727 00:43:39,270 --> 00:43:42,080 You can't say jimaginary. 728 00:43:42,080 --> 00:43:43,150 So that's it. 729 00:43:43,150 --> 00:43:44,080 OK. 730 00:43:44,080 --> 00:43:49,510 And then the next one is 1 I squared, I fourth, I sixth. 731 00:43:49,510 --> 00:43:55,060 And the next one is 1 I cubed, I6 and I9. 732 00:43:55,060 --> 00:43:57,340 Isn't that just beautiful? 733 00:43:57,340 --> 00:44:01,960 And you could show that every one of those four columns, 734 00:44:01,960 --> 00:44:06,970 if you multiply them by Q, you would get the eigenvalues. 735 00:44:06,970 --> 00:44:11,590 And you would see that it's an eigenvector. 736 00:44:11,590 --> 00:44:17,080 And this is just sort of like a discrete Fourier stuff, instead 737 00:44:17,080 --> 00:44:22,460 of e to the I, e to the Ix, e to the 2Ix, e to the 3Ix 738 00:44:22,460 --> 00:44:22,960 and so on. 739 00:44:22,960 --> 00:44:23,835 We just have vectors. 740 00:44:27,640 --> 00:44:32,680 So those are the four eigenvectors 741 00:44:32,680 --> 00:44:34,720 of that permutation. 742 00:44:34,720 --> 00:44:38,960 And those are orthogonal. 743 00:44:38,960 --> 00:44:41,720 Could I just check that? 744 00:44:41,720 --> 00:44:45,050 How do you know that this first column and the second-- well, 745 00:44:45,050 --> 00:44:48,950 I should really say zeros column and first column, 746 00:44:48,950 --> 00:44:53,620 if I'm talking frequencies. 747 00:44:53,620 --> 00:44:56,330 Do you see that that's orthogonal to that? 748 00:44:56,330 --> 00:44:59,480 Well, y is one plus I plus I squared 749 00:44:59,480 --> 00:45:01,850 plus I cubed equal zero. 750 00:45:01,850 --> 00:45:06,360 That's the dot product is this column one dot column two. 751 00:45:10,100 --> 00:45:12,440 Yeah, you're right. 752 00:45:12,440 --> 00:45:14,420 This happens to come out zero. 753 00:45:14,420 --> 00:45:17,205 Is that right? 754 00:45:17,205 --> 00:45:19,130 AUDIENCE: Yeah, that will come out zero. 755 00:45:19,130 --> 00:45:22,540 PROFESSOR: That will come out zero. 756 00:45:22,540 --> 00:45:27,040 But somebody mentioned that this isn't right. 757 00:45:27,040 --> 00:45:29,590 It's true that that came out zero, 758 00:45:29,590 --> 00:45:36,550 but when I have imaginary numbers anywhere around, 759 00:45:36,550 --> 00:45:40,540 this isn't a correct dot product to test orthogonal. 760 00:45:40,540 --> 00:45:42,680 If I have imaginary-- 761 00:45:42,680 --> 00:45:45,160 complex vectors-- if I have complex numbers, 762 00:45:45,160 --> 00:45:52,060 complex vectors, I should test column one conjugate dotted 763 00:45:52,060 --> 00:45:53,770 with column two-- 764 00:45:53,770 --> 00:46:00,730 column I conjugate-- well, let me take column see, 765 00:46:00,730 --> 00:46:02,230 which ones shall I take? 766 00:46:02,230 --> 00:46:04,660 Maybe that guy and that guy. 767 00:46:04,660 --> 00:46:09,030 Many of them, you luck out here. 768 00:46:09,030 --> 00:46:15,000 But really, I should be taking that conjugate. 769 00:46:15,000 --> 00:46:16,670 So these ones-- 770 00:46:16,670 --> 00:46:20,970 But the thing is, the complex conjugate of one is one. 771 00:46:20,970 --> 00:46:23,700 So that was OK. 772 00:46:23,700 --> 00:46:25,980 But in general, if I wanted to take 773 00:46:25,980 --> 00:46:33,570 column two dotted with column maybe four 774 00:46:33,570 --> 00:46:35,820 would be a little dodgy-- 775 00:46:35,820 --> 00:46:36,600 yeah. 776 00:46:36,600 --> 00:46:37,980 Look what happens. 777 00:46:37,980 --> 00:46:41,760 Take that column and that column. 778 00:46:41,760 --> 00:46:44,450 Take their dot product. 779 00:46:44,450 --> 00:46:47,370 Do it the wrong way. 780 00:46:47,370 --> 00:46:49,500 So what's the wrong way? 781 00:46:49,500 --> 00:46:52,110 Forget about the complex conjugate 782 00:46:52,110 --> 00:46:54,195 and just do it the usual way. 783 00:46:54,195 --> 00:46:55,950 So one times one is one. 784 00:46:55,950 --> 00:46:57,570 I times I cube is? 785 00:47:00,930 --> 00:47:02,430 One. 786 00:47:02,430 --> 00:47:05,110 I squared times I6 is? 787 00:47:05,110 --> 00:47:05,610 One. 788 00:47:05,610 --> 00:47:07,380 I'm getting all ones. 789 00:47:07,380 --> 00:47:10,260 I'm not getting orthogonality there. 790 00:47:10,260 --> 00:47:14,430 And that's because I forgot that I should take the complex 791 00:47:14,430 --> 00:47:17,010 conjugate-- well, of these guys-- one-- 792 00:47:17,010 --> 00:47:21,750 I should take minus I, minus I, minus I squared-- 793 00:47:21,750 --> 00:47:22,500 well, that's real. 794 00:47:22,500 --> 00:47:23,760 So it's OK. 795 00:47:23,760 --> 00:47:26,310 Minus there. 796 00:47:26,310 --> 00:47:32,316 So minus I squared is still minus one. 797 00:47:32,316 --> 00:47:36,250 So now if I do it, it comes out zero. 798 00:47:36,250 --> 00:47:39,330 So let me repeat again. 799 00:47:39,330 --> 00:47:41,730 Let me just make this statement. 800 00:47:41,730 --> 00:47:50,280 If Q transpose Q is I, and Qx is lambda x, 801 00:47:50,280 --> 00:47:57,230 and Qy is different eigenvalue y-- 802 00:47:57,230 --> 00:48:00,680 so I'm setting up the main fact. 803 00:48:00,680 --> 00:48:03,440 In the last minute, I'm just going to write down. 804 00:48:03,440 --> 00:48:05,930 So I have an orthogonal matrix. 805 00:48:05,930 --> 00:48:08,990 I have an eigenvector with eigenvalue lambda. 806 00:48:08,990 --> 00:48:12,740 I have another eigenvector with a different eigenvalue, mu. 807 00:48:12,740 --> 00:48:14,930 Then the claim is that-- 808 00:48:14,930 --> 00:48:18,830 what is the claim about eigenvectors? 809 00:48:18,830 --> 00:48:22,310 So here, this has an eigenvalue. 810 00:48:22,310 --> 00:48:24,700 This has a different eigenvalue. 811 00:48:24,700 --> 00:48:27,170 I need them to be different to really know 812 00:48:27,170 --> 00:48:31,520 that the x's and the y's can't be the same. 813 00:48:31,520 --> 00:48:32,990 So what is it that I want to show? 814 00:48:32,990 --> 00:48:34,598 AUDIENCE: [INAUDIBLE]. 815 00:48:34,598 --> 00:48:35,630 PROFESSOR: Yes! 816 00:48:35,630 --> 00:48:39,980 That x-- and I have to remember to do that. 817 00:48:39,980 --> 00:48:43,640 x transpose y is zero. 818 00:48:43,640 --> 00:48:46,340 That's orthogonality. 819 00:48:46,340 --> 00:48:49,460 That's orthogonality for a complex vectors. 820 00:48:49,460 --> 00:48:54,440 I have to remember to change are every I to a minus 821 00:48:54,440 --> 00:48:56,780 I in one of the vectors. 822 00:48:56,780 --> 00:49:00,770 And I can prove that fact by playing with these. 823 00:49:03,470 --> 00:49:07,560 By starting from here, I can get to that. 824 00:49:07,560 --> 00:49:08,060 OK. 825 00:49:08,060 --> 00:49:08,560 That's it. 826 00:49:08,560 --> 00:49:10,820 We've done a lot today, a lot of stuff 827 00:49:10,820 --> 00:49:13,340 about orthogonal matrices. 828 00:49:13,340 --> 00:49:18,560 Important ones, and those sources of important ones, 829 00:49:18,560 --> 00:49:19,400 eigenvectors. 830 00:49:19,400 --> 00:49:23,950 And so it will be eigenvectors on Wednesday.