1 00:00:01,550 --> 00:00:03,920 The following content is provided under a Creative 2 00:00:03,920 --> 00:00:05,310 Commons license. 3 00:00:05,310 --> 00:00:07,520 Your support will help MIT OpenCourseWare 4 00:00:07,520 --> 00:00:11,610 continue to offer high quality educational resources for free. 5 00:00:11,610 --> 00:00:14,180 To make a donation, or to view additional materials 6 00:00:14,180 --> 00:00:18,140 from hundreds of MIT courses, visit MIT OpenCourseWare 7 00:00:18,140 --> 00:00:19,026 at ocw.mit.edu. 8 00:00:22,780 --> 00:00:24,680 PROFESSOR: So, are we ready to go? 9 00:00:24,680 --> 00:00:28,810 Any questions on 18065? 10 00:00:28,810 --> 00:00:34,540 So it will be, as I said before, a mixture of linear algebra 11 00:00:34,540 --> 00:00:42,430 and math questions, along with online using the material. 12 00:00:42,430 --> 00:00:43,240 OK. 13 00:00:43,240 --> 00:00:47,890 So I'm, in this first week or two, 14 00:00:47,890 --> 00:00:51,880 reviewing the highlights of linear algebra. 15 00:00:51,880 --> 00:00:59,240 And I've reached this point, to remember-- 16 00:00:59,240 --> 00:01:00,560 well, so we-- 17 00:01:00,560 --> 00:01:07,430 I just said two words about multiplying matrices 18 00:01:07,430 --> 00:01:12,770 by using column times row as a way to do it. 19 00:01:12,770 --> 00:01:16,040 And now, I want to illustrate that 20 00:01:16,040 --> 00:01:23,480 by the five key factorizations of matrices. 21 00:01:23,480 --> 00:01:24,320 OK. 22 00:01:24,320 --> 00:01:25,620 So what are they? 23 00:01:25,620 --> 00:01:28,070 And do you recognize them? 24 00:01:28,070 --> 00:01:29,780 Everybody uses those letters. 25 00:01:29,780 --> 00:01:33,710 In fact, some of those letters, like LU or QR, 26 00:01:33,710 --> 00:01:38,450 would be the most used MATLAB commands in linear algebra. 27 00:01:38,450 --> 00:01:42,020 So a A equal LU of, maybe-- 28 00:01:42,020 --> 00:01:44,360 say something I'll develop today-- 29 00:01:44,360 --> 00:01:46,310 but it's about elimination. 30 00:01:54,590 --> 00:01:57,300 Solving linear systems. 31 00:01:57,300 --> 00:02:00,080 So that's always the start of a linear algebra course. 32 00:02:00,080 --> 00:02:03,260 But it will go fast here. 33 00:02:03,260 --> 00:02:05,510 I just want to show you a different way 34 00:02:05,510 --> 00:02:09,289 they get to L times U-- lower triangular 35 00:02:09,289 --> 00:02:11,450 times upper triangular. 36 00:02:11,450 --> 00:02:13,880 Probably you've seen that-- 37 00:02:13,880 --> 00:02:16,610 those triangular matrices. 38 00:02:16,610 --> 00:02:18,330 So do you know what QR is? 39 00:02:18,330 --> 00:02:21,482 What's QR about? 40 00:02:21,482 --> 00:02:23,300 AUDIENCE: Least squares? 41 00:02:23,300 --> 00:02:26,462 PROFESSOR: Least squares is the big application, 42 00:02:26,462 --> 00:02:28,110 the factorization. 43 00:02:28,110 --> 00:02:31,335 So what kind of a matrix gets that letter Q? 44 00:02:31,335 --> 00:02:32,210 AUDIENCE: Orthogonal? 45 00:02:32,210 --> 00:02:33,127 PROFESSOR: Orthogonal. 46 00:02:33,127 --> 00:02:35,190 The columns are orthogonal. 47 00:02:35,190 --> 00:02:37,220 Often orthonormal. 48 00:02:37,220 --> 00:02:40,320 So orthogonal means they're perpendicular to each other. 49 00:02:40,320 --> 00:02:44,090 And orthonormal means they're unit vectors. 50 00:02:44,090 --> 00:02:52,710 So that is-- so Q often represents a matrix 51 00:02:52,710 --> 00:02:55,830 with orthonormal columns. 52 00:02:55,830 --> 00:02:58,950 So you-- we could say Gram-Schmidt, 53 00:02:58,950 --> 00:03:06,300 if you want to remember a couple of old timers 54 00:03:06,300 --> 00:03:13,440 whose algorithm produces Q and R. 55 00:03:13,440 --> 00:03:14,640 How about this one? 56 00:03:14,640 --> 00:03:18,360 This is really a central one in math-- 57 00:03:18,360 --> 00:03:21,750 pure math, applied math, everywhere-- applications. 58 00:03:21,750 --> 00:03:24,030 So S stands for symmetric. 59 00:03:24,030 --> 00:03:26,970 So this is a special factorization 60 00:03:26,970 --> 00:03:29,340 for symmetric matrices. 61 00:03:29,340 --> 00:03:31,020 And you can see that it's symmetric. 62 00:03:31,020 --> 00:03:35,460 This lambda is the diagonal eigenvalue matrix-- 63 00:03:35,460 --> 00:03:38,850 always lambda for eigenvalues. 64 00:03:38,850 --> 00:03:41,010 Q is like that Q-- 65 00:03:41,010 --> 00:03:43,220 different Q, of course. 66 00:03:43,220 --> 00:03:46,920 That Q you can find just straight forward 67 00:03:46,920 --> 00:03:48,690 from Gram-Schmidt. 68 00:03:48,690 --> 00:03:52,110 This Q has the eigenvectors. 69 00:03:52,110 --> 00:03:55,630 So you don't find eigenvectors without some extra work. 70 00:03:55,630 --> 00:03:56,130 OK. 71 00:03:56,130 --> 00:03:58,270 So that's eigenvectors. 72 00:03:58,270 --> 00:03:58,770 Yeah. 73 00:03:58,770 --> 00:04:04,380 So that would be worth expanding. 74 00:04:04,380 --> 00:04:08,670 So here are the eigenvectors of the matrix S-- 75 00:04:08,670 --> 00:04:11,340 n of them, normalized. 76 00:04:11,340 --> 00:04:15,540 Here are the eigenvalues lambda 1 to lambda n. 77 00:04:15,540 --> 00:04:19,170 And here are the eigenvectors now transposed. 78 00:04:25,600 --> 00:04:29,170 So remind me of the great fact about-- 79 00:04:29,170 --> 00:04:32,560 two facts, I guess-- one fact about the eigenvalues 80 00:04:32,560 --> 00:04:34,570 and one fact about eigenvectors. 81 00:04:34,570 --> 00:04:41,650 This is an important fact statement in linear algebra. 82 00:04:41,650 --> 00:04:44,120 What do we know about the eigenvectors? 83 00:04:44,120 --> 00:04:46,780 Oh well, I guess I've given it away. 84 00:04:46,780 --> 00:04:50,310 The eigenvectors are orthogonal. 85 00:04:50,310 --> 00:04:52,230 That's very important. 86 00:04:52,230 --> 00:04:55,310 Makes some matrices-- well, they're beautiful matrices. 87 00:04:55,310 --> 00:04:59,110 They're the kings of linear algebra. 88 00:04:59,110 --> 00:05:01,320 Qs are the queens, in my opinion. 89 00:05:01,320 --> 00:05:03,210 Orthogonal matrices are the queens, 90 00:05:03,210 --> 00:05:06,270 and symmetric matrices are the kings. 91 00:05:06,270 --> 00:05:12,630 So these are orthonormal eigenvectors. 92 00:05:12,630 --> 00:05:14,400 And the key point-- 93 00:05:14,400 --> 00:05:16,860 an important point that's implicit 94 00:05:16,860 --> 00:05:19,140 here-- is, there are n of them. 95 00:05:19,140 --> 00:05:20,430 There is a complete set. 96 00:05:20,430 --> 00:05:23,280 The matrix can be diagonalized. 97 00:05:23,280 --> 00:05:28,050 And those-- well, what's special about the eigenvalues? 98 00:05:28,050 --> 00:05:32,250 Other matrices could be Q lambda Q transpose. 99 00:05:32,250 --> 00:05:35,730 But symmetric matrices are something additional 100 00:05:35,730 --> 00:05:36,375 about lambda. 101 00:05:36,375 --> 00:05:37,500 AUDIENCE: They're all real? 102 00:05:37,500 --> 00:05:39,330 PROFESSOR: They're all real. 103 00:05:39,330 --> 00:05:41,530 So eigenvalues are real. 104 00:05:41,530 --> 00:05:45,330 And eigenvectors are orthonormal-- 105 00:05:45,330 --> 00:05:48,150 can be chosen orthonormal-- can be chosen, 106 00:05:48,150 --> 00:05:49,380 I guess I have to say. 107 00:05:49,380 --> 00:05:50,040 OK. 108 00:05:50,040 --> 00:05:50,610 Good. 109 00:05:50,610 --> 00:05:51,110 Good. 110 00:05:51,110 --> 00:05:51,630 Good. 111 00:05:51,630 --> 00:05:54,960 Oh, now maybe I'll use that as an example 112 00:05:54,960 --> 00:05:57,340 of matrix multiplication. 113 00:05:57,340 --> 00:05:59,160 So let me just do that here. 114 00:05:59,160 --> 00:06:03,350 Simple matrix multiplication, but it makes the point. 115 00:06:03,350 --> 00:06:08,100 So Q lambda Q transpose. 116 00:06:10,870 --> 00:06:11,950 OK. 117 00:06:11,950 --> 00:06:15,730 Well, what was my point about matrix multiplication? 118 00:06:15,730 --> 00:06:20,140 Let me-- it really involved two matrices. 119 00:06:20,140 --> 00:06:21,800 Here, I unfortunately have three. 120 00:06:21,800 --> 00:06:25,240 So I'm going to have to squeeze lambda in with one of the Qs, 121 00:06:25,240 --> 00:06:28,840 to see it nicely as two matrices. 122 00:06:28,840 --> 00:06:30,560 Shall I just do that? 123 00:06:30,560 --> 00:06:32,050 Yeah. 124 00:06:32,050 --> 00:06:33,880 Now I've made it two matrices. 125 00:06:33,880 --> 00:06:35,350 That was easy. 126 00:06:35,350 --> 00:06:36,160 OK. 127 00:06:36,160 --> 00:06:38,590 Now what's the rule? 128 00:06:38,590 --> 00:06:43,000 In the first notes, this was A and this was B. 129 00:06:43,000 --> 00:06:45,100 And when you multiply two matrices, 130 00:06:45,100 --> 00:06:54,430 the rule is, this is columns of Q lambda times 131 00:06:54,430 --> 00:06:59,610 rows of Q transpose. 132 00:06:59,610 --> 00:07:02,060 I'm multiplying columns by rows. 133 00:07:02,060 --> 00:07:07,330 And so it's a column vector times a row vector, 134 00:07:07,330 --> 00:07:09,820 and that gives us a matrix. 135 00:07:09,820 --> 00:07:13,450 So each-- and it's a special matrix. 136 00:07:13,450 --> 00:07:14,750 So this is our column. 137 00:07:14,750 --> 00:07:16,330 This is a row. 138 00:07:16,330 --> 00:07:22,510 And when I multiply n by 1 times 1 by n, I get an n by n matrix. 139 00:07:22,510 --> 00:07:24,340 And it's pretty special. 140 00:07:24,340 --> 00:07:27,250 And what is the special fact about I'm 141 00:07:27,250 --> 00:07:29,530 sort of recalling from last time. 142 00:07:29,530 --> 00:07:32,890 What's special about a column times a row? 143 00:07:32,890 --> 00:07:34,630 It's rank is special. 144 00:07:34,630 --> 00:07:36,440 It's rank is 1. 145 00:07:36,440 --> 00:07:42,910 It's column space-- well, the only column around is this one. 146 00:07:42,910 --> 00:07:46,630 So all columns are multiples of this guy. 147 00:07:46,630 --> 00:07:50,410 All rows are multiples of this guy, 148 00:07:50,410 --> 00:07:52,450 as we could see from an example. 149 00:07:52,450 --> 00:07:54,160 Shall I just do an example? 150 00:07:54,160 --> 00:08:00,490 1, 2 times 3, 4, to take a random example. 151 00:08:00,490 --> 00:08:05,020 So that would give us 3, 4, 6, 8. 152 00:08:05,020 --> 00:08:10,630 And sure enough, the columns are multiples of 1, 2. 153 00:08:10,630 --> 00:08:13,270 The rows are multiples of 3, 4. 154 00:08:13,270 --> 00:08:16,610 And the rank is 1. 155 00:08:16,610 --> 00:08:17,110 OK. 156 00:08:17,110 --> 00:08:19,360 So those are the building blocks. 157 00:08:19,360 --> 00:08:21,850 Now I want to build something. 158 00:08:21,850 --> 00:08:22,720 So here we go. 159 00:08:25,280 --> 00:08:27,310 So this is a sum of rank 1. 160 00:08:27,310 --> 00:08:33,100 Sum of rank 1. 161 00:08:33,100 --> 00:08:36,970 Sum of column times row. 162 00:08:36,970 --> 00:08:40,210 So I take column 1 times row 1. 163 00:08:40,210 --> 00:08:42,460 That's my first thing in the sum. 164 00:08:42,460 --> 00:08:43,840 So column 1 of that. 165 00:08:43,840 --> 00:08:46,000 So you see I had to sneak the lambda 166 00:08:46,000 --> 00:08:50,660 to have just two factors. 167 00:08:50,660 --> 00:08:52,840 So what's column 1 of Q lambda? 168 00:08:52,840 --> 00:08:54,830 That's a good question. 169 00:08:54,830 --> 00:08:59,080 So column 1 of Q is Q1-- 170 00:08:59,080 --> 00:09:01,120 the first eigenvector. 171 00:09:01,120 --> 00:09:05,400 But now, I am multiplying by this diagonal matrix. 172 00:09:05,400 --> 00:09:09,360 Do you see in your mind what's the first column of Q lambda? 173 00:09:12,370 --> 00:09:15,290 Just think about that a second. 174 00:09:15,290 --> 00:09:23,200 So here's-- we can steal any little corner for a matrix. 175 00:09:23,200 --> 00:09:27,910 So here's q1, and the rest of the columns. 176 00:09:27,910 --> 00:09:31,210 And then here is lambda 1. 177 00:09:31,210 --> 00:09:32,440 And that's Q lambda. 178 00:09:32,440 --> 00:09:34,600 So I'm putting those together. 179 00:09:34,600 --> 00:09:38,480 And I'm asking, what's the first column of the answer? 180 00:09:38,480 --> 00:09:41,510 Can you see how that works? 181 00:09:41,510 --> 00:09:42,880 AUDIENCE: q1 lambda 1? 182 00:09:42,880 --> 00:09:43,983 PROFESSOR: It's... sorry?. 183 00:09:43,983 --> 00:09:44,900 AUDIENCE: q1 lambda 1? 184 00:09:44,900 --> 00:09:45,940 PROFESSOR: q1 lambda 1. 185 00:09:45,940 --> 00:09:46,900 Exactly. 186 00:09:46,900 --> 00:09:49,660 That lambda 1 will multiply q1. 187 00:09:49,660 --> 00:09:52,250 These other lambdas multiplied later columns. 188 00:09:52,250 --> 00:09:54,980 So the first column is lambda 1 times q1. 189 00:09:54,980 --> 00:09:55,480 Right. 190 00:09:55,480 --> 00:10:00,880 So it's the first guy, lambda 1, q1. 191 00:10:00,880 --> 00:10:06,820 And then the first row of this will be q1 transpose. 192 00:10:06,820 --> 00:10:11,770 That's the first guy in our sum of n things. 193 00:10:11,770 --> 00:10:14,710 And let me put the next one and the last one. 194 00:10:14,710 --> 00:10:20,200 Lambda 2, q2, q2 transpose. 195 00:10:20,200 --> 00:10:25,960 And lambda n, qn, qn transpose. 196 00:10:29,820 --> 00:10:33,320 That's really a nice right way to write 197 00:10:33,320 --> 00:10:39,300 to breakup the product, q lambda q transpose. 198 00:10:39,300 --> 00:10:41,650 This is called the spectral theorem. 199 00:10:41,650 --> 00:10:47,700 So that's the symmetric-- that's S. That's S there, is that. 200 00:10:47,700 --> 00:10:53,370 So that's-- we've broken up S into rank 1 pieces. 201 00:10:53,370 --> 00:10:57,750 That's like a constant theme. 202 00:10:57,750 --> 00:11:01,290 And these rank 1 pieces are quite special because they're 203 00:11:01,290 --> 00:11:02,700 symmetric. 204 00:11:02,700 --> 00:11:05,790 Q1q1 transpose will be symmetric. 205 00:11:05,790 --> 00:11:09,780 And oh, so can we-- 206 00:11:09,780 --> 00:11:14,280 so let's just-- I follow the rule for multiplying matrices. 207 00:11:14,280 --> 00:11:19,350 But maybe I could just check that it's the right thing-- 208 00:11:19,350 --> 00:11:22,350 that it came out right. 209 00:11:22,350 --> 00:11:23,940 So what do I mean by checking? 210 00:11:23,940 --> 00:11:28,350 I guess I'll just check about S times q1. 211 00:11:28,350 --> 00:11:33,810 So look at S-- 212 00:11:33,810 --> 00:11:38,760 this thing-- times the first eigenvector, and what do I get? 213 00:11:38,760 --> 00:11:39,990 OK. 214 00:11:39,990 --> 00:11:43,170 So you'll like this. 215 00:11:43,170 --> 00:11:47,400 I've split up S into a sum of rank 1 pieces. 216 00:11:47,400 --> 00:11:48,870 And that splitting is-- 217 00:11:48,870 --> 00:11:52,620 you see it all over. 218 00:11:52,620 --> 00:11:56,370 It's really showing you what the pieces of the symmetric matrix 219 00:11:56,370 --> 00:11:57,150 are. 220 00:11:57,150 --> 00:11:59,010 And now I'm just going to check that that's 221 00:11:59,010 --> 00:12:04,530 a correct formula for S, so I'll multiply it by q1. 222 00:12:04,530 --> 00:12:06,780 And I'm hoping to get the right thing. 223 00:12:06,780 --> 00:12:08,790 And what do I actually get? 224 00:12:08,790 --> 00:12:14,160 If I multiply this whole business times q1, 225 00:12:14,160 --> 00:12:20,310 I get lambda 1, q1, q1 transpose-- 226 00:12:20,310 --> 00:12:23,400 that's the first guy times my q1-- 227 00:12:23,400 --> 00:12:26,980 plus-- right? 228 00:12:26,980 --> 00:12:31,260 I'm multiplying S by q1, and this first term gave me that. 229 00:12:31,260 --> 00:12:33,010 And what does the next term give me? 230 00:12:33,010 --> 00:12:37,010 Put me out of my misery here. 231 00:12:37,010 --> 00:12:40,310 I'm looking for this thing to simplify like mad. 232 00:12:40,310 --> 00:12:40,830 OK. 233 00:12:40,830 --> 00:12:42,150 So what's the second term? 234 00:12:42,150 --> 00:12:47,833 When I multiply this guy by q1, what do I get? 235 00:12:47,833 --> 00:12:48,760 AUDIENCE: Zero. 236 00:12:48,760 --> 00:12:49,470 PROFESSOR: Zero. 237 00:12:49,470 --> 00:12:50,580 That's right. 238 00:12:50,580 --> 00:12:52,000 That's what we want. 239 00:12:52,000 --> 00:12:56,640 And when I multiply the last guy by q1, 240 00:12:56,640 --> 00:13:01,090 I get zero, because the qs are orthogonal. 241 00:13:01,090 --> 00:13:03,060 So this is all I get. 242 00:13:03,060 --> 00:13:05,790 And then-- so, I don't need this plus anymore. 243 00:13:05,790 --> 00:13:07,020 That's it. 244 00:13:07,020 --> 00:13:10,290 And then what can I do to improve 245 00:13:10,290 --> 00:13:18,060 that little somewhat repetitive formula for the answer? 246 00:13:18,060 --> 00:13:19,920 What do I want to do finally? 247 00:13:19,920 --> 00:13:23,430 I want to remember that the qs are normalized. 248 00:13:23,430 --> 00:13:25,090 They're unit vectors. 249 00:13:25,090 --> 00:13:26,652 So what does that tell me here? 250 00:13:26,652 --> 00:13:27,610 AUDIENCE: Q1 transpose. 251 00:13:27,610 --> 00:13:30,180 PROFESSOR: Q1 transpose times q1. 252 00:13:30,180 --> 00:13:34,170 This is just 1. 253 00:13:34,170 --> 00:13:38,210 It's-- that's what normalized means-- 254 00:13:38,210 --> 00:13:41,630 that the length squared is the length-- 255 00:13:41,630 --> 00:13:44,880 the length of the vector squared. 256 00:13:44,880 --> 00:13:45,900 And it's 1. 257 00:13:45,900 --> 00:13:48,750 So I can cancel that term. 258 00:13:48,750 --> 00:13:50,370 And I'm getting the right answer. 259 00:13:50,370 --> 00:13:53,850 That's all-- that's all this was about. 260 00:13:53,850 --> 00:13:59,600 I was just checking and wanted to see how it would fall out. 261 00:13:59,600 --> 00:14:05,100 And it falls right out that this formula 262 00:14:05,100 --> 00:14:08,250 is the correct matrix S, because it's 263 00:14:08,250 --> 00:14:11,210 got the right eigenvectors, qs. 264 00:14:11,210 --> 00:14:13,460 And it's got the right eigenvalues lambdas. 265 00:14:13,460 --> 00:14:18,390 So it's gotta be the right matrix S. Is that OK? 266 00:14:18,390 --> 00:14:24,830 That's like a first example to see how this-- 267 00:14:28,490 --> 00:14:31,680 splitting into rank 1s-- 268 00:14:31,680 --> 00:14:35,400 gives you back what you expect easily enough. 269 00:14:35,400 --> 00:14:38,200 It gives you the information you expect. 270 00:14:38,200 --> 00:14:38,700 OK. 271 00:14:38,700 --> 00:14:43,940 So that's the symmetric eigenvalue picture 272 00:14:43,940 --> 00:14:45,470 for symmetric matrices. 273 00:14:45,470 --> 00:14:47,360 And we'll see it again. 274 00:14:47,360 --> 00:14:54,410 It's-- well, all five of these are big, are important. 275 00:14:54,410 --> 00:14:56,660 I don't know if you know this one, 276 00:14:56,660 --> 00:15:02,090 but it's going to be a foundational factorization 277 00:15:02,090 --> 00:15:05,220 for this course and for all of data science. 278 00:15:05,220 --> 00:15:07,130 Do you know its name? 279 00:15:07,130 --> 00:15:09,920 So, what does it mean, first of all? 280 00:15:09,920 --> 00:15:12,950 Just a comment on this, and then we'll 281 00:15:12,950 --> 00:15:15,450 save it for a couple of weeks. 282 00:15:15,450 --> 00:15:19,130 So this view is actually an orthogonal matrix. 283 00:15:19,130 --> 00:15:23,870 And so is V. So it has two orthogonal matrices. 284 00:15:23,870 --> 00:15:27,230 So that's why people call them U and V 285 00:15:27,230 --> 00:15:33,470 rather than Q1 and Q2, which was too much to get subscript. 286 00:15:33,470 --> 00:15:37,210 So orthogonal times diagonal times orthogonal. 287 00:15:37,210 --> 00:15:45,880 And we'd say, orthogonal, diagonal, orthogonal. 288 00:15:45,880 --> 00:15:51,860 And 18.06 would, as of now, reach this topic 289 00:15:51,860 --> 00:15:54,710 because it's jumped up in importance. 290 00:15:54,710 --> 00:15:55,640 And it's called? 291 00:15:55,640 --> 00:15:57,265 AUDIENCE: Singular value decomposition. 292 00:15:57,265 --> 00:16:00,210 PROFESSOR: Singular value decomposition. 293 00:16:00,210 --> 00:16:04,910 Well, those are long words, so everybody calls it the SVD-- 294 00:16:04,910 --> 00:16:07,470 the singular value decomposition. 295 00:16:07,470 --> 00:16:15,210 The point is it works for every matrix-- 296 00:16:15,210 --> 00:16:16,650 rectangular matrices. 297 00:16:16,650 --> 00:16:18,690 There's no issue of, does it have 298 00:16:18,690 --> 00:16:21,630 enough eigenvectors or not? 299 00:16:21,630 --> 00:16:23,730 That's an issue here. 300 00:16:23,730 --> 00:16:25,320 Well, it's an issue here. 301 00:16:25,320 --> 00:16:29,910 Not every matrix has got enough eigenvectors to make that work. 302 00:16:29,910 --> 00:16:35,820 Every matrix that one works because instead 303 00:16:35,820 --> 00:16:39,780 of one set of vectors it's got two 304 00:16:39,780 --> 00:16:45,330 matrices-- two different sets of singular vectors. 305 00:16:45,330 --> 00:16:47,470 Oh, we'll see that. 306 00:16:47,470 --> 00:16:50,900 That's important. 307 00:16:50,900 --> 00:16:51,420 OK. 308 00:16:51,420 --> 00:16:55,490 So that's really a quick overview 309 00:16:55,490 --> 00:16:59,210 of fundamental factorizations. 310 00:16:59,210 --> 00:17:03,970 And I'd like to say just another word about elimination, 311 00:17:03,970 --> 00:17:09,540 A equal LU, and then we'll leave it alone. 312 00:17:09,540 --> 00:17:11,609 So elimination. 313 00:17:11,609 --> 00:17:12,109 Yeah. 314 00:17:12,109 --> 00:17:20,079 Do you remember that first beginning of linear algebra, 315 00:17:20,079 --> 00:17:22,290 when you're solving ax equal b? 316 00:17:22,290 --> 00:17:24,460 You do these row operations. 317 00:17:24,460 --> 00:17:28,990 Can I just-- what I want to say is, all those row operations 318 00:17:28,990 --> 00:17:35,330 that you do are perfectly expressed by L times U. 319 00:17:35,330 --> 00:17:40,000 And so that's a key point in 18.06, 320 00:17:40,000 --> 00:17:42,670 but I have a different way to look at it. 321 00:17:42,670 --> 00:17:44,170 So that's what I wanted to show you. 322 00:17:44,170 --> 00:17:45,050 I have a-- 323 00:17:45,050 --> 00:17:51,430 I want to show you a sum of rank 1's, or row times column. 324 00:17:51,430 --> 00:17:53,020 It fits in today. 325 00:17:53,020 --> 00:18:02,230 So I'd just like to see, why does a matrix invertible-- 326 00:18:02,230 --> 00:18:05,780 this is a square matrix, now invertible. 327 00:18:05,780 --> 00:18:11,260 And it factors-- if all goes well with elimination 328 00:18:11,260 --> 00:18:13,270 and the pivots are non-zero-- 329 00:18:13,270 --> 00:18:17,140 it factors into lower triangular times upper triangular. 330 00:18:17,140 --> 00:18:20,260 So that's a key step that is-- 331 00:18:20,260 --> 00:18:23,500 that MATLAB would do with a lu of A-- 332 00:18:23,500 --> 00:18:24,970 would produce those two factors. 333 00:18:28,480 --> 00:18:32,890 Now I want to do them in a column times row 334 00:18:32,890 --> 00:18:39,760 way, which I just realized late, was a neat way to do it. 335 00:18:39,760 --> 00:18:47,850 So can I take a matrix and do elimination? 336 00:18:47,850 --> 00:18:51,690 How big a matrix shall I take? 337 00:18:51,690 --> 00:18:55,140 2 by 2 [INAUDIBLE]. 338 00:18:55,140 --> 00:18:59,250 3 by 3 Somebody's not convinced totally by 2 by 2 339 00:18:59,250 --> 00:19:01,860 Let me do it 2 by 2 and then if you want-- if you really 340 00:19:01,860 --> 00:19:05,140 want a 3 by 3 do it. 341 00:19:05,140 --> 00:19:05,640 OK. 342 00:19:05,640 --> 00:19:11,930 Here's 2 by 2 2, 4, 3, 7. 343 00:19:11,930 --> 00:19:13,760 How's that? 344 00:19:13,760 --> 00:19:14,260 OK. 345 00:19:17,110 --> 00:19:18,650 So what is-- yeah. 346 00:19:18,650 --> 00:19:23,900 So let's remember what elimination does. 347 00:19:23,900 --> 00:19:31,430 It subtracts a multiple of that row from that one, 348 00:19:31,430 --> 00:19:34,420 to get to 2, 3. 349 00:19:34,420 --> 00:19:35,780 And the multiple is 2. 350 00:19:35,780 --> 00:19:37,910 So it knocks out the 4. 351 00:19:37,910 --> 00:19:39,140 Two 3's are 6. 352 00:19:39,140 --> 00:19:40,820 So it leaves a 1. 353 00:19:40,820 --> 00:19:42,590 And we're-- oh, yeah. 354 00:19:42,590 --> 00:19:44,930 Thanks for allowing me to do 2 by 2. 355 00:19:44,930 --> 00:19:46,520 I've already done it now. 356 00:19:46,520 --> 00:19:50,720 I've reached U. So here's A. And here's U-- 357 00:19:50,720 --> 00:19:54,290 the upper triangular guy with the pivots on the diagonal. 358 00:19:56,810 --> 00:20:01,920 And then the question is, express 359 00:20:01,920 --> 00:20:05,910 that step in matrix language. 360 00:20:05,910 --> 00:20:10,440 And the right answer is l times U. So the right answer 361 00:20:10,440 --> 00:20:14,970 is that this A is-- 362 00:20:14,970 --> 00:20:18,660 so I'll just erase that letter U-- 363 00:20:18,660 --> 00:20:23,430 is l times U. So what is l? 364 00:20:23,430 --> 00:20:26,130 L is the lower triangular guy. 365 00:20:26,130 --> 00:20:31,470 And it has there the number that you used here. 366 00:20:31,470 --> 00:20:32,490 And what was that? 367 00:20:32,490 --> 00:20:35,790 I subtracted 2 of that row from this. 368 00:20:35,790 --> 00:20:37,450 So I want a 2 there. 369 00:20:44,790 --> 00:20:46,170 So that would be-- 370 00:20:46,170 --> 00:20:48,090 I would call that a multiplier. 371 00:20:48,090 --> 00:20:50,850 I multiplied row 1 by 2. 372 00:20:50,850 --> 00:20:53,280 Subtracted to get that 0. 373 00:20:53,280 --> 00:20:57,410 And for a 2 by 2 example, I was finished. 374 00:20:57,410 --> 00:20:57,910 OK. 375 00:20:57,910 --> 00:21:00,810 Now I want to see this-- 376 00:21:00,810 --> 00:21:02,770 so there is l times U. It happened. 377 00:21:02,770 --> 00:21:04,770 Right. 378 00:21:04,770 --> 00:21:10,470 I would like to see how l times U comes out of this row-- 379 00:21:10,470 --> 00:21:12,590 column times row. 380 00:21:12,590 --> 00:21:14,370 So let me start-- 381 00:21:14,370 --> 00:21:15,980 let me think again. 382 00:21:20,140 --> 00:21:23,490 So really the point of elimination-- 383 00:21:23,490 --> 00:21:26,220 why did we do this in the first place? 384 00:21:26,220 --> 00:21:30,313 Because here we had two coupled equations. 385 00:21:30,313 --> 00:21:31,480 There were coupled together. 386 00:21:31,480 --> 00:21:33,660 We couldn't solve them instantly. 387 00:21:33,660 --> 00:21:38,250 That step of elimination reduced me to-- 388 00:21:38,250 --> 00:21:39,840 down here in this corner-- 389 00:21:39,840 --> 00:21:44,820 one equation, that I've eliminated the first unknown x 390 00:21:44,820 --> 00:21:47,440 from the second equation. 391 00:21:47,440 --> 00:21:52,140 So the second equation is 0x plus 1y equal right hand side. 392 00:21:52,140 --> 00:21:54,800 And I solve it immediately. 393 00:21:54,800 --> 00:21:55,800 OK. 394 00:21:55,800 --> 00:22:01,680 So how did I get to that 1 by 1 problem, 395 00:22:01,680 --> 00:22:03,480 with these guys removed? 396 00:22:03,480 --> 00:22:06,822 Well-- yeah. 397 00:22:06,822 --> 00:22:09,130 I'll just write-- can I write here the-- 398 00:22:09,130 --> 00:22:13,500 my parallel way to think of it? 399 00:22:13,500 --> 00:22:15,960 2 by 2 is pretty small, I admit. 400 00:22:15,960 --> 00:22:16,620 OK. 401 00:22:16,620 --> 00:22:19,260 So I start with 2, 3, 4, 7. 402 00:22:22,090 --> 00:22:24,790 I want to split it into-- 403 00:22:24,790 --> 00:22:30,770 I want to get the first row and column in one piece. 404 00:22:30,770 --> 00:22:32,780 Something goes there. 405 00:22:32,780 --> 00:22:38,610 And the other piece is something there. 406 00:22:38,610 --> 00:22:40,350 OK. 407 00:22:40,350 --> 00:22:42,750 That's what elimination has done. 408 00:22:42,750 --> 00:22:45,210 It's taken the original matrix. 409 00:22:45,210 --> 00:22:47,440 It's split-- these are both rank 1. 410 00:22:54,080 --> 00:22:56,030 So let's just-- first of all, you 411 00:22:56,030 --> 00:23:01,280 could tell me what goes in that blank space in the first rank 1 412 00:23:01,280 --> 00:23:01,790 matrix. 413 00:23:01,790 --> 00:23:05,120 So what-- can I say this in words? 414 00:23:05,120 --> 00:23:11,630 The first stage of elimination pulls off from A-- 415 00:23:11,630 --> 00:23:14,090 so A is some big matrix. 416 00:23:14,090 --> 00:23:19,070 It pulls off from A. It takes account of the first column 417 00:23:19,070 --> 00:23:20,540 and row. 418 00:23:20,540 --> 00:23:25,910 So it writes A as-- 419 00:23:25,910 --> 00:23:33,560 here we go-- as a first column, say column 420 00:23:33,560 --> 00:23:40,100 1, row 1, plus the easy part. 421 00:23:40,100 --> 00:23:45,740 The easy part will be a matrix with all zeros there-- 422 00:23:45,740 --> 00:23:47,090 all zeros there. 423 00:23:47,090 --> 00:23:49,100 And here I have a 2. 424 00:23:49,100 --> 00:23:51,740 Can I call it a 2? 425 00:23:51,740 --> 00:23:56,690 This is my way now to think about what 426 00:23:56,690 --> 00:23:58,670 elimination is really doing. 427 00:23:58,670 --> 00:24:01,400 It's starting with an n by n matrix. 428 00:24:01,400 --> 00:24:04,580 It's pulling off a rank 1 matrix, 429 00:24:04,580 --> 00:24:09,140 which gets that column and that row correct. 430 00:24:09,140 --> 00:24:11,660 And it gets whatever it has in here. 431 00:24:11,660 --> 00:24:16,400 And then the rest of what's in there is A2. 432 00:24:16,400 --> 00:24:19,520 Do you see that we've done that here? 433 00:24:19,520 --> 00:24:23,270 The first step got the first row and column correct. 434 00:24:23,270 --> 00:24:25,400 And if it's rank 1, what number goes there? 435 00:24:25,400 --> 00:24:26,140 AUDIENCE: 6. 436 00:24:26,140 --> 00:24:27,928 PROFESSOR: 6. 437 00:24:27,928 --> 00:24:29,269 6 for there. 438 00:24:29,269 --> 00:24:31,340 And then this is the rest. 439 00:24:31,340 --> 00:24:35,900 This is what we have, one size smaller to work on. 440 00:24:35,900 --> 00:24:38,380 And it looks like it was 7. 441 00:24:38,380 --> 00:24:39,600 6 has been used. 442 00:24:39,600 --> 00:24:40,340 So it's a 1. 443 00:24:43,520 --> 00:24:52,130 That's really-- I want to think of this rank 1 matrix 444 00:24:52,130 --> 00:24:58,610 as the first column of l times the first row of u. 445 00:24:58,610 --> 00:25:01,670 And then this guy is the second column 446 00:25:01,670 --> 00:25:05,110 of l times the second row of u. 447 00:25:11,850 --> 00:25:13,270 OK. 448 00:25:13,270 --> 00:25:17,820 I haven't presented as proof in a class before. 449 00:25:20,380 --> 00:25:23,560 And for 2 by 2, it's looking like overkill to me. 450 00:25:23,560 --> 00:25:24,610 I mean, why? 451 00:25:24,610 --> 00:25:26,710 You don't have to do all that deep thinking 452 00:25:26,710 --> 00:25:29,540 to get the pieces. 453 00:25:29,540 --> 00:25:35,230 But my idea is that it gives the breakdown. 454 00:25:35,230 --> 00:25:39,370 And this, of course, is, by our column times 455 00:25:39,370 --> 00:25:43,140 row rule, that's LU. 456 00:25:43,140 --> 00:25:48,510 So we're starting with A, and we're breaking it up into LU, 457 00:25:48,510 --> 00:25:49,890 where lu-- 458 00:25:49,890 --> 00:25:52,140 the first piece of lu-- 459 00:25:52,140 --> 00:25:55,920 is the first column times row. 460 00:25:55,920 --> 00:26:01,710 And then the next pieces are the rest of the matrix. 461 00:26:01,710 --> 00:26:06,240 And those get broken down-- the next stage of elimination would 462 00:26:06,240 --> 00:26:07,020 break-- 463 00:26:07,020 --> 00:26:10,650 if I had a 3 by 3, this stage peeled off 464 00:26:10,650 --> 00:26:12,540 the first column and row-- 465 00:26:12,540 --> 00:26:15,790 then the next stage would peel off the second-- 466 00:26:15,790 --> 00:26:17,970 the new second column and row. 467 00:26:17,970 --> 00:26:21,370 And the third stage would have the third column and row-- 468 00:26:21,370 --> 00:26:25,500 just the last pivot does this make any sense to you? 469 00:26:25,500 --> 00:26:30,000 You could email me and say it's not that great. 470 00:26:30,000 --> 00:26:32,420 But I think it's-- 471 00:26:32,420 --> 00:26:39,410 to see that the final result of elimination is l times u is-- 472 00:26:39,410 --> 00:26:45,720 there's a little magic in seeing what you're doing. 473 00:26:45,720 --> 00:26:48,380 And I think this is a way to see what you're doing-- 474 00:26:48,380 --> 00:26:52,970 that you're peeling off a first part 475 00:26:52,970 --> 00:26:55,790 to leave a second part like that. 476 00:26:55,790 --> 00:26:58,250 Then the second part, you would peel off 477 00:26:58,250 --> 00:27:01,970 the second column times the second row, 478 00:27:01,970 --> 00:27:05,030 maybe divide by the pivot to make it correct. 479 00:27:05,030 --> 00:27:09,020 And that would put something in the rest of the box. 480 00:27:09,020 --> 00:27:14,640 And then A3 would be the rest of that box. 481 00:27:14,640 --> 00:27:15,420 OK. 482 00:27:15,420 --> 00:27:17,430 I'm stopping here. 483 00:27:17,430 --> 00:27:23,040 I'm glad you let me do 2 by 2, since I see that 3 by 3 484 00:27:23,040 --> 00:27:24,840 would have ruined the day. 485 00:27:24,840 --> 00:27:25,370 Yeah. 486 00:27:25,370 --> 00:27:26,245 OK. 487 00:27:26,245 --> 00:27:30,450 A question, or let me pause for a minute. 488 00:27:30,450 --> 00:27:35,770 So I've talked about these factorizations. 489 00:27:35,770 --> 00:27:38,200 This one we won't see again. 490 00:27:38,200 --> 00:27:41,440 This one we will see, big time. 491 00:27:41,440 --> 00:27:43,510 And this one we will. 492 00:27:43,510 --> 00:27:44,770 And this one we will. 493 00:27:44,770 --> 00:27:46,270 Yeah. 494 00:27:46,270 --> 00:27:49,510 2, 3 and 5 are the ones that we're really 495 00:27:49,510 --> 00:27:51,820 going to see a lot of. 496 00:27:51,820 --> 00:27:54,250 Questions or thoughts or-- 497 00:27:54,250 --> 00:27:55,450 OK. 498 00:27:55,450 --> 00:28:00,760 I guess I want to tell you now to complete today's-- 499 00:28:00,760 --> 00:28:03,100 moving forward in this subject-- 500 00:28:03,100 --> 00:28:06,070 the fundamental theorem of linear algebra-- 501 00:28:06,070 --> 00:28:08,420 the fundamental theorem of linear algebra. 502 00:28:08,420 --> 00:28:08,920 OK. 503 00:28:08,920 --> 00:28:10,890 Ready for that? 504 00:28:10,890 --> 00:28:13,590 Or you may have seen it already, because it's 505 00:28:13,590 --> 00:28:17,970 like the highlight of this subject-- 506 00:28:17,970 --> 00:28:21,510 of the basic ideas in this subject. 507 00:28:21,510 --> 00:28:22,010 Right. 508 00:28:22,010 --> 00:28:23,300 And then maybe I can-- 509 00:28:23,300 --> 00:28:25,010 after I tell you that theorem-- 510 00:28:28,600 --> 00:28:32,560 people around the world send me homework problems to do. 511 00:28:32,560 --> 00:28:35,680 Now, you would think any sensible professor would never 512 00:28:35,680 --> 00:28:37,000 do those problems. 513 00:28:37,000 --> 00:28:39,190 He would say, it's your problem. 514 00:28:39,190 --> 00:28:43,990 But I get carried away and I solve them sometimes. 515 00:28:43,990 --> 00:28:47,110 So one came from India last week, 516 00:28:47,110 --> 00:28:50,290 and it involved the fundamental theorem of linear algebra. 517 00:28:50,290 --> 00:28:53,970 Whoever teaching it there really was on the ball. 518 00:28:53,970 --> 00:28:56,560 And well, I'll tell you that problem 519 00:28:56,560 --> 00:29:00,160 after the fundamental theorem. 520 00:29:00,160 --> 00:29:00,760 OK. 521 00:29:00,760 --> 00:29:01,600 Fundamental theorem. 522 00:29:06,190 --> 00:29:09,410 It's about four sub spaces. 523 00:29:09,410 --> 00:29:13,720 So I invented the name four fundamental sub spaces. 524 00:29:13,720 --> 00:29:16,390 So can I list the four sub places? 525 00:29:16,390 --> 00:29:20,830 Fundamental sub spaces. 526 00:29:20,830 --> 00:29:23,980 Well, we know one of them already. 527 00:29:23,980 --> 00:29:29,670 The column space-- so, for a matrix. 528 00:29:29,670 --> 00:29:32,060 We are given a matrix A-- 529 00:29:32,060 --> 00:29:35,090 that's m by n of rank r. 530 00:29:37,660 --> 00:29:39,600 That's our normal starting point. 531 00:29:43,490 --> 00:29:45,220 So what are those four sub spaces, 532 00:29:45,220 --> 00:29:46,540 and how are they related? 533 00:29:46,540 --> 00:29:47,860 And what's their dimension? 534 00:29:47,860 --> 00:29:51,830 And what-- those are key facts. 535 00:29:51,830 --> 00:29:52,330 OK. 536 00:29:52,330 --> 00:29:54,010 We already know the column space-- 537 00:29:59,960 --> 00:30:02,780 column space of a matrix. 538 00:30:02,780 --> 00:30:07,760 And actually, we already know the row space of a matrix. 539 00:30:07,760 --> 00:30:09,770 And we have the notation for that, 540 00:30:09,770 --> 00:30:13,430 column space of A transpose. 541 00:30:13,430 --> 00:30:16,010 And what is the dimension-- 542 00:30:16,010 --> 00:30:18,410 so that was the key point in the first lecture. 543 00:30:18,410 --> 00:30:20,300 Anybody who missed the first lecture, 544 00:30:20,300 --> 00:30:28,880 should go back to the notes of 1.1 545 00:30:28,880 --> 00:30:36,950 for the thinking that goes into the dimension equals what? 546 00:30:36,950 --> 00:30:39,580 Which of those three numbers do I want to-- 547 00:30:39,580 --> 00:30:42,530 is the dimension of the column space? 548 00:30:42,530 --> 00:30:46,990 R. And what can I say about r, right away, 549 00:30:46,990 --> 00:30:47,950 compared to m and n? 550 00:30:47,950 --> 00:30:49,733 AUDIENCE: [INAUDIBLE]. 551 00:30:49,733 --> 00:30:50,400 PROFESSOR: Yeah. 552 00:30:50,400 --> 00:30:52,320 Less or equal. 553 00:30:52,320 --> 00:30:57,120 r couldn't-- I couldn't have more independent columns than I 554 00:30:57,120 --> 00:30:58,760 have columns. 555 00:30:58,760 --> 00:31:01,260 So I've got n columns. 556 00:31:01,260 --> 00:31:04,890 So r of them are independent. 557 00:31:04,890 --> 00:31:09,120 So r is somewhere, less or equal-- hopefully equal to n. 558 00:31:09,120 --> 00:31:11,440 What about the dimension of the row space? 559 00:31:11,440 --> 00:31:15,116 How many independent rows has the matrix got? 560 00:31:15,116 --> 00:31:15,616 AUDIENCE: R. 561 00:31:15,616 --> 00:31:16,710 PROFESSOR: R. Thank you. 562 00:31:16,710 --> 00:31:23,910 That's the great fact with a new proof last time 563 00:31:23,910 --> 00:31:26,910 in section 1.1-- 564 00:31:26,910 --> 00:31:30,670 that those have the same dimension, same dimension. 565 00:31:30,670 --> 00:31:33,700 Which is-- you think, oh, OK. 566 00:31:33,700 --> 00:31:35,680 You look at a simple example. 567 00:31:35,680 --> 00:31:36,760 It's true. 568 00:31:36,760 --> 00:31:42,760 But if you're given a matrix that's 50 by 100, 569 00:31:42,760 --> 00:31:46,580 really the fact that those 100 columns 570 00:31:46,580 --> 00:31:50,450 have the same number of independent ones as those 50 571 00:31:50,450 --> 00:31:51,110 row-- 572 00:31:51,110 --> 00:31:53,840 that's like great. 573 00:31:53,840 --> 00:31:54,470 OK. 574 00:31:54,470 --> 00:31:59,700 Now the other spaces are the null space 575 00:31:59,700 --> 00:32:03,820 of the matrix, N of A. 576 00:32:03,820 --> 00:32:07,250 And just to make everything naturally symmetric, 577 00:32:07,250 --> 00:32:11,650 the null space of A transpose. 578 00:32:14,170 --> 00:32:16,600 Those are the last two. 579 00:32:16,600 --> 00:32:21,020 Those are the four fundamental sub spaces, which you've seen. 580 00:32:21,020 --> 00:32:25,150 And they're even on the cover of the linear algebra textbook. 581 00:32:25,150 --> 00:32:26,680 OK. 582 00:32:26,680 --> 00:32:28,822 So what's the null space? 583 00:32:28,822 --> 00:32:30,640 AUDIENCE: It's the set of [INAUDIBLE].. 584 00:32:30,640 --> 00:32:32,920 PROFESSOR: It's the set of solutions to Ax 585 00:32:32,920 --> 00:32:34,510 equals 0, right. 586 00:32:34,510 --> 00:32:48,130 Null space is all solutions to Ax equal 0. 587 00:32:48,130 --> 00:32:50,050 So the null space has vector-- 588 00:32:50,050 --> 00:32:51,720 these vectors in it-- the x's. 589 00:32:54,270 --> 00:32:57,410 The null space isn't taken from the matrix. 590 00:32:57,410 --> 00:33:01,100 The row space and the column space-- those numbers 591 00:33:01,100 --> 00:33:03,170 are sitting in the matrix. 592 00:33:03,170 --> 00:33:07,100 The null space, and the null space of A transpose, 593 00:33:07,100 --> 00:33:09,930 are solutions to-- 594 00:33:09,930 --> 00:33:14,670 the word null is reflecting the fact that that's a 0. 595 00:33:14,670 --> 00:33:16,710 And that's what makes it a space. 596 00:33:16,710 --> 00:33:18,210 Now can you just-- 597 00:33:18,210 --> 00:33:20,890 let me just ask you to think again. 598 00:33:20,890 --> 00:33:23,580 What's implied when I say-- 599 00:33:23,580 --> 00:33:25,480 when I use the word space-- 600 00:33:25,480 --> 00:33:27,645 a space of vectors? 601 00:33:27,645 --> 00:33:29,020 AUDIENCE: Closed under addition-- 602 00:33:29,020 --> 00:33:30,040 PROFESSOR: I can add-- 603 00:33:30,040 --> 00:33:31,020 yeah. 604 00:33:31,020 --> 00:33:37,930 So I can do the most important operations of linear algebra 605 00:33:37,930 --> 00:33:39,460 in that space. 606 00:33:39,460 --> 00:33:41,120 I can add two vectors. 607 00:33:41,120 --> 00:33:42,970 Here, let me just add them. 608 00:33:42,970 --> 00:33:45,760 So here I'll have a vector x, and let 609 00:33:45,760 --> 00:33:49,960 me say another one, a vector y. 610 00:33:49,960 --> 00:33:53,140 Then, I do addition. 611 00:33:53,140 --> 00:33:54,970 I follow the rules. 612 00:33:54,970 --> 00:34:00,830 I see that this can be written as Ax plus y is 0 plus 0. 613 00:34:00,830 --> 00:34:03,340 So what have I learned? 614 00:34:03,340 --> 00:34:07,090 I've learned that if x is in the null space, 615 00:34:07,090 --> 00:34:10,750 and y is in the null space, then x plus y. 616 00:34:10,750 --> 00:34:13,659 So the null space is, as you said, closed, 617 00:34:13,659 --> 00:34:16,449 meaning I don't go outside it. 618 00:34:16,449 --> 00:34:19,929 If x is in it, and y is in it, then the sum is in it. 619 00:34:19,929 --> 00:34:27,429 And similarly, from Ax equals 0, I get to A times cx equals 0. 620 00:34:27,429 --> 00:34:31,420 Just multiply by c-- by a number c. 621 00:34:31,420 --> 00:34:33,460 So those two facts-- 622 00:34:33,460 --> 00:34:37,210 that means I can do linear algebra. 623 00:34:37,210 --> 00:34:39,760 I can multiply by numbers. 624 00:34:39,760 --> 00:34:40,960 And I can add. 625 00:34:40,960 --> 00:34:44,199 In other words, I can take linear combinations. 626 00:34:44,199 --> 00:34:46,480 That's what you do with vectors. 627 00:34:46,480 --> 00:34:48,550 And the point is, if I do it-- 628 00:34:48,550 --> 00:34:52,719 if I take combinations of two null space guys, 629 00:34:52,719 --> 00:34:55,310 I'm still in the null space. 630 00:34:55,310 --> 00:34:56,300 OK. 631 00:34:56,300 --> 00:35:01,350 So that's the point of the null space. 632 00:35:01,350 --> 00:35:06,800 And-- well now-- so now part of the fundamental theorem 633 00:35:06,800 --> 00:35:11,630 is to figure out how many independent vectors are 634 00:35:11,630 --> 00:35:12,680 in the null space. 635 00:35:12,680 --> 00:35:16,490 How many solutions-- independent solutions-- 636 00:35:16,490 --> 00:35:19,260 does that system of equations have? 637 00:35:19,260 --> 00:35:21,380 So that would be the dimension. 638 00:35:21,380 --> 00:35:25,070 And I have to ask you what it is. 639 00:35:25,070 --> 00:35:27,080 Let me draw a picture, while you're 640 00:35:27,080 --> 00:35:29,180 thinking about those spaces. 641 00:35:32,190 --> 00:35:35,710 It's fantastic to have these beautiful clean boards. 642 00:35:35,710 --> 00:35:36,210 OK. 643 00:35:36,210 --> 00:35:39,870 So here's my picture of the row space. 644 00:35:43,630 --> 00:35:48,680 Row-- that's the column space of A transpose. 645 00:35:48,680 --> 00:35:58,265 And here's my picture of the null space, N of A. 646 00:35:58,265 --> 00:36:01,085 And that's the solutions to Ax equals 0. 647 00:36:03,680 --> 00:36:06,650 And why have I put these two together, 648 00:36:06,650 --> 00:36:08,910 and these two together? 649 00:36:08,910 --> 00:36:17,120 So-- and the other pair will be the column space, C of A. 650 00:36:17,120 --> 00:36:26,172 and the null space of A transpose. 651 00:36:26,172 --> 00:36:27,380 So there are the four spaces. 652 00:36:29,930 --> 00:36:33,380 Their relationship is the fundamental theorem 653 00:36:33,380 --> 00:36:35,630 of linear algebra. 654 00:36:35,630 --> 00:36:40,800 So first of all, what-- 655 00:36:40,800 --> 00:36:42,285 so I have an m by n matrix. 656 00:36:45,650 --> 00:36:48,360 So that tells me that my row-- 657 00:36:48,360 --> 00:36:51,550 a typical row has n components, right? 658 00:36:51,550 --> 00:36:53,250 I look at an m by n matrix. 659 00:36:53,250 --> 00:36:55,805 Let's do a 2 by 3 matrix. 660 00:37:00,060 --> 00:37:04,890 So if I look at the row space, this is m. 661 00:37:04,890 --> 00:37:06,380 And this is n. 662 00:37:06,380 --> 00:37:08,270 So I see three-- 663 00:37:08,270 --> 00:37:12,280 the rows have length three. 664 00:37:12,280 --> 00:37:14,950 And of course, they multiply the x's, 665 00:37:14,950 --> 00:37:19,030 which also have the length three, x1, x2, x3. 666 00:37:19,030 --> 00:37:21,220 That's why these are together, because they're 667 00:37:21,220 --> 00:37:23,650 both in n dimensional space. 668 00:37:29,150 --> 00:37:31,250 Then why are these together? 669 00:37:31,250 --> 00:37:36,540 Because the columns are in two dimensional space, 670 00:37:36,540 --> 00:37:38,050 for this example. 671 00:37:38,050 --> 00:37:41,350 And the null space of A transpose 672 00:37:41,350 --> 00:37:49,390 would be just two components, like y1 and y2, to give 0s. 673 00:37:49,390 --> 00:37:51,580 So do you see that this is R-- 674 00:37:51,580 --> 00:37:54,730 these guys are in Rn? 675 00:37:54,730 --> 00:37:56,980 So that's the first-- 676 00:37:56,980 --> 00:37:58,930 get things straight. 677 00:37:58,930 --> 00:38:02,770 Two spaces in Rn, two spaces in Rm. 678 00:38:08,040 --> 00:38:12,570 Now, what am I going to ask about these spaces? 679 00:38:12,570 --> 00:38:14,430 I guess I've already started asking 680 00:38:14,430 --> 00:38:16,610 and didn't wait for an answer. 681 00:38:16,610 --> 00:38:18,270 Their dimension. 682 00:38:18,270 --> 00:38:24,690 So this has dimension R. And what's the dimension-- 683 00:38:24,690 --> 00:38:31,450 how many-- this is really such a key fact. 684 00:38:31,450 --> 00:38:37,880 If I have m equations, Ax equals 0. 685 00:38:37,880 --> 00:38:41,630 And if R of those equations are independent, 686 00:38:41,630 --> 00:38:43,730 how many solutions? 687 00:38:43,730 --> 00:38:45,500 So the dimension of this space is 688 00:38:45,500 --> 00:38:48,950 going to tell me how many solutions 689 00:38:48,950 --> 00:38:53,360 to Ax in two m equations, but really 690 00:38:53,360 --> 00:39:02,610 only are genuine independent equations in this system Ax 691 00:39:02,610 --> 00:39:03,870 equals 0. 692 00:39:03,870 --> 00:39:04,840 How many? 693 00:39:04,840 --> 00:39:06,650 So can I ask the question again? 694 00:39:06,650 --> 00:39:12,570 And I want the answer in terms of m, and n, and R. So I have-- 695 00:39:12,570 --> 00:39:15,610 I really have R equations. 696 00:39:15,610 --> 00:39:17,680 If I look at Ax equals 0, it looks 697 00:39:17,680 --> 00:39:19,660 like m separate equations. 698 00:39:19,660 --> 00:39:23,020 But m minus R-- 699 00:39:23,020 --> 00:39:27,210 of those-- are just copies or combinations of others. 700 00:39:27,210 --> 00:39:30,180 So there are independent equations. 701 00:39:30,180 --> 00:39:32,648 So what's-- how many have I got? 702 00:39:32,648 --> 00:39:35,483 AUDIENCE: [INAUDIBLE]. 703 00:39:35,483 --> 00:39:37,900 PROFESSOR: And And that's what I'm going to write in here. 704 00:39:40,870 --> 00:39:41,740 So-- yeah. 705 00:39:41,740 --> 00:39:45,970 So x has n components. 706 00:39:45,970 --> 00:39:51,490 And there are real active equations 707 00:39:51,490 --> 00:39:53,680 that they have to satisfy. 708 00:39:53,680 --> 00:39:58,282 And that leaves n minus R. That's the key point. 709 00:39:58,282 --> 00:40:04,790 That's the key point that there are n components of x-- 710 00:40:04,790 --> 00:40:07,610 n unknowns-- n unknowns. 711 00:40:07,610 --> 00:40:11,020 And there are R constraints-- 712 00:40:11,020 --> 00:40:13,170 independent constraints. 713 00:40:13,170 --> 00:40:16,270 So those n get-- 714 00:40:16,270 --> 00:40:19,960 if I want to satisfy those constraints that knocks out 715 00:40:19,960 --> 00:40:23,050 dimension R, and leaves n minus R. So that's 716 00:40:23,050 --> 00:40:23,920 the dimension here. 717 00:40:26,860 --> 00:40:31,960 And the beauty of the count is that those two numbers add up 718 00:40:31,960 --> 00:40:33,070 to n. 719 00:40:33,070 --> 00:40:35,110 Everybody's accounted for. 720 00:40:35,110 --> 00:40:39,590 Every vector has a piece in the row space 721 00:40:39,590 --> 00:40:41,450 and a piece and the null space. 722 00:40:41,450 --> 00:40:47,680 And that's-- those two pieces give you back the vector. 723 00:40:47,680 --> 00:40:48,650 Do you see that? 724 00:40:48,650 --> 00:40:51,300 That's just nice, that the numbers come out right. 725 00:40:51,300 --> 00:40:54,560 And of course, they come out right here, too. 726 00:40:54,560 --> 00:40:57,200 You could say, just transpose the matrix 727 00:40:57,200 --> 00:40:59,660 and write it-- write the same thing again. 728 00:40:59,660 --> 00:41:01,580 What's the dimension of the column space? 729 00:41:05,537 --> 00:41:06,130 Equals? 730 00:41:06,130 --> 00:41:08,370 The other column space in the matrix has dimension-- 731 00:41:08,370 --> 00:41:09,360 AUDIENCE: R. 732 00:41:09,360 --> 00:41:11,190 PROFESSOR: R. Right. 733 00:41:11,190 --> 00:41:15,870 And the row-- this guy is left out of some linear algebra 734 00:41:15,870 --> 00:41:18,660 books, as if it doesn't belong. 735 00:41:18,660 --> 00:41:23,610 But isn't it clear that without it, everything 736 00:41:23,610 --> 00:41:28,050 is only three quarters done? 737 00:41:28,050 --> 00:41:29,820 We have to have this guy. 738 00:41:29,820 --> 00:41:32,526 And its dimension is 739 00:41:32,526 --> 00:41:34,280 AUDIENCE: M minus R. 740 00:41:34,280 --> 00:41:37,580 PROFESSOR: M minus R. Yeah. 741 00:41:37,580 --> 00:41:41,780 That count is just, for A transpose, what 742 00:41:41,780 --> 00:41:44,390 this count was for A. Yeah. 743 00:41:44,390 --> 00:41:50,120 So we've got those dimensions, R and n minus R, R and m 744 00:41:50,120 --> 00:41:51,830 minus R. Yeah. 745 00:41:54,580 --> 00:41:57,780 You'll have known this, but we need 746 00:41:57,780 --> 00:42:05,070 to see it once again in 2018, before we start using it. 747 00:42:05,070 --> 00:42:06,980 Now is that the fundamental theorem? 748 00:42:06,980 --> 00:42:10,040 Is that all to it there is? 749 00:42:10,040 --> 00:42:12,330 No. 750 00:42:12,330 --> 00:42:14,910 There is another piece to the fundamental theorem, 751 00:42:14,910 --> 00:42:19,170 which is, sort of you could say, the geometry. 752 00:42:19,170 --> 00:42:20,890 Here I have a sub space. 753 00:42:20,890 --> 00:42:26,630 Here I have a subspace of this big n dimensional space. 754 00:42:26,630 --> 00:42:31,010 So I visualize those sub spaces as some kind of a plane-- 755 00:42:31,010 --> 00:42:36,840 an R dimensional plane- and an n minus R dimensional plane. 756 00:42:36,840 --> 00:42:41,970 And I want to see how are those two planes connected. 757 00:42:41,970 --> 00:42:45,050 How are those two planes connected? 758 00:42:45,050 --> 00:42:49,240 And let me get a piece of the-- 759 00:42:49,240 --> 00:42:52,460 blank piece of the board to remember the final step. 760 00:42:52,460 --> 00:42:53,350 Right. 761 00:42:53,350 --> 00:43:02,860 So we've got dimensions r and n minus r, and then over here r, 762 00:43:02,860 --> 00:43:05,470 and m minus r. 763 00:43:05,470 --> 00:43:05,970 OK. 764 00:43:05,970 --> 00:43:09,310 And this is for the rows. 765 00:43:09,310 --> 00:43:12,590 This is for the null space. 766 00:43:12,590 --> 00:43:15,400 So this has the rows in it. 767 00:43:15,400 --> 00:43:16,540 This has the null-- 768 00:43:16,540 --> 00:43:19,210 the solutions to Ax equals 0. 769 00:43:19,210 --> 00:43:23,850 What is the beautiful geometry-- 770 00:43:26,470 --> 00:43:30,005 how do you visualize those two spaces? 771 00:43:30,005 --> 00:43:30,880 How do you visualize? 772 00:43:30,880 --> 00:43:32,445 Let me take an example. 773 00:43:32,445 --> 00:43:39,250 Let A be 1, 2, 4, 2, 4, 8. 774 00:43:39,250 --> 00:43:40,380 Sorry about that. 775 00:43:40,380 --> 00:43:41,980 That's kind of a-- 776 00:43:41,980 --> 00:43:43,765 you see it hoked up example. 777 00:43:46,330 --> 00:43:49,480 So this is 2 by 3. 778 00:43:49,480 --> 00:43:52,250 So there n is 3. 779 00:43:52,250 --> 00:43:54,440 What's in the null space of this matrix? 780 00:43:58,770 --> 00:44:04,660 Can you see a vector that solves Ax equals 0? 781 00:44:04,660 --> 00:44:06,780 And in fact, how many will there be? 782 00:44:06,780 --> 00:44:09,150 What's-- yeah, what's r for this matrix? 783 00:44:09,150 --> 00:44:11,910 Just tell me all the good stuff. 784 00:44:11,910 --> 00:44:20,110 For that example, m is 2, n is 3, and r is-- 785 00:44:20,110 --> 00:44:21,010 AUDIENCE: 1. 786 00:44:21,010 --> 00:44:22,260 PROFESSOR: 1. 787 00:44:22,260 --> 00:44:24,780 Everybody sees 1 for the rank? 788 00:44:24,780 --> 00:44:28,060 The rows are dependent. 789 00:44:28,060 --> 00:44:30,300 There's only one independent row. 790 00:44:30,300 --> 00:44:31,980 The columns are dependent. 791 00:44:31,980 --> 00:44:35,910 There's only-- every column is a multiple of 1, 2. 792 00:44:35,910 --> 00:44:38,120 It's a rank 1 matrix. 793 00:44:38,120 --> 00:44:38,700 OK. 794 00:44:38,700 --> 00:44:45,640 What about its-- so its row space has dimension-- 795 00:44:45,640 --> 00:44:46,140 AUDIENCE: 1. 796 00:44:46,140 --> 00:44:47,250 PROFESSOR: 1. 797 00:44:47,250 --> 00:44:51,760 And it's null space has dimension-- 798 00:44:51,760 --> 00:44:53,030 AUDIENCE: 2. 799 00:44:53,030 --> 00:44:57,390 PROFESSOR: 2 So, cause n minus r will be 2. 800 00:44:57,390 --> 00:45:01,950 So I'm looking for a couple of vectors that both give 0. 801 00:45:01,950 --> 00:45:02,910 I believe there-- 802 00:45:02,910 --> 00:45:07,670 I think I've only got one independent row there. 803 00:45:07,670 --> 00:45:10,910 So I should be able to find two different vectors 804 00:45:10,910 --> 00:45:17,180 that solve Ax equals 0. 805 00:45:17,180 --> 00:45:20,630 So what what's the solution to Ax equals 0? 806 00:45:20,630 --> 00:45:22,010 AUDIENCE: 0 minus 2, 1. 807 00:45:22,010 --> 00:45:27,020 PROFESSOR: 0 minus 2, 1. 808 00:45:27,020 --> 00:45:29,070 Yeah, that works. 809 00:45:29,070 --> 00:45:31,280 And what's an independent solution? 810 00:45:31,280 --> 00:45:32,700 AUDIENCE: 4, 0, negative 1? 811 00:45:32,700 --> 00:45:36,180 PROFESSOR: 4, 0-- don't throw me off-- 812 00:45:36,180 --> 00:45:38,290 4, 0 and-- 813 00:45:38,290 --> 00:45:39,040 AUDIENCE: Minus 1. 814 00:45:39,040 --> 00:45:39,480 PROFESSOR: Minus 1. 815 00:45:39,480 --> 00:45:40,140 Yeah. 816 00:45:40,140 --> 00:45:42,080 That looks good. 817 00:45:42,080 --> 00:45:44,220 That looks good. 818 00:45:44,220 --> 00:45:46,980 And then the claim is that every solution would 819 00:45:46,980 --> 00:45:49,390 be a combination of those two. 820 00:45:49,390 --> 00:45:51,930 And this is how many there are. 821 00:45:51,930 --> 00:45:56,240 And now, it's the geometry I'm completing. 822 00:45:56,240 --> 00:45:58,700 So we have two minutes left in this lecture. 823 00:45:58,700 --> 00:46:01,380 You just have to tell me how-- 824 00:46:01,380 --> 00:46:06,060 what's the relation between these guys in the row space, 825 00:46:06,060 --> 00:46:08,780 and that guy in the null space? 826 00:46:08,780 --> 00:46:12,350 What's the relation between the rows of A, 827 00:46:12,350 --> 00:46:15,200 the solutions to Ax equals 0? 828 00:46:15,200 --> 00:46:17,960 Between-- if you see it-- if you saw that vector and that 829 00:46:17,960 --> 00:46:18,710 vector-- 830 00:46:18,710 --> 00:46:23,070 well, A times x is 0. 831 00:46:23,070 --> 00:46:24,470 So what does that tell us? 832 00:46:24,470 --> 00:46:29,570 What do we see for the relation between 1, 2, 4, and 0 833 00:46:29,570 --> 00:46:30,627 minus 2, 1. 834 00:46:30,627 --> 00:46:31,460 AUDIENCE: Orthogonal 835 00:46:31,460 --> 00:46:32,752 PROFESSOR: They are orthogonal. 836 00:46:32,752 --> 00:46:33,560 Terrific. 837 00:46:33,560 --> 00:46:34,550 Yes. 838 00:46:34,550 --> 00:46:37,985 Orthogonal I test by the dot product, 0 minus 4, 839 00:46:37,985 --> 00:46:39,840 4, add to 0. 840 00:46:39,840 --> 00:46:40,340 Yes. 841 00:46:40,340 --> 00:46:46,280 So the-- and that's a completely general fact. 842 00:46:46,280 --> 00:46:50,420 When I look at Ax equals 0, it's telling me 843 00:46:50,420 --> 00:46:53,250 that x is orthogonal to the rows. 844 00:46:53,250 --> 00:46:54,450 Do you see that? 845 00:46:54,450 --> 00:46:57,380 Just to put it in again here. 846 00:46:57,380 --> 00:47:03,080 If I look at Ax, A has a bunch of rows. 847 00:47:03,080 --> 00:47:05,480 X has one column. 848 00:47:05,480 --> 00:47:06,590 And I get 0. 849 00:47:06,590 --> 00:47:08,135 That's the point of the null space. 850 00:47:11,360 --> 00:47:17,600 And that equation is just saying that row 1 is orthogonal, 851 00:47:17,600 --> 00:47:20,750 because that's the dot product of row 1 with x. 852 00:47:20,750 --> 00:47:24,550 So here is row 1, row 2, row 3, and row 4 with x. 853 00:47:24,550 --> 00:47:26,360 The rows with x-- 854 00:47:26,360 --> 00:47:27,920 and I get 0s. 855 00:47:27,920 --> 00:47:36,310 So the point is then, these two spaces are at 90 degree angles. 856 00:47:36,310 --> 00:47:41,380 That's really a neat picture of the four sub spaces. 857 00:47:41,380 --> 00:47:43,990 And these two are for the same reason-- 858 00:47:43,990 --> 00:47:45,550 at 90 degree angles-- 859 00:47:45,550 --> 00:47:49,900 off in m dimensional space. 860 00:47:49,900 --> 00:47:54,220 So this is the fundamental theorem of linear algebra-- 861 00:47:54,220 --> 00:47:57,670 to see that the dimensions come out right, 862 00:47:57,670 --> 00:48:00,160 and the geometry comes out right. 863 00:48:00,160 --> 00:48:00,670 Yeah. 864 00:48:00,670 --> 00:48:02,950 And then, now, next time-- 865 00:48:02,950 --> 00:48:06,140 following the notes-- and I have a few more copies 866 00:48:06,140 --> 00:48:11,810 of the one hand out. 867 00:48:11,810 --> 00:48:18,610 We'll move on quickly next week to eigenvalues and positive 868 00:48:18,610 --> 00:48:20,980 definite matrices. 869 00:48:20,980 --> 00:48:25,650 Good this is really linear algebra moving on.