1
00:00:02,310 --> 00:00:05,720
GILBERT STRANG: So
this is the key video
2
00:00:05,720 --> 00:00:11,750
about solving a system of n
linear constant coefficient
3
00:00:11,750 --> 00:00:13,220
equations.
4
00:00:13,220 --> 00:00:16,560
So how do I write
those equations?
5
00:00:16,560 --> 00:00:21,630
Y is now a vector, a
vector with n components.
6
00:00:21,630 --> 00:00:25,480
Instead of one scalar,
just a single number y--
7
00:00:25,480 --> 00:00:28,760
do you want me to
put an arrow on y?
8
00:00:28,760 --> 00:00:31,600
No, I won't repeat it again.
9
00:00:31,600 --> 00:00:36,640
But that's to emphasize
that y is a vector.
10
00:00:36,640 --> 00:00:40,156
Its first derivative,
it's a first order system.
11
00:00:40,156 --> 00:00:45,650
System meaning that there can
be and will be more than one
12
00:00:45,650 --> 00:00:49,720
unknown, y1, y2, to yn.
13
00:00:49,720 --> 00:00:52,420
So how do we solve
such a system?
14
00:00:52,420 --> 00:00:57,390
Then the matrix is multiplying
that y and they equate.
15
00:00:57,390 --> 00:01:00,760
The y's are coupled
together by that matrix.
16
00:01:00,760 --> 00:01:04,099
They're coupled together,
and how do we uncouple them?
17
00:01:04,099 --> 00:01:09,070
That is the magic of
eigenvalues and eigenvectors.
18
00:01:09,070 --> 00:01:15,330
Eigenvectors are vectors
that go in their own way.
19
00:01:15,330 --> 00:01:18,150
So when you have an
eigenvector, it's
20
00:01:18,150 --> 00:01:20,470
like you have a
one by one problem
21
00:01:20,470 --> 00:01:25,840
and the a becomes
just a number, lambda.
22
00:01:25,840 --> 00:01:31,700
So for a general vector,
everything is a mixed together.
23
00:01:31,700 --> 00:01:34,580
But for an
eigenvector, everything
24
00:01:34,580 --> 00:01:38,510
stays one dimensional.
25
00:01:38,510 --> 00:01:45,450
The a changes just to a lambda
for that special direction.
26
00:01:45,450 --> 00:01:50,650
And of course, as always, we
need n of those eigenvectors
27
00:01:50,650 --> 00:01:54,370
because we want to take
the starting value.
28
00:01:54,370 --> 00:01:57,220
Just as we did for
powers, we're doing it now
29
00:01:57,220 --> 00:02:00,790
for differential equations.
30
00:02:00,790 --> 00:02:05,580
I take my starting vector, which
is probably not an eigenvector.
31
00:02:05,580 --> 00:02:08,850
I'd make it a combination
of eigenvectors.
32
00:02:08,850 --> 00:02:11,700
And I'm OK because I'm
assuming that I have
33
00:02:11,700 --> 00:02:14,950
n independent eigenvectors.
34
00:02:14,950 --> 00:02:22,140
And now I follow each
starting value c1 x1--
35
00:02:22,140 --> 00:02:23,300
what does that have?
36
00:02:23,300 --> 00:02:28,010
What happens if I'm in
the direction of x1,
37
00:02:28,010 --> 00:02:35,330
then all the messiness
of A disappears.
38
00:02:35,330 --> 00:02:40,740
It acts just like lambda
1 on that vector x1.
39
00:02:40,740 --> 00:02:41,940
Here's what you get.
40
00:02:41,940 --> 00:02:46,180
You get c1, that's
just a number, times e
41
00:02:46,180 --> 00:02:51,220
to the lambda 1t x1.
42
00:02:51,220 --> 00:02:54,440
You see there,
instead of powers,
43
00:02:54,440 --> 00:02:58,500
which we had-- that we had
lambda 1 to the kth power
44
00:02:58,500 --> 00:03:01,380
when we were doing
powers of a matrix,
45
00:03:01,380 --> 00:03:03,900
now we're solving
differential equations.
46
00:03:03,900 --> 00:03:06,620
So we get an e to the lambda 1t.
47
00:03:06,620 --> 00:03:11,330
And of course, next
by superposition,
48
00:03:11,330 --> 00:03:14,630
I can add on the solution
for that one, which
49
00:03:14,630 --> 00:03:19,530
is e to the lambda
2t x2 plus so on,
50
00:03:19,530 --> 00:03:28,270
plus cne to the lambda nt xn.
51
00:03:28,270 --> 00:03:32,250
You can see when, I could
ask, when is this stable?
52
00:03:32,250 --> 00:03:34,380
When do the solutions go to 0?
53
00:03:34,380 --> 00:03:39,580
Well, as t gets
large, this number
54
00:03:39,580 --> 00:03:44,350
will go to 0, provided
lambda 1 is negative.
55
00:03:44,350 --> 00:03:48,530
Or provided its real
part is negative.
56
00:03:48,530 --> 00:03:52,630
We can understand
everything from this piece
57
00:03:52,630 --> 00:03:54,910
by piece formula.
58
00:03:54,910 --> 00:03:57,260
Let me just do an example.
59
00:03:57,260 --> 00:04:07,290
Take a matrix A. In the powers
of a matrix-- in that video
60
00:04:07,290 --> 00:04:13,270
I took a Markov matrix-- let
me take here the equivalent
61
00:04:13,270 --> 00:04:14,690
for differential equations.
62
00:04:14,690 --> 00:04:20,829
So this will give us a
Markov differential equation.
63
00:04:20,829 --> 00:04:22,570
So let me take A now.
64
00:04:26,660 --> 00:04:30,360
The columns of a
Markov matrix add to 1
65
00:04:30,360 --> 00:04:35,830
but in the differential equation
situation, they'll add to 0.
66
00:04:35,830 --> 00:04:41,790
Like minus 1 and 1,
or like minus 2 and 2.
67
00:04:41,790 --> 00:04:53,450
So there is the eigenvalue
of 1 for our powers
68
00:04:53,450 --> 00:04:59,370
is like the eigenvalue 0
for differential equations.
69
00:04:59,370 --> 00:05:02,400
Because e to the 0t is 1.
70
00:05:02,400 --> 00:05:05,240
So anyway, let's find
the eigenvalues of that.
71
00:05:05,240 --> 00:05:08,220
The first eigenvalue is 0.
72
00:05:08,220 --> 00:05:10,160
That's what I'm interested in.
73
00:05:10,160 --> 00:05:12,830
That column adds to 0,
that column adds to 0.
74
00:05:12,830 --> 00:05:15,560
That tells me there's
an eigenvalue of 0.
75
00:05:15,560 --> 00:05:16,884
And what's its eigenvector?
76
00:05:20,120 --> 00:05:26,480
Probably 2, 1 because if
I multiply that matrix
77
00:05:26,480 --> 00:05:29,050
by that vector, I get 0.
78
00:05:29,050 --> 00:05:30,900
So lambda 1 is 0.
79
00:05:30,900 --> 00:05:37,740
And my second eigenvalue,
well the trace is minus 3
80
00:05:37,740 --> 00:05:43,520
and the lambda 1 plus
lambda 2 must give minus 3.
81
00:05:43,520 --> 00:05:48,887
And its eigenvector is-- it's
probably 1 minus 1 again.
82
00:05:52,110 --> 00:05:55,420
So I've done the
preliminary work.
83
00:05:55,420 --> 00:05:59,640
Given this matrix, we've got the
eigenvalues and eigenvectors.
84
00:05:59,640 --> 00:06:03,670
Now I take u0-- what
should we say for u0?
85
00:06:03,670 --> 00:06:07,870
U0-- y0, say y of 0 to start.
86
00:06:11,170 --> 00:06:19,310
Y of 0 as some number c1
times x1 plus c2 times x2.
87
00:06:19,310 --> 00:06:22,330
Yes, no problem, no problem.
88
00:06:22,330 --> 00:06:29,390
Whatever we have, we take this--
some combination of that vector
89
00:06:29,390 --> 00:06:32,930
and that eigenvector
will give us y of 0.
90
00:06:32,930 --> 00:06:45,750
And now the y of t is c1 e
to the 0t-- e to the lambda
91
00:06:45,750 --> 00:06:50,450
1t times x1, right?
92
00:06:50,450 --> 00:06:54,970
You see, we started with
c1x1 but after a time t,
93
00:06:54,970 --> 00:06:59,156
it's either the lambda
t and here's c2.
94
00:06:59,156 --> 00:07:06,510
E to the lambda
2 is minus 3t x2.
95
00:07:06,510 --> 00:07:13,520
That's the evolution of a Markov
process, a continuous Markov
96
00:07:13,520 --> 00:07:14,930
process.
97
00:07:14,930 --> 00:07:17,920
Compared to the
powers of a matrix,
98
00:07:17,920 --> 00:07:25,000
this is a continuous evolving
evolution of this vector.
99
00:07:25,000 --> 00:07:29,740
Now, I'm interested
in steady state.
100
00:07:29,740 --> 00:07:35,920
Steady state is what
happens as t gets large.
101
00:07:35,920 --> 00:07:40,700
As t gets large, that
number goes quickly to 0.
102
00:07:40,700 --> 00:07:46,270
In our Markov matrix example,
we had 1/2 to a power,
103
00:07:46,270 --> 00:07:47,980
and that went quickly to 0.
104
00:07:47,980 --> 00:07:54,300
Now we have the exponential with
a minus 3, that goes to zero.
105
00:07:54,300 --> 00:07:57,260
E to the 0t is the 1.
106
00:07:57,260 --> 00:08:01,960
This e to the 0t equals 1.
107
00:08:01,960 --> 00:08:05,230
So that 1 is the signal
of a steady state.
108
00:08:05,230 --> 00:08:08,400
Nothing changing, nothing
really depending on time,
109
00:08:08,400 --> 00:08:09,730
just sits there.
110
00:08:09,730 --> 00:08:15,095
So I have c1x1 is
the steady state.
111
00:08:19,820 --> 00:08:22,840
And x1 was this.
112
00:08:22,840 --> 00:08:25,640
So what am I thinking?
113
00:08:25,640 --> 00:08:28,020
I'm thinking that
no matter how you
114
00:08:28,020 --> 00:08:34,760
start, no matter what y
of 0 is, as time goes on,
115
00:08:34,760 --> 00:08:37,830
the x2 part is
going to disappear.
116
00:08:37,830 --> 00:08:43,250
And if you only have the
x1 part in that ratio 2:1.
117
00:08:43,250 --> 00:08:49,960
So again, if we had
movement between Y1 Y2
118
00:08:49,960 --> 00:08:52,830
or we have things
evolving in time,
119
00:08:52,830 --> 00:09:01,200
the steady state is--
this is the steady state.
120
00:09:06,470 --> 00:09:09,910
There is an example of
a differential equation,
121
00:09:09,910 --> 00:09:12,420
happen to have a Markov matrix.
122
00:09:12,420 --> 00:09:15,490
And with a Markov
matrix, then we
123
00:09:15,490 --> 00:09:18,900
know that we'll have
an eigenvalue of -
124
00:09:18,900 --> 00:09:25,890
in the continuous case and a
negative eigenvalue that will
125
00:09:25,890 --> 00:09:28,050
disappear as time goes forward.
126
00:09:28,050 --> 00:09:30,980
E to the minus 3t goes to 0.
127
00:09:30,980 --> 00:09:32,020
Good.
128
00:09:32,020 --> 00:09:38,510
I guess I might just add a
little bit to this video, which
129
00:09:38,510 --> 00:09:45,420
is to explain why is
0 an eigenvalue when
130
00:09:45,420 --> 00:09:51,290
whenever-- if the columns added
to 0, minus 1 plus 1 is 0.
131
00:09:51,290 --> 00:09:53,000
2 minus 2 is zero.
132
00:09:53,000 --> 00:09:55,820
That tells me 0
is an eigenvalue.
133
00:09:55,820 --> 00:10:01,540
For a Markov matrix empowers
the columns added to 1 and 1
134
00:10:01,540 --> 00:10:02,710
was an eigenvalue.
135
00:10:02,710 --> 00:10:09,410
So I guess I have now two
examples of the following fact.
136
00:10:09,410 --> 00:10:21,330
That if all columns add
to some-- what shall
137
00:10:21,330 --> 00:10:28,290
I say for the sum, s for
the sum-- then lambda
138
00:10:28,290 --> 00:10:31,980
equal s is an eigenvalue.
139
00:10:37,950 --> 00:10:43,050
That was the point from
Markov matrices, s was 1.
140
00:10:43,050 --> 00:10:46,730
Every column added to 1
and then lambda equal 1
141
00:10:46,730 --> 00:10:48,170
was an eigenvalue.
142
00:10:48,170 --> 00:10:50,580
And for this video,
every column added
143
00:10:50,580 --> 00:10:54,960
to 0 and then lambda
equal 0 was an eigenvalue.
144
00:10:54,960 --> 00:10:59,300
And also, this is another
point about eigenvalues,
145
00:10:59,300 --> 00:11:00,840
good to make.
146
00:11:00,840 --> 00:11:03,720
The eigenvalues
of a transpose are
147
00:11:03,720 --> 00:11:05,700
the same as the
eigenvalues of A.
148
00:11:05,700 --> 00:11:17,460
So I could also say if
all rows of A add to s,
149
00:11:17,460 --> 00:11:23,205
then lambda equal
s is an eigenvalue.
150
00:11:26,440 --> 00:11:31,020
I'm saying that the eigenvalues
of a matrix and the eigenvalues
151
00:11:31,020 --> 00:11:33,320
of the transpose are the same.
152
00:11:33,320 --> 00:11:37,560
And maybe you would like to
just see why that's true.
153
00:11:37,560 --> 00:11:40,320
If I want the
eigenvalues of a matrix,
154
00:11:40,320 --> 00:11:44,720
I look at the determinant of
lambda I minus A. That gives me
155
00:11:44,720 --> 00:11:51,000
eigenvalues of A. If I want
the eigenvalues of a transpose,
156
00:11:51,000 --> 00:11:57,650
I would look at this
equals 0, right?
157
00:11:57,650 --> 00:12:00,670
This equaling 0.
158
00:12:00,670 --> 00:12:03,170
That equation would
give me the eigenvalues
159
00:12:03,170 --> 00:12:06,250
of a transpose just the
way this one gives me
160
00:12:06,250 --> 00:12:07,720
the eigenvalues of A.
161
00:12:07,720 --> 00:12:11,030
But why are they the same?
162
00:12:11,030 --> 00:12:15,400
Because the determinant of
a matrix and the determinant
163
00:12:15,400 --> 00:12:18,060
of its transpose are equal.
164
00:12:18,060 --> 00:12:21,630
A matrix and its transpose
have the same determinant.
165
00:12:21,630 --> 00:12:25,723
Let me just check
that with A, B, C,
166
00:12:25,723 --> 00:12:31,860
D. And the transpose
would be A, C, B, D.
167
00:12:31,860 --> 00:12:33,950
And the determinant
in both cases
168
00:12:33,950 --> 00:12:39,520
is AD minus BC, AD minus BC.
169
00:12:39,520 --> 00:12:41,060
Transposing doesn't affect.
170
00:12:41,060 --> 00:12:46,730
So this, that is
the same as that.
171
00:12:46,730 --> 00:12:49,670
And the lambdas are the same.
172
00:12:49,670 --> 00:12:52,890
And therefore we can look
at the columns adding to s
173
00:12:52,890 --> 00:12:54,615
or the rows added to s.
174
00:12:57,200 --> 00:13:03,750
So this explains why those
two statements are both true
175
00:13:03,750 --> 00:13:09,170
together because I could look
at the rows or the columns
176
00:13:09,170 --> 00:13:11,660
and reach this conclusion.
177
00:13:11,660 --> 00:13:16,490
That if all columns add
to s-- now why is that,
178
00:13:16,490 --> 00:13:18,730
or all rows add to s?
179
00:13:18,730 --> 00:13:22,930
Let me just-- I'll just
show you the eigenvector.
180
00:13:22,930 --> 00:13:30,710
In this case, A times the
eigenvector would be all ones.
181
00:13:30,710 --> 00:13:33,700
Suppose the matrix is 4 by 4.
182
00:13:33,700 --> 00:13:37,790
If I multiply A by all ones,
when you multiply a matrix
183
00:13:37,790 --> 00:13:40,730
by a vector of
ones, then the dot
184
00:13:40,730 --> 00:13:44,180
product of this row
with that is the sum,
185
00:13:44,180 --> 00:13:47,920
is that plus that plus that
plus that, would be we s.
186
00:13:47,920 --> 00:13:52,770
This would be s because
this first row-- here is
187
00:13:52,770 --> 00:13:58,106
A-- first row of A adds to s.
188
00:13:58,106 --> 00:14:02,070
So these numbers
add to s, I get s.
189
00:14:02,070 --> 00:14:05,320
These numbers add
to s, I get s again.
190
00:14:05,320 --> 00:14:07,286
These numbers add to s.
191
00:14:07,286 --> 00:14:10,660
And these, finally
those numbers add to s.
192
00:14:10,660 --> 00:14:15,575
And I have s times 1, 1, 1, 1.
193
00:14:19,930 --> 00:14:22,450
Are you OK with this?
194
00:14:22,450 --> 00:14:25,130
When all the rows add to
s, I can tell you what
195
00:14:25,130 --> 00:14:28,696
the eigenvector is, 1, 1, 1, 1.
196
00:14:28,696 --> 00:14:34,300
And then the eigenvalue, I
can see that that's the sum s.
197
00:14:34,300 --> 00:14:39,170
So again, for special
matrices, in this case
198
00:14:39,170 --> 00:14:44,810
named after Markov, we are
able to identify important fact
199
00:14:44,810 --> 00:14:52,600
about their eigenvalue, which is
that it's that common row sum s
200
00:14:52,600 --> 00:14:58,650
equal 1 in the case of
powers and s equal 0
201
00:14:58,650 --> 00:15:05,280
in this video's case with--
let me bring down A again.
202
00:15:05,280 --> 00:15:09,630
So here, every
column added to 0.
203
00:15:09,630 --> 00:15:12,790
It didn't happen that
the rows added to 0.
204
00:15:12,790 --> 00:15:14,870
I'm not requiring that.
205
00:15:14,870 --> 00:15:18,420
I'm just saying either
way, A or A transpose
206
00:15:18,420 --> 00:15:22,500
has the same eigenvalues
and one of them is 0
207
00:15:22,500 --> 00:15:28,180
and the other is whatever
the trace tells us, that one.
208
00:15:28,180 --> 00:15:32,630
These collection of useful
fact about eigenvalues
209
00:15:32,630 --> 00:15:36,430
show up when you have
a particular matrix
210
00:15:36,430 --> 00:15:40,460
and you need to know something
about its eigenvalues.
211
00:15:40,460 --> 00:15:42,880
Good, thank you.