11.2 When is a Critical Point a Maximum, Minimum or Saddle? Criterion

We have seen that at a critical point a function in two dimensions can have a minimum, or a maximum or a saddle.

We want to know how to determine which will happen from the formula for the function.

We are particularly concerned about the quadratic behavior of the function at the critical point.

This behavior is determined from the second derivatives, and is that of the quadratic that those second derivatives determine.

So we really want to know: given a quadratic function of two variables, with no linear term, when will it have a maximum and when a minimum and when neither at the origin?

And here is the answer:

We can associate a matrix to the function, the matrix of its second partial derivatives at its critical point, as defined in the last section.

The behavior of the quadratic is determined by the eigenvalues of that matrix.

When they are real and positive you get a minimum, when real and negative you get a maximum, and otherwise a saddle, unless one is 0 in which case you get flatness. (Which means that for a general function you must look to higher derivatives in such directions.)

Why is that?

The eigenvector corresponding to an eigenvalue is a linear combination of the basis vectors i and j. It represents a direction in which you can move from the critical point in which the quadratic will behave like its eigenvalue multiplied by distance in that direction.

Thus if both eigenvalues are real and positive the function will look like ax'² + by'² for positive a and b with x' and y' coordinates in appropriate directions and our function will have a minimum.

On the other hand if they have opposite signs, the function will increase in one direction and decrease in the other and we will have a saddle.

And if there are complex eigenvalues?

There can't be any! Because our matrix is real and symmetric (remember that mixed partial derivatives are independent of the order in which you take them), all its eigenvalues are real.

And further, the eigenvectors corresponding to different eigenvalues are always normal to one another!

Which means that the behavior of the quadratic looks exactly like a'² + b'² does at the origin, except that the axes may be rotated into the directions of the eigenvectors.

A brief discussion of the relevant properties of matrices and eigenvalues appears in Chapter 32. You can look for two by two eigenvalues graphically with the applet: Multiplication of a Vector by a Matrix.

What happens in three dimensions?

Exactly the same statements apply, except that now we have a three by three symmetric matrix of second partial derivatives.

Its eigenvalues will be real, and you get a minimum if they all are positive and a maximum if they all are negative. With mixed signs you get a saddle, and if some eigenvectors are 0 you must look at higher derivatives in those directions to determine what goes on. Again the eigenvectors corresponding to different eigenvalues will be orthogonal.

In fact, all the same statements hold in any finite dimension.

Exercise 11.2 Find eigenvalues for the second partial matrix of the quadratic 3x² + 2xy - xz + z² + y².

Is it necessary to know the eigenvalues of the matrix of second derivatives in order to determine whether you have a maximum or minimum?

No!

You can just look at the characteristic equation. If all the possible terms in it are present and alternate in sign, you get a minimum, and if they all have the same sign you get a maximum, else neither.

Exercise 11.3 How come?