14.7 Optimization in Several Variables

Recall that optimization is the process of finding the extreme values of a function. This amounts to finding the highest and lowest points on the graph over a given domain. As we saw in the one-variable case, it is important to distinguish between local and global extreme values. A local extreme value is a value \(f(a,b)\) that is a maximum or minimum in some small open disk around \((a,b)\) (Figure 14.56).

Fermat’s Theorem states that if \(f(a)\) is a local extreme value, then \(a\) is a critical point and thus the tangent line (if it exists) is horizontal at \(x=a\). We can expect a similar result for functions of two variables, but in this case, it is the tangent plane that must be horizontal (Figure 14.57). The tangent plane to \(z=f(x,y)\) at \(P = (a,b)\) has equation \[ z = f(a,b) + f_x(a,b)(x-a)+ f_y(a,b)(y-b) \]

Thus, the tangent plane is horizontal if \(f_x(a,b) = f_y(a,b) =0\)—that is, if the equation reduces to \(z=f(a,b)\). This leads to the following definition of a critical point, where we take into account the possibility that one or both partial derivatives do not exist.

As in the single-variable case, we have

proof If \(f(x,y)\) has a local minimum at \(P=(a,b)\), then \( f(x,y)\ge f(a,b)\) for all \((x,y)\) near \((a,b)\). In particular, there exists \(r>0\) such that \(f(x,b)\ge f(a,b)\) if \(|x-a|<r\). In other words, \(g(x)=f(x,b)\) has a local minimum at \(x=a\). By Fermat’s Theorem for functions of one variable, either \(g'(a)=0\) or \(g'(a)\) does not exist. Since \(g'(a) = f_x(a,b)\), we conclude that either \(f_x(a,b)=0\) or \(f_x(a,b)\) does not exist. Similarly, \(f_y(a,b)=0\) or \(f_y(a,b)\) does not exist. Therefore, \(P=(a,b)\) is a critical point. The case of a local maximum is similar.

Usually, we deal with functions whose partial derivatives exist. In this case, finding the critical points amounts to solving the simultaneous equations \(f_x(x,y)=0\) and \(f_y(x,y)=0\).

It is not always possible to find the solutions exactly, but we can use a computer to find numerical approximations.

We know that in one variable, a function \(f(x)\) may have a point of inflection rather than a local extremum at a critical point. A similar phenomenon occurs in several variables. Each of the functions in Figure 14.60 has a critical point at \((0,0)\). However, the function in Figure 14.60 has a saddle point, which is neither a local minimum nor a local maximum. If you stand at the saddle point and begin walking, some directions take you uphill and other directions take you downhill.

As in the one-variable case, there is a Second Derivative Test for determining the type of a critical point \((a,b)\) of a function \(f(x,y)\) in two variables. This test relies on the sign of the discriminant \(D = D(a,b)\), defined as follows: \[ \boxed{ D = D(a,b) = f_{xx}(a,b)f_{yy}(a,b)-f_{xy}^2(a,b)} \]

A proof of this theorem is discussed at the end of this section.

Global Extrema

Often we are interested in finding the minimum or maximum value of a function \(f\) on a given domain \({\mathcal{D}}\). These are called global or absolute extreme values. However, global extrema do not always exist. The function \(f(x,y)=x+y\) has a maximum value on the unit square \({\mathcal{D}}_1\) in Figure 14.64 (the max is \(f(1,1)=2\)), but it has no maximum value on the entire plane \({\bf{R}}^2\).

To state conditions that guarantee the existence of global extrema, we need a few definitions. First, we say that a domain \({\mathcal{D}}\) is bounded if there is a number \(M>0\) such that \({\mathcal{D}}\) is contained in a disk of radius \(M\) centered at the origin. In other words, no point of \({\mathcal{D}}\) is more than a distance \(M\) from the origin [Figures 11(A) and 11(B)]. Next, a point \(P\) is called:

• An interior point of \({\mathcal{D}}\) if \({\mathcal{D}}\) contains some open disk \(D(P,r)\) centered at \(P\).
• A boundary point of \({\mathcal{D}}\) if every disk centered at \(P\) contains points in \({\mathcal{D}}\) and points not in \({\mathcal{D}}\).

The interior of \({\mathcal{D}}\) is the set of all interior points, and the boundary of \({\mathcal{D}}\) is the set of all boundary points. In Figure 14.66, the boundary is the curve surrounding the domain. The interior consists of all points in the domain not lying on the boundary curve.

A domain \({\mathcal{D}}\) is called closed if \({\mathcal{D}}\) contains all its boundary points (like a closed interval in \({\bf{R}}\)). A domain \({\mathcal{D}}\) is called open if every point of \({\mathcal{D}}\) is an interior point (like an open interval in \({\bf{R}}\)). The domain in Figure 14.66 is closed because the domain includes its boundary curve. In Figure 14.66, some boundary points are included and some are excluded, so the domain is neither open nor closed.

In Section 4.2, we stated two basic results. First, a continuous function \(f(x)\) on a closed, bounded interval \([a,b]\) takes on both a minimum and a maximum value on \([a,b]\). Second, these extreme values occur either at critical points in the interior \((a,b)\) or at the endpoints. Analogous results are valid in several variables.

Proof of the Second Derivative Test The proof is based on “completing the square” for quadratic forms. A quadratic form is a function \[ \boxed{Q(h,k) = ah^2+2bhk+ck^2} \] where \(a, b, c\) are constants (not all zero). The discriminant of \(Q\) is the quantity \[ \boxed{D=ac-b^2} \]

Some quadratic forms take on only positive values for \((h,k) \ne (0,0)\), and others take on both positive and negative values. According to the next theorem, the sign of the discriminant determines which of these two possibilities occurs.

proof Assume first that \(a\ne 0\) and rewrite \(Q(h,k)\) by “completing the square”: \begin{equation*} \begin{array}{rcl} Q(h,k) &=& ah^2+2bhk+ck^2 = a\left(h+\frac{b}{a}k\right)^2 + \left(c-\frac{b^2}{a}\right)k^2\\ &=& a\left(h+\frac{b}{a}k\right)^2 + \frac{D}{a} k^2 \end{array}\tag{1} \end{equation*}

If \(D>0\) and \(a>0\), then \(D/a>0\) and both terms in Eq. (1) are nonnegative. Furthermore, if \(Q(h,k)=0\), then each term in Eq. (1) must equal zero. Thus \(k=0\) and \(h+\tfrac{b}{a}k=0\), and then, necessarily, \(h=0\). This shows that \(Q(h,k)> 0\) if \((h,k)\ne 0\), and (i) is proved. Part (ii) follows similarly. To prove (iii), note that if \(a\ne 0\) and \(D<0\), then the coefficients of the squared terms in Eq. (1) have opposite signs and \(Q(h,k)\) takes on both positive and negative values. Finally, if \(a = 0\) and \(D < 0\), then \(Q(h,k) = 2bhk + ck^2\) with \(b\neq 0\). In this case, \(Q(h,k)\) again takes on both positive and negative values.

Now assume that \(f(x,y)\) has a critical point at \(P=(a,b)\). We shall analyze \(f\) by considering the restriction of \(f(x,y)\) to the line (Figure 14.69) through \(P=(a,b)\) in the direction of a unit vector \(\left\langle h,k\right\rangle\): \[ F(t) = f(a+th,b+tk) \]

Then \(F(0) = f(a,b)\). By the Chain Rule, \[ F'(t)= f_x(a+th,b+tk)h+f_y(a+th,b+tk)k \]

Because \(P\) is a critical point, we have \(f_x(a,b) = f_y(a,b) = 0\), and therefore, \[ F'(0) = f_x(a,b)h+f_y(a,b)k = 0 \]

Thus \(t=0\) is a critical point of \(F(t)\).

Now apply the Chain Rule again: \begin{equation*} \begin{array}{rcl} F''(t)&=& \frac{d }{d t}\Big(f_x(a+th,b+tk)h+f_y(a+th,b+tk)k\Big)\\ &=& \Big(f_{xx}(a+th,b+tk)h^2 + f_{xy}(a+th,b+tk)hk\Big)\\ &&\,\qquad\qquad\qquad\qquad\quad + \Big(f_{yx}(a+th,b+tk)kh + f_{yy}(a+th,b+tk)k^2\Big)\\ &=& f_{xx}(a+th,b+tk)h^2 + 2f_{xy}(a+th,b+tk)hk + f_{yy}(a+th,b+tk)k^2 \end{array}\tag{2} \end{equation*}

We see that \(F''(t)\) is the value at \((h,k)\) of a quadratic form whose discriminant is equal to \(D(a+th,b+tk)\). Here, we set \[ D(r,s)=f_{xx}(r,s)f_{yy}(r,s)-f_{xy}(r,s)^2 \]

Note that the discriminant of \(f(x,y)\) at the critical point \(P = (a,b)\) is \(D=D(a,b)\).

Case 1: \(D(a,b) > 0\) and \(f_{xx}(a,b)>0\). We must prove that \(f(a,b)\) is a local minimum. Consider a small disk of radius \(r\) around \(P\) (Figure 14.69). Because the second derivatives are continuous near \(P\), we can choose \(r>0\) so that for every unit vector \(\left\langle h,k\right\rangle\), \[ \begin{matrix} D(a+th,b+tk)>0 &\textrm{for }|t|<r\\ f_{xx}(a+th,b+tk)>0 &\textrm{for }|t|<r \end{matrix} \]

Then \(F''(t)\) is positive for \(|t|<r\) by Theorem 4(i). This tells us that \(F(t)\) is concave up, and hence \(F(0)<F(t)\) if \(0<|t|<|r|\) (see Exercise 64 in Section 4.4). Because \(F(0) = f(a,b)\), we may conclude that \(f(a,b)\) is the minimum value of \(f\) along each segment of radius \(r\) through \((a,b)\). Therefore, \(f(a,b)\) is a local minimum value of \(f\) as claimed. The case that \(D(a,b)>0\) and \(f_{xx}(a,b)<0\) is similar.

Case 2: \(D(a,b)<0\). For \(t=0\), Eq. (2) yields \[ F''(0) = f_{xx}(a,b)h^2 + 2f_{xy}(a,b)hk + f_{yy}(a,b)k^2 \]

Since \(D(a,b)<0\), this quadratic form takes on both positive and negative values by Theorem 4(iii). Choose \(\left\langle h, k\right\rangle\) for which \(F''(0)>0\). By the Second Derivative Test in one variable, \(F(0)\) is a local minimum of \(F(t)\), and hence, there is a value \(r>0\) such that \(F(0)<F(t)\) for all \(0<|t|<r\). But we can also choose \(\left\langle h, k\right\rangle\) so that \(F''(0)<0\), in which case \(F(0) >F(t)\) for \(0<|t|<r\) for some \(r>0\). Because \(F(0) = f(a,b)\), we conclude that \(f(a,b)\) is a local min in some directions and a local max in other directions. Therefore, \(f\) has a saddle point at \(P=(a,b)\).

14.7.1 Summary

• We say that \(P=(a,b)\) is a critical point of \(f(x,y)\) if
  • – \(f_x(a,b) =0\) or \(f_x(a,b)\) does not exist, and
  • – \(f_y(a,b)=0\) or \(f_y(a,b)\) does not exist.
  In \(n\)-variables, \(P=(a_1,\ldots,a_n)\) is a critical point of \(f(x_1,\ldots,x_n)\) if each partial derivative \(f_{x_j}(a_1,\ldots,a_n)\) either is zero or does not exist.
• The local minimum or maximum values of \(f\) occur at critical points.
• The discriminant of \(f(x,y)\) at \(P = (a,b)\) is the quantity \[ D(a,b) = f_{xx}(a,b)f_{yy}(a,b) - f_{xy}^2(a,b) \]
• Second Derivative Test: If \(P=(a,b)\) is a critical point of \(f(x,y)\), then \[ \begin{array}{rl} D(a,b)>0, \,\,\,f_{xx}(a,b)>0 &\quad\Rightarrow\quad f(a,b)\textrm{ is a local minimum}\\[4pt] D(a,b)>0, \,\,\, f_{xx}(a,b)<0 &\quad\Rightarrow\quad f(a,b)\textrm{ is a local maximum}\\[4pt] D(a,b)<0 &\quad\Rightarrow\quad \textrm{saddle point}\\[4pt] D(a,b)=0 &\quad\Rightarrow\quad \textrm{test inconclusive} \end{array} \]
• A point \(P\) is an interior point of a domain \({\mathcal{D}}\) if \({\mathcal{D}}\) contains some open disk \(D(P,r)\) centered at \(P\). A point \(P\) is a boundary point of \({\mathcal{D}}\) if every open disk \(D(P,r)\) contains points in \({\mathcal{D}}\) and points not in \({\mathcal{D}}\). The interior of \({\mathcal{D}}\) is the set of all interior points, and the boundary is the set of all boundary points. A domain is closed if it contains all of its boundary points and open if it is equal to its interior.
• Existence and location of global extrema: If \(f\) is continuous and \({\mathcal{D}}\) is closed and bounded, then
  • – \(f\) takes on both a minimum and a maximum value on \({\mathcal{D}}\).
  • – The extreme values occur either at critical points in the interior of \({\mathcal{D}}\) or at points on the boundary of \({\mathcal{D}}\).
  To determine the extreme values, first find the critical points in the interior of \({\mathcal{D}}\). Then compare the values of \(f\) at the critical points with the minimum and maximum values of \(f\) on the boundary.