13.3 Differentiation

In Section 13.1 we considered a few methods for graphing functions. By these methods alone it may be impossible to compute enough information to grasp even the general features of a complicated function. From elementary calculus, we know that the idea of the derivative can greatly aid us in this task; for example, it enables us to locate maxima and minima and to compute rates of change. The derivative also has many applications beyond this, as you may have seen in calculus of one variable.

Intuitively, we know from our work in Section 13.2 that a continuous function is one that has no “breaks” in its graph. A differentiable function from \({\mathbb R}^2\) to \({\mathbb R}\) ought to be such that not only are there no breaks in its graph, but there is a well-defined plane tangent to the graph at each point. Thus, there must not be any sharp folds, corners, or peaks in the graph (see Figure 13.35). In other words, the graph must be smooth.

In Chapter 2 we defined the notion of a differentiable path \({\bf c}(t)\), which is a function of a single variable \(t\) but takes values in \(\mathbb{R}^n\), Euclidean \(n\)-space. The derivative \({\bf c}'(t)\) of such a path at time \(t\) is then a vector in \(\mathbb{R}^n\).

But what about functions \(f\), say, from \(\mathbb{R}^n\) to \(\mathbb{R}^m\)? What does it mean for \(f\) to be differentiable, and more importantly, what is the derivative of a differentiable function from \(\mathbb{R}^n\) to \(\mathbb{R}^m\)? It turns out that answers to these questions was necessary both for the development of mathematics as well as theoretical physics. Without such answers the development of the mathematical fields of differential topology, differential geometry, advanced mechanics and Einstein's General Theory of Relativity (about which we will later speak) would not have been possible.

The answers to what it means to be differentiable, what a derivative actually is requires some insights. We begin with the more basic idea of a partial derivative of a real valued function of \(n\) variables.

Partial Derivatives

To make these ideas precise, we need a sound definition of what we mean by the phrase \({\rm “}f(x_1,\ldots, x_n)\) is differentiable at \({\bf x} = (x_1,\ldots ,x_n)\).” Actually, this definition is not quite as simple as one might think. Toward this end, however, let us introduce the notion of the partial derivative. This notion relies only on our knowledge of one-variable calculus. (A quick review of the definition of the derivative in a one-variable calculus text might be advisable at this point.)

In other words, \(\partial f/\partial x_j\) is just the derivative of \(f\) with respect to the variable \(x_j\), with the other variables held fixed. If \(f\colon\, {\mathbb R}^3\rightarrow {\mathbb R}\), we shall often use the notation \(\partial f/\partial x, \partial f/\partial y, \partial f/\partial z\) in place of \(\partial f/\partial x_1, \partial f/\partial x_2, \partial f/\partial x_3\). If \(f\colon\, U\subset {\mathbb R}^n\rightarrow {\mathbb R}^m\), then we can write \[ f(x_1,\ldots,x_n)=(f_1(x_1,\ldots,x_n),\ldots,f_m(x_1,\ldots,x_n)), \] so that we can speak of the partial derivatives of each component; for example, \(\partial f_m/\partial x_n\) is the partial derivative of the \(m\)th component with respect to \(x_n\), the \(n\)th variable.

To indicate that a partial derivative is to be evaluated at a particular point, for example, at \((x_0,y_0)\), we write \[ \frac{\partial f}{\partial x}(x_0,y_0)\qquad\hbox{or} \qquad \frac{\partial f}{\partial x}\bigg|_{x=x_0,y=y_0} \qquad \hbox{or}\qquad \frac{\partial f}{\partial x}\bigg|_{(x_0,y_0)}. \]

When we write \(z=f(x,y)\) for the dependent variable, we sometimes write \(\partial z/\partial x\) for \(\partial f/\partial x\). Strictly speaking, since \(z\) also represents a variable, this is an abuse of notation, but it is common practice to use these two notations interchangeably.

Sometimes the notation \(f_x, f_y, f_z\) is used for the partial derivatives: \(f_x=\dfrac{\partial f}{\partial x}, f_y= \dfrac{\partial f}{\partial y}, f_z=\dfrac{\partial f}{\partial z}\) and so on.

Because the partial derivative \(f_x(a,b)\) is the derivative \(f(x,b)\), viewed as a function of \(x\) alone, we can estimate the change \(\Delta f\) when \(x\) changes from \(a\) to \(a+\Delta x\) as in the single-variable case. Similarly, we can estimate the change when \(y\) changes by \(\Delta y\). For small \(\Delta x\) and \(\Delta y\) (just how small depends on \(f\) and the accuracy required): \[ \boxed{\begin{array}{rcl} f(a+\Delta x,b) -f(a,b)\,\,&\approx&\,\, f_x(a,b)\Delta x \\ \\ f(a,b+\Delta y) -f(a,b)\,\,&\approx&\,\, f_y(a,b)\Delta y \end{array}} \]

This applies to functions \(f\) in any number of variables. For example, \(\Delta f \approx f_w \Delta w\) if one of the variables \(w\) changes by \(\Delta w\) and all other variables remain fixed.

We can interpret the meaning of the partial derivatives \(f_x(a,b), f_y(a,b)\) geometrically in terms of the curves (or traces) which result from the intersection of the graph of \(f\) with the vertical planes through the point \((a,b,f(a,b))\) parallel to the \((x,z)\) and \((y,z)\) planes respectively, as depicted in the figure below.

It is important to understand that a definition of differentiability that requires only the existence of partial derivatives turns out to be insufficient. Many standard results, such as the chain rule for functions of several variables, would not follow, as Example 4 shows. Below, we shall see how to rectify this situation.

The Linear or Affine Approximation

To “motivate” our definition of differentiability, let us compute what the equation of the plane tangent to the graph of \(f\colon \,{\mathbb R}^2\rightarrow {\mathbb R}, (x,y)\mapsto f(x,y)\) at \((x_0,y_0)\) ought to be if \(f\) is smooth enough. In \({\mathbb R}^3\), a nonvertical plane has an equation of the form \[ z=ax+by+c. \]

If it is to be the plane tangent to the graph of \(f\), the slopes along the \(x\) and \(y\) axes must be equal to \(\partial f/\partial x\) and \(\partial f/\partial y\), the rates of change of \(f\) with respect to \(x\) and \(y\). Thus, \(a=\partial f/\partial x, b=\partial f/\partial y\) [evaluated at \((x_0,y_0)\)]. Finally, we may determine the constant \(c\) from the fact that \(z=f(x_0,y_0)\) when \(x=x_0,y=y_0\). Thus, we get the linear approximation (or, more accurately said, affine approximation): \begin{equation} z=f(x_0,y_0)+\bigg[\frac{\partial f}{\partial x}(x_0,y_0)\bigg](x-x_0)+ \bigg[\frac{\partial f}{\partial y}(x_0,y_0)\bigg](y-y_0), \end{equation} which should be the equation of the plane tangent to the graph of \(f\) at \((x_0,y_0)\), if \(f\) is “smooth enough” (see Figure 13.41).

Our definition of differentiability will mean in effect that the plane defined by the linear approximation (1) is a “good” approximation of \(f\) near \((x_0,y_0)\). To get an idea of what we might mean by a good approximation, let us return for a moment to one-variable calculus. If \(f\) is differentiable at a point \(x_0\), then we know that \[ \mathop {{\rm limit}\ }_{\Delta x \rightarrow 0} \frac{f(x_0+\Delta x)-f(x_0)}{\Delta x}=f'(x_0). \]

Let \(x=x_0+\Delta x\) and rewrite this as \[ \mathop {{\rm limit}\ }_{x \rightarrow x_0} \frac{f(x)-f(x_0)}{x-x_0}=f'(x_0). \]

Using the trivial limit \({\rm limit}_{x\rightarrow x_0}f'(x_0)=f'(x_0)\), we can rewrite the preceding equation as \[ \mathop {{\rm limit}\ }_{x \rightarrow x_0} \frac{f(x)-f(x_0)}{x-x_0}= \mathop {{\rm limit}\ }_{x\rightarrow x_0}f'(x_0); \] that is, \[ \mathop {{\rm limit}\ }_{x \rightarrow x_0} \bigg[\frac{f(x)-f(x_0)}{x-x_0}-f'(x_0)\bigg]=0; \] that is, \[ \mathop {{\rm limit}\ }_{x \rightarrow x_0} \frac{f(x)-f(x_0)-f'(x_0)(x-x_0)}{x-x_0}=0. \]

Thus, the tangent line \(l\) through \((x_0,f(x_0))\) with slope \(f'(x_0)\) is close to \(f\) in the sense that the difference between \(f(x)\) and \(l(x)=f(x_0)+f'(x_0)(x-x_0)\), the equation of the tangent line goes to zero even when divided by \(x -{x_0}\) as \(x\) goes to \(x_0\). This is the notion of a “good approximation” that we will adapt to functions of several variables, with the tangent line replaced by the tangent plane [see equation (1), given earlier].

Differentiability for Functions of Two Variables

Using the linear approximation, we are ready to define the notion of differentiability.

It is not always easy to use this definition to see whether \(f\) is differentiable, but it will be easy to use another criterion, given shortly in Theorem 9.

Tangent Plane

We have used the informal notion of the plane tangent to the graph of a function to motivate our definition of differentiability. Now we are ready to adopt a formal definition of the tangent plane.

Let us write \({\bf D} f(x_0,y_0)\) for the row matrix \[ \bigg[\frac{\partial f}{\partial x}(x_0,y_0)\quad \frac{\partial f}{\partial y} (x_0,y_0) \bigg], \] so that the definition of differentiability asserts that \begin{equation} \begin{array}{l} f(x_0,y_0)+{\bf D} f(x_0,y_0) \Big[ \begin{array}{c} x-x_0\\ y-y_0 \end{array} \Big]\\ \quad =f(x_0,y_0)+\bigg[ \frac{\partial f}{\partial x}(x_0,y_0) \bigg](x-x_0) + \bigg[\frac{\partial f}{\partial y} (x_0,y_0) \bigg](y-y_0) \end{array} \end{equation} is our good approximation to \(f\) near \((x_0,y_0)\). As earlier, “good” is taken in the sense that expression (3) differs from \(f(x,y)\) by something small times \(\sqrt{(x-x_0)^2 + (y-y_0)^2}\). We say that expression (3) is the best linear approximation to \(f\) near \((x_0,y_0)\).

Best linear approximation means that near \((x_0, y_0, f(x_0,y_0))\), the surface is closely approximated by the tangent plane; we sometimes say that \(f(x,y)\) is locally linear near \((x_0, y_0, f(x_0,y_0))\).

The next example shows again that the existence of the partial derivatives is not sufficient to guarantee the existence of a tangent plane as a linear approximation.

The Derivative and Differentiability: The General Case

Now we are ready to give a definition of differentiability for maps \(f\) of \({\mathbb R}^n\) to \({\mathbb R}^m\), using the preceding discussion as motivation. The derivative \({\bf D} f({\bf x}_0)\) of \(f=(f_1,\ldots ,f_m)\) at a point \({\bf x}_0\) is a matrix \({\bf T}\) whose elements are \(t_{ij}=\partial f_i/\partial x_j\) evaluated at \({\bf x}_0\).footnote #

We shall always denote the derivative \({\bf T}\) of \(f\) at \({\bf x}_0\) by \({\bf D}f({\bf x}_0)\), although in some books it is denoted \(df({\bf x}_0)\) and referred to as the differential of \(f\). In the case where \(m=1\), the matrix \({\bf T}\) is just the row matrix \[ \bigg[ \frac{\partial f}{\partial x_1}({\bf x}_0)\quad\cdots \quad \frac{\partial f}{\partial x_n}({\bf x}_0) \bigg] . \] (Sometimes, when there is danger of confusion, we separate the entries by commas.) Furthermore, setting \(n=2\) and putting the result back into equation (4), we see that conditions (2) and (4) do agree. Thus, if we let \({\bf h}={\bf x}-{\bf x}_0\), a real-valued function \(f\) of \(n\) variables is differentiable at a point \({\bf x}_0\) if \[ \mathop {{\rm limit}\ }_{{\bf h}\rightarrow 0}\frac{1}{ \| {\bf h} \| } \bigg|f({\bf x}_0+{\bf h})-f({\bf x}_0) -\sum^n_{j=1} \frac{\partial f}{\partial x_j}({\bf x}_0)h_j\bigg|=0, \] because \[ {\bf Th}=\sum^n_{j=1}h_j \frac{\partial f}{\partial x_j}({\bf x}_0). \]

For the general case of \(f\) mapping a subset of \({\mathbb R}^n\) to \({\mathbb R}^m\), the derivative is the \(m\times n\) matrix given by \[ {\bf D} f({\bf x}_0)=\left[ \begin{array}{c@{\quad}c@{\quad}c} \displaystyle \frac{\partial f_1}{\partial x_1} & \cdots & \displaystyle \frac{\partial f_1}{\partial x_n}\\ \vdots & &\vdots \\[3pt] \displaystyle \frac{\partial f_m}{\partial x_1} & \cdots & \displaystyle \frac{\partial f_m}{\partial x_n} \end{array}\right], \] where \(\partial f_i/\partial x_j\) is evaluated at \({\bf x}_0\). The matrix \({\bf D} f({\bf x}_0)\) is, appropriately, called the matrix of partial derivatives of \(f\) at \({\bf x}_0\).

Gradients

For real-valued functions we use special terminology for the derivative.

From the definition, we see that for \(f\colon\, {\mathbb R}^3\rightarrow {\mathbb R}\), \[ {\nabla} f=\frac{\partial f}{\partial x}{\bf i}+\frac{\partial f}{\partial y}{\bf j}+ \frac{\partial f}{\partial z}{\bf k}, \] while for \(f\colon\, {\mathbb R}^2\rightarrow {\mathbb R}\), \[ {\nabla} f=\frac{\partial f}{\partial x}{\bf i}+\frac{\partial f}{\partial y}{\bf j}. \]

The geometric significance of the gradient will be discussed in Section 13.5. In terms of inner products, we can write the derivative of \(f\) as \[ {\bf D} f({\bf x})({\bf h})={\nabla} f({\bf x})\,{ \cdot}\, {\bf h}. \]

In one-variable calculus it is shown that if \(f\) is differentiable, then \(f\) is continuous. We will state in Theorem 8 that this is also true for differentiable functions of several variables. As we know, there are plenty of functions of one variable that are continuous but not differentiable, such as \(f(x)=|x|\). Before stating the result, let us give an example of a function of two variables whose partial derivatives exist at a point, but which is not continuous at that point.

Some Basic Theorems

The first of these basic theorems relates differentiability and continuity.

This result is very reasonable, because “differentiability" means that there is enough smoothness to have a tangent plane, which is stronger than just being continuous. Consult the Internet supplement for Chapter 2 for the formal proof.

As we have seen, it is usually easy to tell when the partial derivatives of a function exist using what we know from one-variable calculus. However, the definition of differentiability looks somewhat complicated, and the required approximation condition in equation (4) may seem, and sometimes is, difficult to verify. Fortunately, there is a simple criterion, given in the following theorem, that tells us when a function is differentiable.

We give the proof in the Internet supplement for Chapter 2. Notice the following hierarchy:

Each converse statement, obtained by reversing an implication, is invalid. [For a counterexample to the converse of the first implication, use \(f(x)=x^2\sin\, (1/x),\) \(f(0)=0\); for the second, see Example 1 in the Internet supplement for Chapter 2 or use Example 4 in this section.]

A function whose partial derivatives exist and are continuous is said to be of class \(C^1\). Thus, Theorem 9 says that any \(C^1\) function is differentiable.

One can show that \(f(x,y)=xy/\sqrt{x^2+y^2}\) [with \(f(0,0)\,{=}\,0\)] is continuous, has partial derivatives at (0, 0), yet is not differentiable there. See Figure 13.43. By Theorem 9, its partial derivatives cannot be continuous at (0, 0).