Theorem 10 Sums, Products, Quotients

• (i) Constant Multiple Rule. Let \(f\colon\, U \subset {\mathbb R}^n \rightarrow {\mathbb R}^m\) be differentiable at \({\bf x}_0\) and let \(c\) be a real number. Then \(h ({\bf x}) = cf ( {\bf x})\) is differentiable at \({\bf x}_0\) and \[ {\bf D} h ( {\bf x}_0) = c {\bf D} f ( {\bf x}_0) \qquad \hbox{(equality of matrices)}. \]
• (ii) Sum Rule. Let \(f\colon\, U \subset {\mathbb R}^n \rightarrow {\mathbb R}^m\) and \(g\colon\, U \subset {\mathbb R}^n \rightarrow {\mathbb R}^m\) be differentiable at \({\bf x}_0\). Then \(h ( {\bf x}) = f ( {\bf x})+ g ( {\bf x})\) is differentiable at \({\bf x}_0\) and \[ {\bf D} h ( {\bf x}_0) = {\bf D} f ( {\bf x}_0) + {\bf D} g ( {\bf x}_0) \qquad \hbox{(sum of matrices).} \]
• (iii) Product Rule. Let \(f\colon\, U \subset {\mathbb R}^n \rightarrow {\mathbb R}\) and \(g\colon\, U \subset {\mathbb R}^n \rightarrow {\mathbb R}\) be differentiable at \({\bf x}_0\) and let \(h ( {\bf x}) = g ({\bf x}) f ({\bf x})\). Then \(h\colon\, U \subset {\mathbb R}^n \rightarrow {\mathbb R}\) is differentiable at \({\bf x}_0\) and \[ {\bf D} h ( {\bf x}_0 ) = g ( {\bf x}_0) {\bf D}f ( {\bf x}_0)+ f ( {\bf x}_0) {\bf D} g ( {\bf x}_0). \] (Note that each side of this equation is a \(1\times n\) matrix; a more general product rule is presented in Exercise \(31\) at the end of this section.)
• (iv) Quotient Rule. With the same hypotheses as in rule (iii), let \(h ( {\bf x}) = f ({\bf x}) / g ( {\bf x})\) and suppose \(g\) is never zero on \(U\). Then \(h\) is differentiable at \({\bf x}_0\) and \[ {\bf D} h ( {\bf x}_0) = \frac{ g ( {\bf x}_0) {\bf D} f ( {\bf x}_0) - f ( {\bf x}_0) {\bf D} g ( {\bf x}_0)}{ [ g ( {\bf x}_0)]^2} . \]

proof

The proofs of rules (i) through (iv) proceed almost exactly as in the one-variable case, with a slight difference in notation. We shall prove rules (i) and (ii), leaving the proofs of rules (iii) and (iv) as Exercise 27.

• (i) To show that \({\bf D} h ( {\bf x}_0) = c {\bf D}f ({\bf x}_0)\), we must show that \[ {\mathop {{\rm limit} }_{{\bf x} \to {\bf x}_0}} \ \frac{ \| h ( {\bf x} ) - h ( {\bf x}_0) - c {\bf D} f ( {\bf x}_0) ( {\bf x}- {\bf x}_0) \| }{ \| {\bf x} - {\bf x}_0 \| } =0, \] that is, that \[ {\mathop {{\rm limit} }_{{\bf x} \to {\bf x}_0}} \ \frac{ \| cf( {\bf x}) - cf( {\bf x}_0) - c {\bf D} f ( {\bf x}_0) ( {\bf x}- {\bf x}_0) \| }{ \| {\bf x} - {\bf x}_0 \| } =0, \] [see equation (4) of Section 13.3]. This is certainly true, since \(f\) is differentiable and the constant \(c\) can be factored out [see Theorem 3(i), Section 13.2].
• (ii) By the triangle inequality, we may write \begin{eqnarray*} &&\frac{ \| h ({\bf x}) - h ( {\bf x}_0) - [{\bf D} f ( {\bf x}_0) + {\bf D} g ({\bf x}_0) ] ( {\bf x}- {\bf x}_0) \| }{ \| {\bf x} - {\bf x}_0 \|} \\[3pt] && =\frac{ \| f({\bf x})-f({\bf x}_0)-[{\bf D} f({\bf x}_0)]({\bf x}-{\bf x}_0)+g({\bf x})-g({\bf x}_0)-[{\bf D}g({\bf x}_0)]({\bf x}-{\bf x}_0) \|}{\| {\bf x}-{\bf x}_0 \| } \\[3pt] && \leq \frac{ \| f({\bf x})-f({\bf x}_0)-[{\bf D} f({\bf x}_0)]({\bf x}-{\bf x}_0) \| }{ \| {\bf x}-{\bf x}_0 \| }+ \frac{ \| g({\bf x})-g({\bf x}_0)-[{\bf D}g({\bf x}_0)] ({\bf x}-{\bf x}_0) \| }{ \| {\bf x}-{\bf x}_0 \| }, \end{eqnarray*} and each term approaches 0 as \({\bf x}\rightarrow {\bf x}_0\). Hence, rule (ii) holds.

example 1

Verify the formula for \({\bf D}h\) in rule (iv) of Theorem 10 with \[ f(x,y,z)=x^2+y^2+z^2 \hbox{ and } g(x,y,z)=x^2+1. \]

solution Here \[ h(x,y,z)=\frac{x^2+y^2+z^2}{x^2+1}, \] so that by direct differentiation \begin{eqnarray*} {\bf D}h(x,y,z) &\!=\!& \bigg[\frac{\partial h}{\partial x},\frac{\partial h}{\partial y}, \frac{\partial h}{\partial z}\bigg] \!=\!\bigg[\frac{(x^2+1)2x-(x^2+y^2+z^2)2x}{(x^2+1)^2}, \frac{2y}{x^2+1},\frac{2z}{x^2+1}\bigg]\\[6pt] &=&\!\bigg[\frac{2x(1-y^2-z^2)}{(x^2+1)^2},\frac{2y}{x^2+1},\frac{2z}{x^2+1}\bigg]. \end{eqnarray*}

By rule (iv), we get \[ {\bf D}h=\frac{g{\bf D} f-f{\bf D}g}{g^2}= \frac{(x^2+1)[2x,2y,2z]-(x^2+y^2+z^2)[2x,0,0]}{(x^2+1)^2}, \] which is the same as what we obtained directly.

Theorem 11 Chain Rule

Let \(U\subset {\mathbb R}^n\) and \(V \subset {\mathbb R}^m\) be open sets. Let \(g\colon\, U\subset {\mathbb R}^n\rightarrow {\mathbb R}^m\) and \(f\colon\, V \subset {\mathbb R}^m\rightarrow {\mathbb R}^p\) be given functions such that \(g\) maps \(U\) into \(V\), so that \(f \circ g\) is defined. Suppose \(g\) is differentiable at \({\bf x}_0\) and \(f\) is differentiable at \({\bf y}_0=g({\bf x}_0)\). Then \(f \circ g\) is differentiable at \({\bf x}_0\) and \begin{equation*} {\bf D}(f\circ g)({\bf x}_0)={\bf D} f({\bf y}_0){\bf D}g({\bf x}_0).\tag{1} \end{equation*}

The right-hand side is the matrix product of \({\bf D} f({\bf y}_0)\) with \({\bf D} g({\bf x}_0)\).