Multivariable Derivative

Geometric Definition of a Derivative

Before we get to the total derivative, what does the single-variable derivative mean? Well it’s a linear approximation at a given point. Now, imagine when we add a second variable, it’s no longer an approximation with a line, it’s a plane. Generalizing to higher dimensions, the derivative is a hyperplane approximation of a function.

Mathematical Definition

The derivative $Df(p)$ is a linear transformation from $\mathbb{R}^n \to \mathbb{R}^m$ that satisfies this equation $$\lim_{h \to 0} \dfrac{f(p + h) - f(p) - Df(p)(h)}{||h||} = 0$$ This is also called the total derivative. The equatino above doesn't help us figure out what it is at all. In order to start getting some intuition into what this mysterious function could be, let's look at an example.

Total Derivative of a Linear Transformation

Let's start by writing out the definition $$\lim_{h \to 0} \dfrac{||T(p + h) - T(p) - DT(p)(h)||}{||h||} = 0$$ $$\lim_{h \to 0} \dfrac{||T(p) + T(h) - T(p) - DT(p)(h)||}{||h||} = 0$$ $$\lim_{h \to 0} \dfrac{||T(h) - DT(p)(h)||}{||h||} = 0$$ Because we know that both the derivative and T are linear transformations, we can cosnider their matrix definition: $T(x) = [T]x$. $$\lim_{h \to 0} \dfrac{||T*h - DT(p)*h||}{||h||} = \lim_{h \to 0} ||T - DT(p)|| = 0$$ If $DT(p) = T$, then the limit definition is satisfied. Therefore, the derivative of a linear transformation is the linear transformation itself.


Let's look at an example. What's the derivative of $2x + 3y + 4z$? Well let's look at it like a matrix $$\begin{bmatrix}2 & 3 & 4\end{bmatrix}\begin{bmatrix}x\\y\\z\end{bmatrix}$$ The derivative is $\begin{bmatrix}2 & 3 & 4\end{bmatrix}$.

Jacobian

The Jacobian matrix is a matrix whose columns consist of partial derivatives. $$\begin{bmatrix}\dfrac{\partial f}{\partial x_1} & \dfrac{\partial f}{\partial x_2} & ... & \dfrac{\partial f}{\partial x_n}\end{bmatrix}$$ So, the Jacobian of $2x + 3y + 4z$ is $\begin{bmatrix}2 & 3 & 4\end{bmatrix}$. Interestingly, that's also the derivative. Is this always the case? Yes! The derivative of a function equals its Jacobian.

Chain Rule

We begin with this fact of derivatives. $$D(f \circ g)(p) = J(f \circ g)(p) = Jf(g(p))Jg(p)$$ Now, let's put this into practice! Let $z = xe^{xy} \begin{cases}x = t^2\\ y = \frac{1}{t}\end{cases}$. Find the $\dfrac{dz}{dt}$. Think of this as finding the derivative of $z(x(t), y(t))$. We begin by finding $Jz(x(p), y(p))$. Now, $x(p)$ and $y(p)$ are just x and y. That's it. $$\begin{bmatrix}\left(1 + xy\right)e^{xy} & x^2e^{xy}\end{bmatrix}$$ Now, we look at $x(p)$, $y(p)$. $$\begin{bmatrix}x_t \\ y_t\end{bmatrix} = \begin{bmatrix}2t\\\frac{1}{t^2}\end{bmatrix}$$ Why is this one column? x and y are technically part of the same function $f(t) = < x(t), y(t)>$. We are only taking partials of one variable, so it's one column. Now, let's multiply the values: $$2t(1 + xy)e^{xy} - \frac{x^2e^{xy}}{t^2}$$ Now, in order to use one variable, we can substitute x(t) and y(t) with their values. $$2t(1 + t)e^{t} - t^2e^{t} = 2te^t + t^2e^t$$