Quadratic Forms

Definition

A quadratic form is one of the form $$Q(x) = x^TBx$$ Where B is an $n \times n$ matrix. Interestingly, we can turn B into a symmetric matrix by doing $A = \frac{1}{2}\left(B + B^T\right)$. A cool fact about this is $$Q(x) = x^TBx = x^TAx$$

Example

As we’ll see later, having this matrix be a symmetric matrix is very convenient.

$$\text{Let A} = \begin{bmatrix}5 & 0\\0& 2\end{bmatrix}$$ $$Q(x) = \begin{bmatrix}x & y\end{bmatrix}\begin{bmatrix}5 & 0\\0& 2\end{bmatrix}\begin{bmatrix}x \\ y\end{bmatrix} = 5x^2 + 2y^2$$

$$\text{Example 2: Write this in quadratic form } 5x^2 + 3y^2 + 2z^2 - xy + 8yz$$ Well we know that the diagonals are the square terms, and we can fill in the other terms, verifying by checking: $$A = \begin{bmatrix}5 & -\frac{1}{2} & 0\\-\frac{1}{2} & 3 & 4\\0 &4 & 2\end{bmatrix}$$

Types of Quadratic Forms

Here are some definitions:

Positive Semidefinite

For every point, $q(x) \geq 0$.

Positive Definite

For every non-zero point, $q(x) > 0$.
Note that if a quadratic form is positive definite, then it is also positive semidefinite, because q(0) is always 0.

Negative Semidefinite

For every point, $q(x) \leq 0$.

Negative Definite

For every non-zero point, $q(x) < 0$.

Indefinite

There exists a point x such that q(x) > 0. There exists a point y such that q(y) < 0

Eigenvalues

Let's say we have this equation $$Q(x) = x^TAx$$ We can diagonalize A meaning $ A = P^{-1}DP$. According to the spectral theorem, P has a special property. P is orthogonal, meaning we can rewrite $A = P^TDP$ $$Q(x) = x^TP^TDPx = (Px)^TD(Px)$$ If we let y = Px, we get this new expression $$y^TDy =\begin{bmatrix}y_1 & y_2 & ... & y_n\end{bmatrix}\begin{bmatrix}\lambda_1 & 0 & 0 & 0 \\ 0 & \lambda_2 & 0 & 0 \\ 0 & 0 & ... & 0 \\0 & 0 & 0 & \lambda_n\end{bmatrix}\begin{bmatrix}y_1 \\ y_2 \\ ... \\ y_n\end{bmatrix}$$ $$ = \lambda_1y_1^2 + \lambda_2y_2^2 + \lambda_3y_3^2 + ... + \lambda_ny_n^2 $$ The sign of these values does not depend on the value of $y_i$. It only depends on the eigenvalue.

Positive Definite

$$q(x) \gt 0 \text{ iff } \lambda_1 ... \lambda_n > 0$$

Negative Definite

$$q(x) \lt 0 \text{ iff } \lambda_1 ... \lambda_n < 0$$

Indefinite

$$q(x) \lt 0 \text{ iff } \lambda_i > 0 \text{ and } \lambda_j < 0$$

Semidefinite

$$q(x) \lt 0 \text{ iff } \lambda_i = 0 \text{ and all are non-negative or non-positive}$$