A super important notion in linear algebra is that of duality. Put simply, duality is the following fact:

there is an isomorphism between vectors and linear functionals

The lay-man’s explanation of this is “the dual of a column vector is a row vector, sometimes also called the transpose”. I don’t think this really does the idea justice though.

The way I like to think about it is like this. Let’s say that we have a vector space \(V\), with some basis \(e_1,\ldots, e_n\). Then, the dual space \(V^*\) is the set of linear-functionals on \(V\). \(V^{*}\) has a basis \(\hat{e_1}, \ldots, \hat{e_n}\) which are the linear functionals that act as follows on vectors \(v=c_1 e_1 + c_2 e_2 + \cdots\): \[\hat{e_i}(v) = c_i.\]

It’s kinda interesting. Think about it a bit.

Let’s see this in action, to solve the following problem:

Theorem. \(Im A = Im AA^T\)

But actually this isn’t the real theorem. The real theorem is:

Theorem. Let \(A: \mathbb{R}^n \to \mathbb{R}^m\), and \(A^T : \mathbb{R}^m \to \mathbb{R}^n\). Then

\[\ker A^{T} \oplus Im A \cong \mathbb{R}^m\] \[\ker A \oplus Im A^T \cong \mathbb{R}^n.\]

and in-fact, \(\ker A^T\) is the orthogonal complement of \(Im A\); similarly \(\ker A\) is the orthogonal complement of \(Im A^T\).

This theorem immediately implies the other theorem via the following picture ink_img009

Proof. Say \(A^T y = 0\). Then \(y^T A = 0\), so \(y^T A x = 0\) for all \(x\), i.e. \(y \perp Ax\).