Trace

This is just the sum of the diagonal

Quick Properties

Why do we care about this operation?

• Linear mapping, which means that all linear properties hold

• $tr(A) = \sum \lambda_i$, i.e. the trace is the sum of the eigenvalues
  • Proof: $tr(A) = tr(U\Sigma V^T) = tr(V^TU\Sigma) = tr(\Sigma) = \sum\lambda_i$

• $tr(A^T) = tr(A)$

• $tr(AB) = tr(BA)$ (the inner product definition really helps)

• Trace of matrix product is just the sum of their hadarmard product.

• Trace of outer product is the same as inner product (think about this) $tr(ba^T) = a^Tb$.
  • useful for turning norms into potentially covariances (if you’re dealing with expectation)

• In general, the trace is invariant under circular shifts (not permutations though)

Derivative through a trace

The trace derivative have a few identities to know. These can all be derived from the summation definition of a trace