Notation / Names

Homogeneous: a function such that $g(\alpha x) = \alpha g(x)$ or some similar thing

Change of variables

If you have $f(x)$ and you wanted to express it as $\bar{f}(u)$, you need to express $x = g(u)$, and then $\bar{f}(u) = f(g(u))$

When do you want to change variables?

• If you have $x = \phi(z)$ and $f(x)$ is simple. This is the most common example

• If you have $x = \phi(z)$ and $f(z)$ is siple.

Derivatives (common)

• $\nabla_x b^TAx = A^Tb$. You can construct the proof directly. This shows up pretty often when you do Lagrangians

• $\nabla_x x^TAx = 2Ax$

• $\nabla_A x^TAx = xx^T$

Notation

• $(a, b, c)$ represents column vector

• $\succ$, $\succeq$ are generalized inequalities that are used for vectors (element-wise) and matrix inequalities

• $S^k$ is the set of symmetric $k\times k$ matrices

• $S_{++}$ as positive definite

Mappings

• The image of a function is what happens to $S$ after applying $f$

• The inverse image is the set of $S’$ such that $f(S’) = S$. As long as $f$ has the range of $S$, the infverse image always exists (even if $f$ is not invertible, because if $f(x_1) = f(x_2) \in S$, then $x_1, x_2 \in S’$.)

Understanding hyperplanes

Why is $a^T x = 0$ actually mean? Do we need to define an origin point?

Well, actually this type of hyperplane passes through the origin necessarily, and it forms the sheet that is perpendicular to $a$.

What about $a^T(x - x_0)$ actually mean? Well, if you imagine some vector $x_0$, This hyperplane passes through $x_0$ and is perpendicular to $a$. So that’s all you need to know!