* **Square matrix:** If the matrix has same number of columns as rows

* **Symmetric matrix:** A square matrix that is equal to its transpose. $latex \mathbf{A} = \mathbf{A}^T

* **Diagonal matrix:** A matrix where all non-diagonal entries are zero $latex $\mathbf{A}_{i,j} = 0, \forall i \ne j$. Square diagonal matrices are denoted as $latex \text{diag}(v)$. But note that diagonal matrices are not required to be square.

* **Identity matrix:** A diagonal matrix where all diagonal entries are 1

* **Inverse matrix:** An inverse of a matrix is another matrix such that their product is an identity matrix. $latex \mathbf{A}^{-1}\mathbf{A} = \mathbf{A}\mathbf{A}^{-1} = 1$

* **Inverse of diagonal matrix:** Is easy to compute: $latex \text{diag}(v)^{-1} = \text{diag}([1/v_1,\ldots,1/v_n])^T$

* **Unit vector:** A vector with unit norm $latex ||\mathbf{x}\||_2 = 1$

* **Orthogonal vectors:** If $latex \mathbf{x}^T\mathbf{y} = 0$, then $latex \mathbf{x}$ and $latex \mathbf{y}$ are considered orthogonal. If their norms are non-zero, then it is the case that the angle between them is $latex 90^{\circ}$. Why? Because a product of vectors can be represented as $latex \mathbf{x}^T\mathbf{y} = |\mathbf{x}||\mathbf{y}|\cos\theta$, where $latex \theta$ is the angle between the two vectors.

* **Orthonormal vectors:** If two orthogonal vectors are also unit vectors, then they are orthonormal

* **Orthogonal Matrix:** A square matrix whose rows are mutually orthornomal and the columns are mutually orthonormal. Therefore $latex \mathbf{A}^T\mathbf{A} = \mathbf{A}\mathbf{A}^T = \mathbf{I}$

* **Positive definite Matrix:** A square symmetric matrix $latex \mathbf{A}$ such that for any non-zero vector $latex \mathbf{x}$, $latex \mathbf{x}^T\mathbf{A}\mathbf{x} > 0$. It is sometimes denoted as $latex \mathbf{A} > 0$

* **Positive semi-definite Matrix:** A square symmetric matrix $latex \mathbf{A}$ such that for any non-zero vector $latex \mathbf{x}$, $latex \mathbf{x}^T\mathbf{A}\mathbf{x} \ge 0$. It is sometimes denoted as $latex \mathbf{A} \ge 0$.