Linear Algebra

Linear algebra is the language behind regression, multivariate analysis and most methods in statistics. This poster brings together the basic theory and the formulas used most often.

Matrices and Operations

• Addition and scalar multiplication: element by element
• Matrix product: defined when the columns of the first equal the rows of the second, generally not commutative
• Transpose: ${(AB)}^T=B^T{A}^T$
• Inverse: exists only for square matrices with a non-zero determinant, ${(AB)}^{-1}=B^{-1}{A}^{-1}$
• Trace: sum of the diagonal elements, $\mathrm{tr}(AB)=\mathrm{tr}(BA)$

Determinants

• 2×2 matrix: $ad-bc$
• Laplace expansion: along a row or column for larger matrices
• Properties: $\det (AB)=\det (A)\det (B)$, $\det (A^T)=\det (A)$
• Singular matrix: determinant equal to zero, not invertible

Linear Systems and Rank

• Form $Ax=b$: solved by Gaussian elimination or Cramer’s rule
• Rank: number of linearly independent rows or columns
• Consistency: the system has a solution when the rank of $A$ equals the rank of the augmented matrix

Eigenvalues and Eigenvectors

• Definition: $Av=\lambda v$, with $v\ne 0$
• Characteristic equation: $\det (A-\lambda I)=0$
• Sum and product: the sum of the eigenvalues equals the trace, the product equals the determinant
• Symmetric matrices: real eigenvalues and orthogonal eigenvectors
• Positive definite matrix: all eigenvalues positive, like the covariance matrix