Linear Algebra Study Guide

From systems of equations to eigenvalues. Everything you need for your university linear algebra course, whether it's MATA22, MATH 136, MATH 221, or any intro linear algebra class.

1. Systems of Linear Equations

This is where every linear algebra course starts. A system of linear equations is a collection of equations like:

Augmented Matrix Form

Write the system as an augmented matrix [A | b] and use row operations to solve:

Row Reduction (Gaussian Elimination)

The three elementary row operations:

• Swap two rows (R_i ↔ R_j)
• Scale a row by a nonzero constant (kR_i → R_i)
• Replace a row by itself plus a multiple of another (R_i + kR_j → R_i)

Goal: get the matrix into row echelon form (REF) or reduced row echelon form (RREF).

Solution Types

Situation	Meaning	Example (RREF)
Pivot in every column of A	Unique solution	[1 0 | 3] [0 1 | 5]
Free variables (non-pivot columns)	Infinitely many solutions	[1 2 | 3] [0 0 | 0]
Row like [0 0 ... 0 | k], k ≠ 0	No solution (inconsistent)	[1 0 | 2] [0 0 | 5]

2. Vectors and Vector Operations

Vectors are ordered lists of numbers. In R^n, a vector has n components.

Key Operations

Operation	Formula	Result Type
Addition	u + v = (u1+v1, u2+v2, ...)	Vector
Scalar multiplication	cu = (cu1, cu2, ...)	Vector
Dot product	u · v = u1v1 + u2v2 + ...	Scalar
Cross product (R^3 only)	u × v = (u2v3-u3v2, u3v1-u1v3, u1v2-u2v1)	Vector
Length (norm)	||u|| = sqrt(u · u)	Scalar

Linear Combinations

A linear combination of vectors v1, v2, ..., vk is any expression of the form:

where c1, c2, ..., ck are scalars. This is the single most important concept in linear algebra - almost everything else builds on it.

Span

The span of a set of vectors is the set of all their linear combinations. If vectors v1, ..., vk span R^n, then every vector in R^n can be written as a linear combination of them.

3. Matrices and Matrix Operations

A matrix is a rectangular array of numbers. An m×n matrix has m rows and n columns.

Matrix Multiplication

For AB to be defined, the number of columns of A must equal the number of rows of B.

Special Matrices

Type	Property	Notation
Identity	AI = IA = A for all A	I or I_n
Diagonal	All off-diagonal entries are 0	diag(d1, d2, ...)
Symmetric	A = A^T	-
Triangular (upper)	All entries below diagonal are 0	-
Triangular (lower)	All entries above diagonal are 0	-
Inverse	AA^(-1) = A^(-1)A = I	A^(-1)

Finding the Inverse

To find A^(-1), augment A with I and row reduce:

If you can't reduce A to I (you get a row of zeros on the left), then A is not invertible (singular).

Properties of Inverses

4. Determinants

The determinant is a single number computed from a square matrix. It tells you whether the matrix is invertible and how it scales volume.

2×2 Determinant

3×3 and Beyond: Cofactor Expansion

Expand along any row or column. For a 3×3 matrix expanding along row 1:

Key Determinant Properties

• det(A) = 0 if and only if A is singular (not invertible)
• det(AB) = det(A) · det(B)
• det(A^T) = det(A)
• det(cA) = c^n · det(A) for an n×n matrix
• Row swap flips the sign of the determinant
• Row scaling by k multiplies the determinant by k
• Row replacement (R_i + kR_j → R_i) doesn't change the determinant

5. Vector Spaces and Subspaces

A vector space is a set V with addition and scalar multiplication that satisfies 10 axioms (closure, associativity, commutativity, identity, inverses, etc.). The main examples:

• R^n - the space of all n-tuples of real numbers
• M_{m×n} - the space of all m×n matrices
• P_n - the space of all polynomials of degree ≤ n
• C[a,b] - the space of all continuous functions on [a,b]

Subspaces

A subspace of V is a subset W that is itself a vector space. To verify W is a subspace, check three things:

1. Zero vector: 0 ∈ W
2. Closed under addition: if u, w ∈ W, then u + w ∈ W
3. Closed under scalar multiplication: if w ∈ W and c is a scalar, then cw ∈ W

Four Fundamental Subspaces

For any m×n matrix A:

Subspace	Definition	Dimension
Column space Col(A)	{Ax : x ∈ R^n} - span of columns	rank(A) = r
Row space Row(A)	span of rows = Col(A^T)	rank(A) = r
Null space Nul(A)	{x : Ax = 0}	n - r (nullity)
Left null space Nul(A^T)	{y : A^T y = 0}	m - r

6. Linear Independence, Basis, and Dimension

Linear Independence

Vectors v1, v2, ..., vk are linearly independent if the only solution to:

is c1 = c2 = ... = ck = 0. If any other solution exists, the vectors are linearly dependent - meaning at least one vector is a linear combination of the others.

How to Check Independence

Form a matrix with the vectors as columns. Row reduce. If every column has a pivot, they're independent. If any column lacks a pivot, they're dependent.

Basis

A basis for a vector space V is a set of vectors that is:

1. Linearly independent
2. Spans V (every vector in V is a linear combination of the basis vectors)

Every basis for the same space has the same number of vectors - this number is the dimension of the space.

Finding a Basis for Col(A)

Row reduce A. The columns of the original matrix A that correspond to pivot columns in the RREF form a basis for Col(A). Don't use the RREF columns themselves.

Finding a Basis for Nul(A)

Solve Ax = 0 by row reducing [A | 0]. Write the general solution in parametric vector form. The vectors multiplied by the free variables form a basis for Nul(A).

7. Linear Transformations

A function T: R^n → R^m is a linear transformation if:

1. T(u + v) = T(u) + T(v) for all u, v
2. T(cu) = cT(u) for all u and scalars c

Every linear transformation from R^n to R^m can be represented as multiplication by an m×n matrix A: T(x) = Ax.

Standard Matrix

To find the matrix for a transformation T:

Apply T to each standard basis vector and make those the columns.

Common Geometric Transformations in R^2

Transformation	Matrix
Rotation by θ	[cos θ -sin θ; sin θ cos θ]
Reflection across x-axis	[1 0; 0 -1]
Reflection across y-axis	[-1 0; 0 1]
Reflection across y = x	[0 1; 1 0]
Scaling by k	[k 0; 0 k]
Projection onto x-axis	[1 0; 0 0]

Key Properties

• Kernel (null space) of T: all vectors x where T(x) = 0. If kernel = {0}, T is one-to-one.
• Range (column space) of the matrix: all vectors that T can produce. If range = R^m, T is onto.
• Composition: T2 ∘ T1 corresponds to matrix product B·A (right to left).

8. Eigenvalues and Eigenvectors

An eigenvector of a matrix A is a nonzero vector v such that:

The scalar λ is the eigenvalue. Geometrically: A doesn't change the direction of v, only scales it by λ.

Finding Eigenvalues

1. Write the characteristic equation: det(A - λI) = 0
2. Expand and solve the polynomial for λ

Finding Eigenvectors

For each eigenvalue λ:

1. Compute A - λI
2. Row reduce A - λI
3. Solve (A - λI)x = 0
4. The nonzero solutions are the eigenvectors for λ

Key Properties

• The eigenspace for λ is Nul(A - λI) - it's always a subspace
• trace(A) = sum of eigenvalues (with multiplicity)
• det(A) = product of eigenvalues
• If A is triangular, the eigenvalues are the diagonal entries
• Eigenvectors for distinct eigenvalues are always linearly independent

Algebraic vs Geometric Multiplicity

Type	Definition	Notation
Algebraic multiplicity	Power of (λ - λ_i) in characteristic polynomial	a_i
Geometric multiplicity	dim(eigenspace) = dim(Nul(A - λI))	g_i

Always: 1 ≤ g_i ≤ a_i. If g_i = a_i for every eigenvalue, the matrix is diagonalizable.

9. Orthogonality and Least Squares

Orthogonal Vectors

Vectors u and v are orthogonal if u · v = 0. An orthogonal set is a set of pairwise orthogonal nonzero vectors. An orthonormal set is an orthogonal set where every vector has length 1.

Gram-Schmidt Process

Convert any basis {x1, x2, ..., xk} into an orthogonal basis {v1, v2, ..., vk}:

To get an orthonormal basis, normalize each vector: u_i = v_i / ||v_i||.

Orthogonal Projection

The projection of vector y onto a subspace W with orthogonal basis {u1, ..., uk}:

Least Squares

When Ax = b has no solution (inconsistent system), the least squares solution minimizes ||Ax - b||:

Solve this system (the "normal equations") for x̂. This is the foundation of linear regression.

10. Diagonalization

A matrix A is diagonalizable if there exists an invertible matrix P and a diagonal matrix D such that:

When is a Matrix Diagonalizable?

• A is diagonalizable if and only if it has n linearly independent eigenvectors
• Sufficient condition: n distinct eigenvalues (always diagonalizable)
• If eigenvalue has algebraic multiplicity > 1: diagonalizable only if geometric = algebraic multiplicity
• Real symmetric matrices are always diagonalizable (with an orthogonal P)

Why Diagonalize?

Diagonalization makes matrix powers trivial:

This is essential for solving systems of differential equations, computing Markov chain steady states, and analyzing dynamical systems.

11. Common Mistakes

1. Forgetting AB ≠ BA. Matrix multiplication order matters. T2 ∘ T1 = B·A, not A·B.
2. Using RREF columns for Col(A) basis. The basis for Col(A) comes from the original matrix A, using the pivot column positions from RREF.
3. Row reducing the augmented matrix for det. Don't augment - determinant only applies to square matrices. And track sign changes from row swaps.
4. det(A + B) = det(A) + det(B). This is false. Determinant is multiplicative (det(AB) = det(A)det(B)), not additive.
5. Confusing eigenvalues of A and A^(-1). If λ is an eigenvalue of A, then 1/λ is an eigenvalue of A^(-1) (same eigenvector).
6. Declaring "no solution" when there's a free variable. Free variables mean infinitely many solutions, not no solution. No solution only occurs with an inconsistent row.
7. Writing 0 as an eigenvector. Eigenvectors must be nonzero. The eigenspace includes 0 but the eigenvectors don't.
8. Forgetting to check subspace conditions. To prove W is a subspace, you need all three conditions (zero vector, closure under addition, closure under scalar multiplication).
9. Mixing up rank and nullity. Rank = number of pivot columns. Nullity = number of free variables = n - rank. They add up to n (columns), not m (rows).
10. Gram-Schmidt order errors. Subtract projections onto ALL previous vectors, not just the first one. Each step removes the component along every vector computed so far.

12. Exam Strategy

Before the Exam

• Build a "toolkit" sheet: determinant formulas, rank-nullity theorem, eigenvalue procedure, Gram-Schmidt steps, diagonalization checklist, inverse method
• Practice row reduction until it's automatic - most problems reduce to row reduction at some point
• Know the big equivalences: A is invertible ↔ det(A) ≠ 0 ↔ rank(A) = n ↔ Nul(A) = {0} ↔ columns are independent ↔ rows are independent ↔ 0 is not an eigenvalue
• Do full past exams under time pressure - linear algebra exams are usually time-constrained

During the Exam

• Read the whole exam first. Start with problems you know how to do. Save the tricky proofs for last.
• Check dimensions. If you're multiplying matrices, verify the dimensions work out. If you're finding eigenvectors of a 3×3 matrix, you should get vectors in R^3.
• Use the Invertible Matrix Theorem. If any part of a problem establishes that A is invertible (det ≠ 0, full rank, independent columns, etc.), you get ALL the equivalent conditions for free.
• Verify by substitution. Found an eigenvalue/eigenvector? Multiply Av and check that it equals λv. Found a basis for Nul(A)? Multiply Ax and check you get 0.
• Partial credit on proofs. State what you'd need to show, write the definitions, set up the structure - even if you can't finish.

13. Frequently Asked Questions