Overview
Lecture 1: Linear equations
Lecture 2: Vector equations and spans
Lecture 3: Matrix-vector product and solution sets
Lecture 4: Linear independence and transformations
Lecture 5: Matrix of linear transformations
Lecture 6: Matrix operations
Lecture 7: Inverse matrices
Lecture 8: Subspaces
Lecture 9: Coordinates and LU factorization
Lecture 10: Determinants part 1
Lecture 11: Determinants part 2
Lecture 12: Eigenvalues and Eigenvectors
Lecture 13: Characteristics equation and similarity
Lecture 14: Diagonalization
Lecture 15: Change of basis
Lecture 16: Complex eigenvalues
Lecture 17: Inner product and orthogonality
Lecture 18: Orthogonal set, basis and matrix
Lecture 19: Orthogonal projections
Lecture 20: The Gram-Schmidt process
Lecture 21: Least-squares problems
,Lecture 1: Linear equations
§ Material: 1.1 and 1.2 (up to “Solutions of linear systems”)
§ Determine the augmented matrix associated with a system of linear equations.
§ Row reduce augmented matrices to (row) echelon and reduced (row) echelon form.
1.1 System of linear equations
Linear equation: an equation that can be written in the form 𝑎! 𝑥! + 𝑎" 𝑥" + ⋯ + 𝑎# 𝑥# = 𝑏.
System of linear equations (linear system): a collection of one or more linear equations
involving the same variables.
Solution of a linear system: a list of numbers (𝑠! , 𝑠" , … , 𝑠# ) that makes each equation a true
statement when the values 𝑠! , … , 𝑠# are substituted for 𝑥! , … , 𝑥# .
Solution set of a linear system: the set of all possible solutions of the linear system.
§ Two linear systems are equivalent if they have the same solution set.
A system of linear equations has no solution, exactly one solution, or infinitely many solutions.
Consistent system: a system of linear equations that has either one solution or infinitely many.
Inconsistent system: a system of linear equations that has no solution.
𝒎 × 𝒏 matrix: a rectangular array of numbers with 𝑚 rows and 𝑛 columns.
Coefficient matrix: a matrix with the coefficients of each variable aligned in columns.
Augmented matrix: a coefficient matrix with an added column containing the constants from the
right sides of the equations.
Basic strategy in solving a linear system: replace one system with an equivalent system (i.e. one
with the same solution set) that is easier to solve. For this we use 3 basic operations:
1. Replace one equation by the sum of itself and a multiple of another equation.
2. Interchange two equations.
3. Multiple all the terms in an equation by a nonzero constant.
Two matrices are called row equivalent if there is a sequence of elementary row operations that
transform one matrix into the other. If the augmented matrices of two linear systems are row
equivalent, then the two systems have the same solution set.
Two fundamental questions about a linear system.
1. Is the system consistent; that is, does at least one solution exist?
2. If a solution exists, is it the only one; that is, is the solution unique?
1.2 Row reduction and echelon forms
A rectangular matrix is in (row) echelon form (REF) if it has the following three properties:
1. All nonzero rows are above any rows of all zeros.
2. Each leading entry of a row is in a column to the right of the leading entry of the above row.
3. All entries in a column below a leading entry are zeros.
If a matrix in echelon form satisfies the following additional conditions, then it is in reduced (row)
echelon form (RREF):
4. The leading entry in each nonzero row is 1.
5. Each leading 1 is the only nonzero entry in its column.
Any nonzero matrix may be row reduced into more than one matrix in echelon form. However,
each matrix is row equivalent to one and only one reduced echelon matrix.
§ If a matrix 𝐴 is row equivalent to an echelon matrix 𝐵, we say: 𝐵 is an echelon form of 𝐴.
§ If a matrix 𝐴 is row equivalent to a reduced echelon matrix 𝐵, we say: 𝐵 is the reduced
echelon form of 𝐴.
A pivot position in a matrix 𝐴 is a location in 𝐴 that corresponds to a leading 1 in the reduced
echelon form of 𝐴. A pivot column is a column of 𝐴 that contains a pivot position. A pivot is a
nonzero number in a pivot position.
1
, The row reduction algorithm:
Forward phase: step 1-4, backward phase: step 5.
§ Step 1: Begin with leftmost nonzero column. A pivot column. The pivot position is at the top.
§ Step 2: Select a nonzero entry in the pivot column as a pivot. If necessary, interchange
rows to move this entry into the pivot position.
§ Step 3: Use row replacement operations to create zeros in all positions below the pivot.
§ Step 4: Cover (or ignore) the row containing the pivot position and cover all rows, if any,
above it. Apply steps 1-3 to the submatrix that remains. Repeat the process until there are
no more nonzero rows to modify.
§ Step 5: Beginning with the rightmost pivot and working upward and to the left, create zeros
above each pivot. If a pivot is not 1, make it 1 by a scaling operation.
The variables corresponding to pivot columns in the matrix are called basic variables. The other
variables are called free variables. Whenever a system is consistent, the solution set can be
described explicitly by solving the reduced system of equations for the basic variables in terms of
the free variables. Now, each different choice of the free variables determines a different solution
of the system, and every solution of the system is determined by a choice of the free variables.
Lecture 2: Vector equations and spans
§ Material: 1.2 (starting from “Solutions of linear systems”) and 1.3
§ Perform calculations with vectors.
§ Solve vector equations.
§ Determine whether a system of linear equations has one, zero or infinitely many solutions.
§ Describe the span of vector in ℝ$ geometrically, to verify whether a vector is contained in a
span and to verify whether a span of vectors in ℝ# equals ℝ# .
1.2 Row reduction and echelon forms (continue)
Solving a system amounts to finding a parametric description of the solution set or determining
that the solution set is empty (so no parametric representation). Whenever a system is consistent
and has free variables, the solution set has many parametric descriptions. We make the
convention of always using the free variables as the parameters for describing a solution set.
Solving a linear system:
1. Get to the echelon form using elementary row operations.
2. a) use backward substitution.
b) obtain a reduced echelon form (preferable).
From the echelon form we can answer the two fundamental questions about a linear system
(existence and uniqueness of a solution).
Existence and Uniqueness Theorem: a linear system is consistent if and only if the rightmost
column of the augmented matrix is not a pivot column – that is, if and only if an echelon form of the
augmented matrix has no row of the form [0 … 0 | 𝑏 ] (with 𝑏 nonzero).
If a linear system is consistent, then the solution set contains:
§ A unique solution: when there are no free variables.
§ Infinitely many solutions: when there is at least one free variable.
Row reduction to solve a linear system:
1. Write augmented matrix and use row reduction algorithm to obtain echelon form.
2. Decide whether the system is consistent, if so continue.
3. Continue row reduction to obtain the reduced echelon form.
4. Write the system of equations corresponding to the matrix obtained in step 3
5. Rewrite each nonzero equation so that its one basic variable is expressed in terms of any
free variables appearing in the equation.
2
Lecture 1: Linear equations
Lecture 2: Vector equations and spans
Lecture 3: Matrix-vector product and solution sets
Lecture 4: Linear independence and transformations
Lecture 5: Matrix of linear transformations
Lecture 6: Matrix operations
Lecture 7: Inverse matrices
Lecture 8: Subspaces
Lecture 9: Coordinates and LU factorization
Lecture 10: Determinants part 1
Lecture 11: Determinants part 2
Lecture 12: Eigenvalues and Eigenvectors
Lecture 13: Characteristics equation and similarity
Lecture 14: Diagonalization
Lecture 15: Change of basis
Lecture 16: Complex eigenvalues
Lecture 17: Inner product and orthogonality
Lecture 18: Orthogonal set, basis and matrix
Lecture 19: Orthogonal projections
Lecture 20: The Gram-Schmidt process
Lecture 21: Least-squares problems
,Lecture 1: Linear equations
§ Material: 1.1 and 1.2 (up to “Solutions of linear systems”)
§ Determine the augmented matrix associated with a system of linear equations.
§ Row reduce augmented matrices to (row) echelon and reduced (row) echelon form.
1.1 System of linear equations
Linear equation: an equation that can be written in the form 𝑎! 𝑥! + 𝑎" 𝑥" + ⋯ + 𝑎# 𝑥# = 𝑏.
System of linear equations (linear system): a collection of one or more linear equations
involving the same variables.
Solution of a linear system: a list of numbers (𝑠! , 𝑠" , … , 𝑠# ) that makes each equation a true
statement when the values 𝑠! , … , 𝑠# are substituted for 𝑥! , … , 𝑥# .
Solution set of a linear system: the set of all possible solutions of the linear system.
§ Two linear systems are equivalent if they have the same solution set.
A system of linear equations has no solution, exactly one solution, or infinitely many solutions.
Consistent system: a system of linear equations that has either one solution or infinitely many.
Inconsistent system: a system of linear equations that has no solution.
𝒎 × 𝒏 matrix: a rectangular array of numbers with 𝑚 rows and 𝑛 columns.
Coefficient matrix: a matrix with the coefficients of each variable aligned in columns.
Augmented matrix: a coefficient matrix with an added column containing the constants from the
right sides of the equations.
Basic strategy in solving a linear system: replace one system with an equivalent system (i.e. one
with the same solution set) that is easier to solve. For this we use 3 basic operations:
1. Replace one equation by the sum of itself and a multiple of another equation.
2. Interchange two equations.
3. Multiple all the terms in an equation by a nonzero constant.
Two matrices are called row equivalent if there is a sequence of elementary row operations that
transform one matrix into the other. If the augmented matrices of two linear systems are row
equivalent, then the two systems have the same solution set.
Two fundamental questions about a linear system.
1. Is the system consistent; that is, does at least one solution exist?
2. If a solution exists, is it the only one; that is, is the solution unique?
1.2 Row reduction and echelon forms
A rectangular matrix is in (row) echelon form (REF) if it has the following three properties:
1. All nonzero rows are above any rows of all zeros.
2. Each leading entry of a row is in a column to the right of the leading entry of the above row.
3. All entries in a column below a leading entry are zeros.
If a matrix in echelon form satisfies the following additional conditions, then it is in reduced (row)
echelon form (RREF):
4. The leading entry in each nonzero row is 1.
5. Each leading 1 is the only nonzero entry in its column.
Any nonzero matrix may be row reduced into more than one matrix in echelon form. However,
each matrix is row equivalent to one and only one reduced echelon matrix.
§ If a matrix 𝐴 is row equivalent to an echelon matrix 𝐵, we say: 𝐵 is an echelon form of 𝐴.
§ If a matrix 𝐴 is row equivalent to a reduced echelon matrix 𝐵, we say: 𝐵 is the reduced
echelon form of 𝐴.
A pivot position in a matrix 𝐴 is a location in 𝐴 that corresponds to a leading 1 in the reduced
echelon form of 𝐴. A pivot column is a column of 𝐴 that contains a pivot position. A pivot is a
nonzero number in a pivot position.
1
, The row reduction algorithm:
Forward phase: step 1-4, backward phase: step 5.
§ Step 1: Begin with leftmost nonzero column. A pivot column. The pivot position is at the top.
§ Step 2: Select a nonzero entry in the pivot column as a pivot. If necessary, interchange
rows to move this entry into the pivot position.
§ Step 3: Use row replacement operations to create zeros in all positions below the pivot.
§ Step 4: Cover (or ignore) the row containing the pivot position and cover all rows, if any,
above it. Apply steps 1-3 to the submatrix that remains. Repeat the process until there are
no more nonzero rows to modify.
§ Step 5: Beginning with the rightmost pivot and working upward and to the left, create zeros
above each pivot. If a pivot is not 1, make it 1 by a scaling operation.
The variables corresponding to pivot columns in the matrix are called basic variables. The other
variables are called free variables. Whenever a system is consistent, the solution set can be
described explicitly by solving the reduced system of equations for the basic variables in terms of
the free variables. Now, each different choice of the free variables determines a different solution
of the system, and every solution of the system is determined by a choice of the free variables.
Lecture 2: Vector equations and spans
§ Material: 1.2 (starting from “Solutions of linear systems”) and 1.3
§ Perform calculations with vectors.
§ Solve vector equations.
§ Determine whether a system of linear equations has one, zero or infinitely many solutions.
§ Describe the span of vector in ℝ$ geometrically, to verify whether a vector is contained in a
span and to verify whether a span of vectors in ℝ# equals ℝ# .
1.2 Row reduction and echelon forms (continue)
Solving a system amounts to finding a parametric description of the solution set or determining
that the solution set is empty (so no parametric representation). Whenever a system is consistent
and has free variables, the solution set has many parametric descriptions. We make the
convention of always using the free variables as the parameters for describing a solution set.
Solving a linear system:
1. Get to the echelon form using elementary row operations.
2. a) use backward substitution.
b) obtain a reduced echelon form (preferable).
From the echelon form we can answer the two fundamental questions about a linear system
(existence and uniqueness of a solution).
Existence and Uniqueness Theorem: a linear system is consistent if and only if the rightmost
column of the augmented matrix is not a pivot column – that is, if and only if an echelon form of the
augmented matrix has no row of the form [0 … 0 | 𝑏 ] (with 𝑏 nonzero).
If a linear system is consistent, then the solution set contains:
§ A unique solution: when there are no free variables.
§ Infinitely many solutions: when there is at least one free variable.
Row reduction to solve a linear system:
1. Write augmented matrix and use row reduction algorithm to obtain echelon form.
2. Decide whether the system is consistent, if so continue.
3. Continue row reduction to obtain the reduced echelon form.
4. Write the system of equations corresponding to the matrix obtained in step 3
5. Rewrite each nonzero equation so that its one basic variable is expressed in terms of any
free variables appearing in the equation.
2
