Linear Algebra (2DBI00) Summary Q4 2020
Class 1 Vectors and Matrices
A matrix is a rectangular table with numbers. A number is a 1 × 1 matrix. The size of a matrix is also called
the dimension of a matrix. A 2 × 3 matrix is also written as A є ℝ2x3. The notation of matrices is usually a
capital letter (A, B), that of a vector is a lower case letter or boldface (x, y) and constants are denoted with
lower case Greek letters (α, β). A vector is a matrix with just 1 column. Suppose a = and b = . The
2 2
inner product is a ⋅ b = a1b1 + a2b2. The length of a is ||a|| = √𝑎1 + 𝑎2 . The angle between a and b is
cos(a,b) = . Vectors a and b are orthogonal/perpendicular if a ⋅ b = 0. Trick: to find a vector
perpendicular to a, switch a1 and a2 from place and place a minus sign in front of one.
An application of matrices is recommender systems (Netflix) with movie ratings by predicting a value. This
can be done by comparing angles (how much alike) between vectors (ratings).
Matrices contain data points. Most of applied math/data science states that the data points are
columns and in probability and statistics, the data points are rows.
aij is the element of the ith row and the jth column of a matrix. See matrix A: a11 = 1, a12 = 2, a21 =
3, a22 = 4. xj is the jth element of the vector (in the jth row).
Adding and subtracting is simple: (A+B)ij = Aij + Bij and (A-B)ij = Aij – Bij. Example:
So you can only add/subtract matrices of the same sizes.
Scalar (= real number) multiplication is a real number times a matrix: (αA)ij = αAij. Ex.:
A matrix times a vector yields a new matrix. Let A be a m × n matrix and x
an n-vector. Then y = Ax is an m-vector. The 1st component is the inner
product of the 1st row of matrix with vector y1 = a11x1 + a12x2. Example:
A matrix-vector product is a linear combination (α1x1 + … +
αnxn) of columns of A. Example:
A matrix times a matrix gives a new matrix. Let A be a m × n
matrix and B a n × p matrix. Then C = AB is an m × p matrix. Ex.:
A matrix-matrix product can be seen as a number of matrix-vector products together. Useful trick: “leave
“Inner dimensions” have to agree (n must be equal). Result has outer dimensions m × p. space” for AB: [x x x]
A matrix-vector product (m × n matrix) is linear. A function is called linear if it satis- B so: [x x x]
fies two conditions: - A(x + y) = Ax + Ay for all x,y є - A(αy) = αAy for all y є and α є A I AB [x x] | [x x x]
Special matrices: The diagonal of a matrix are the elements a11, a22, …, ann.
Zero matrix (O2) Identity matrix I2 Diagonal matrix Upper triangular Lower triangular Symmetric
aij = 0 for all i,j 1’s on diagonal, rest 0 0’s out of diagonal aij = 0 if i > j aij = 0 if j > 1 aij = aji for all i,j
Zeros can be omitted for convenience: there are large matrices with lots of zeros which are called sparse.
For all A we have: A ∙ I = A, I ∙ A = A. For all A we have A ∙ O = O, O ∙ A = O.
Matrix-matrix multiplication is associative: (AB)C = A(BC), (αA)B = α(AB), (αA)x = α(Ax), A(Bx) = (AB)x.
Matrix-matrix multiplication is not commutative so in general AB ≠ BA. When AB = BA, B and A commute.
Multiplication with diagonal matrix: left-multiplication (in A ∙ B, A is a diagonal matrix)
works on the rows and right-multiplication works on columns. Example left-multi.:
The standard basis vectors has one 1 and for the rest only 0’s and are
denoted by ex , depending on the row the 1 is in. “e” is the vector with only ones.
Matrix times a standard basis vector: A ∙ jth standard basis vector = jth column of A. So Aej = Aj.
The transposed matrix AT is the matrix that changes rows in columns and columns in rows: (AT)ij = Aji. A
matrix for which AT = A is called symmetric. Rule for the transpose of a sum: (A + B)T = AT + BT. Rule for
the transpose of a product: (AB)T = BTAT. Thus (ABC)T = CTBTAT. Taking the transpose twice: (AT)T = A.
The length / norm of the vector x is ||x||2 = xTx. We cannot multiply 2 columns vectors, since the
dimensions do not match but we can multiply 2 column vectors if you transpose 1 (it is then a row vector).
1
By Isabel Rutten
, Powers of a matrix: A2 means A ∙ A. This is only possible when A is square. When adding two matrices and
taking the power of that, remember: (A + B)2 = A2 + AB + BA + B2 ≠ A2 + 2AB + B2.
A+AT is always symmetric but A must be square. ATA is always symmetric but A doesn’t need to be square.
Calculations on a computer can be done with Python, Julia, Matlab, R or Wolfram Alpha.
Class 2 Vector and matrix norms
The standard norm ||x||, the length of a vector, is called the 2-norm. Calculation: ↑.
This norm is derived from the standard inner product on , i.e. can be expressed in xTx.
A more general family of norms is the p-norms from which the 2-norm
can be derived. There is also the 1-norm and the ∞-norm:
and .
The different norms have different unit “circles”. 2-norm: circle. 1-norm: diamond. ∞-norm: square.
Vectors can be normalized/scaled such that a vector has p-norm = 1 by dividing the vector by its p-norm.
Vector norms have the following definition: A norm || ∙ || is a function with: 1. ||x|| ≥ 0;
2. ||x|| = 0 x = 0 (0-vector); 3. ||αx|| = |α| ||x|| if α є ; 4. ||x + y|| ≤ ||x|| + ||y|| (triangle inequality).
The matrix norms can be derived from the vector norms. If || ∙ || is a vector norm, then the derived matrix
norm is or, equivalently, since : . It turns out that:
=> Maximal => Maximal => No easy
…
column sum row sum formula
Usually, ||A|| means ||A||2 and ||x|| means ||x||2.
There are also matrix norms not derived for a vector norm, for example:
Matrix norms have the following definition: A norm || ∙ || is a function with: 1. ||A|| ≥ 0;
2. ||A|| = 0 A = 0 (0-vector); 3. ||αA|| = |α| ||A|| if α є ; 4. ||A + B|| ≤ ||A|| + ||B|| (triangle inequality).
From the definition follows an important property: ||Ax|| ≤ ||A|| ||x||. Applying this twice: ||AB|| ≤ ||A|| ||B||.
There is also the max-norm: .
2
By Isabel Rutten
Class 1 Vectors and Matrices
A matrix is a rectangular table with numbers. A number is a 1 × 1 matrix. The size of a matrix is also called
the dimension of a matrix. A 2 × 3 matrix is also written as A є ℝ2x3. The notation of matrices is usually a
capital letter (A, B), that of a vector is a lower case letter or boldface (x, y) and constants are denoted with
lower case Greek letters (α, β). A vector is a matrix with just 1 column. Suppose a = and b = . The
2 2
inner product is a ⋅ b = a1b1 + a2b2. The length of a is ||a|| = √𝑎1 + 𝑎2 . The angle between a and b is
cos(a,b) = . Vectors a and b are orthogonal/perpendicular if a ⋅ b = 0. Trick: to find a vector
perpendicular to a, switch a1 and a2 from place and place a minus sign in front of one.
An application of matrices is recommender systems (Netflix) with movie ratings by predicting a value. This
can be done by comparing angles (how much alike) between vectors (ratings).
Matrices contain data points. Most of applied math/data science states that the data points are
columns and in probability and statistics, the data points are rows.
aij is the element of the ith row and the jth column of a matrix. See matrix A: a11 = 1, a12 = 2, a21 =
3, a22 = 4. xj is the jth element of the vector (in the jth row).
Adding and subtracting is simple: (A+B)ij = Aij + Bij and (A-B)ij = Aij – Bij. Example:
So you can only add/subtract matrices of the same sizes.
Scalar (= real number) multiplication is a real number times a matrix: (αA)ij = αAij. Ex.:
A matrix times a vector yields a new matrix. Let A be a m × n matrix and x
an n-vector. Then y = Ax is an m-vector. The 1st component is the inner
product of the 1st row of matrix with vector y1 = a11x1 + a12x2. Example:
A matrix-vector product is a linear combination (α1x1 + … +
αnxn) of columns of A. Example:
A matrix times a matrix gives a new matrix. Let A be a m × n
matrix and B a n × p matrix. Then C = AB is an m × p matrix. Ex.:
A matrix-matrix product can be seen as a number of matrix-vector products together. Useful trick: “leave
“Inner dimensions” have to agree (n must be equal). Result has outer dimensions m × p. space” for AB: [x x x]
A matrix-vector product (m × n matrix) is linear. A function is called linear if it satis- B so: [x x x]
fies two conditions: - A(x + y) = Ax + Ay for all x,y є - A(αy) = αAy for all y є and α є A I AB [x x] | [x x x]
Special matrices: The diagonal of a matrix are the elements a11, a22, …, ann.
Zero matrix (O2) Identity matrix I2 Diagonal matrix Upper triangular Lower triangular Symmetric
aij = 0 for all i,j 1’s on diagonal, rest 0 0’s out of diagonal aij = 0 if i > j aij = 0 if j > 1 aij = aji for all i,j
Zeros can be omitted for convenience: there are large matrices with lots of zeros which are called sparse.
For all A we have: A ∙ I = A, I ∙ A = A. For all A we have A ∙ O = O, O ∙ A = O.
Matrix-matrix multiplication is associative: (AB)C = A(BC), (αA)B = α(AB), (αA)x = α(Ax), A(Bx) = (AB)x.
Matrix-matrix multiplication is not commutative so in general AB ≠ BA. When AB = BA, B and A commute.
Multiplication with diagonal matrix: left-multiplication (in A ∙ B, A is a diagonal matrix)
works on the rows and right-multiplication works on columns. Example left-multi.:
The standard basis vectors has one 1 and for the rest only 0’s and are
denoted by ex , depending on the row the 1 is in. “e” is the vector with only ones.
Matrix times a standard basis vector: A ∙ jth standard basis vector = jth column of A. So Aej = Aj.
The transposed matrix AT is the matrix that changes rows in columns and columns in rows: (AT)ij = Aji. A
matrix for which AT = A is called symmetric. Rule for the transpose of a sum: (A + B)T = AT + BT. Rule for
the transpose of a product: (AB)T = BTAT. Thus (ABC)T = CTBTAT. Taking the transpose twice: (AT)T = A.
The length / norm of the vector x is ||x||2 = xTx. We cannot multiply 2 columns vectors, since the
dimensions do not match but we can multiply 2 column vectors if you transpose 1 (it is then a row vector).
1
By Isabel Rutten
, Powers of a matrix: A2 means A ∙ A. This is only possible when A is square. When adding two matrices and
taking the power of that, remember: (A + B)2 = A2 + AB + BA + B2 ≠ A2 + 2AB + B2.
A+AT is always symmetric but A must be square. ATA is always symmetric but A doesn’t need to be square.
Calculations on a computer can be done with Python, Julia, Matlab, R or Wolfram Alpha.
Class 2 Vector and matrix norms
The standard norm ||x||, the length of a vector, is called the 2-norm. Calculation: ↑.
This norm is derived from the standard inner product on , i.e. can be expressed in xTx.
A more general family of norms is the p-norms from which the 2-norm
can be derived. There is also the 1-norm and the ∞-norm:
and .
The different norms have different unit “circles”. 2-norm: circle. 1-norm: diamond. ∞-norm: square.
Vectors can be normalized/scaled such that a vector has p-norm = 1 by dividing the vector by its p-norm.
Vector norms have the following definition: A norm || ∙ || is a function with: 1. ||x|| ≥ 0;
2. ||x|| = 0 x = 0 (0-vector); 3. ||αx|| = |α| ||x|| if α є ; 4. ||x + y|| ≤ ||x|| + ||y|| (triangle inequality).
The matrix norms can be derived from the vector norms. If || ∙ || is a vector norm, then the derived matrix
norm is or, equivalently, since : . It turns out that:
=> Maximal => Maximal => No easy
…
column sum row sum formula
Usually, ||A|| means ||A||2 and ||x|| means ||x||2.
There are also matrix norms not derived for a vector norm, for example:
Matrix norms have the following definition: A norm || ∙ || is a function with: 1. ||A|| ≥ 0;
2. ||A|| = 0 A = 0 (0-vector); 3. ||αA|| = |α| ||A|| if α є ; 4. ||A + B|| ≤ ||A|| + ||B|| (triangle inequality).
From the definition follows an important property: ||Ax|| ≤ ||A|| ||x||. Applying this twice: ||AB|| ≤ ||A|| ||B||.
There is also the max-norm: .
2
By Isabel Rutten
