linear algebra cheat sheet, Cheat Sheet of Linear
Algebra
Linear algebra explained in four pages
Excerpt from the GUIDE TO LINEAR ALGEBRA
Abstract—This document will review the fundamental ideas of linear algebra. B. Matrix operations
We will learn about matrices, matrix operations, linear transformations and
discuss both the theoretical and computational aspects of linear algebra. The We denote by A the matrix as a whole and refer to its entries as aij .
tools of linear algebra open the gateway to the study of more advanced The mathematical operations defined for matrices are the following:
mathematics. A lot of knowledge buzz awaits you if you choose to follow the • addition (denoted +)
path of understanding, instead of trying to memorize a bunch of formulas.
C = A+B ⇔ cij = aij + bij .
I. INTRODUCTION
• subtraction (the inverse of addition)
Linear algebra is the math of vectors and matrices. Let n be a positive • matrix product. The product of matrices A ∈ Rm×n and B ∈ Rn×Æ
integer and let R denote the set of real numbers, then Rn is the set of all is another matrix C ∈ Rm×Æ given by the formula
n-tuples of real numbers. A vector →v∈ Rn is an n-tuple of real numbers. n
X
The notation “∈ S” is read “element of S.” For example, consider a vector C = AB ⇔ cij = aikbkj,
that has three components: k=1
2 3 2 3
→v = (v1, v2, v3) ∈ (R, R, R) ≡ R .
3 a11 a12 » – a11b11 + a12b21 a11b12 + a12b22
4a21 b11 b12
a225 = 4a21b11 + a22b21 a21b12 + a22b22 5
A matrix A ∈ Rm×n is a rectangular array of real numbers with m rows b21 b22
a31 a32 a31b11 + a32b21 a31b12 + a32b22
and n columns. For example, a 3 × 2 matrix looks like this:
2 3 2 3 • matrix inverse (denoted A−1)
a11 a12 R R • matrix transpose (denoted T):
3×2
A= 4 a21 a22 5 ∈ 4 R R 5≡R .
» 2 3
a31 a32 R R α1 α2 –T α1 β1
α3
The purpose of this document is to introduce you to the mathematical = 4α2 β25 .
β1 β2 β3
operations that we can perform on vectors and matrices and to give you a α3 β3
feel of the power of linear algebra. Many problems in science, business, Pn
• matrix trace: Tr[A] ≡ i=1 aii
and technology can be described in terms of vectors and matrices so it is
• determinant (denoted det(A) or |A|)
important that you understand how to work with these.
Note that the matrix product is not a commutative operation: AB /= BA.
Prerequisites
The only prerequisite for this tutorial is a basic understanding of high school C. Matrix-vector product
math concepts1 like numbers, variables, equations, and the fundamental The matrix-vector product is an important special case of the matrix-
arithmetic operations on real numbers: addition (denoted +), subtraction matrix product. The product of a 3 × 2 matrix A and the 2 × 1 column
(denoted −), multiplication (denoted implicitly), and division (fractions). vector →x results in a 3 × 1 vector2→y given3 by: 2 3
You should also be familiar with functions that take real numbers as 2 3 a11 a12 » – a11x1 + a12x2
y1
inputs and give real numbers as outputs, f : R → R. Recall that, by 4y25=4a21 a225 x1 = 4a21x1 + a22x25
definition, the inverse function f−1 undoes the effect of f . If you are →y = A→x ⇔
y3 a31 a32 x2
given f (x) and you want to find x, you can use the inverse function as 2 3 2 3 a31x1 + a32x2
a11 a12
follows: f−1 (f (x)) = x. For example, the function f (x) = ln(x) has the
√ = x1 a21 +x2 a225
4 5 4 (C)
inverse f−1(x) = ex, and the inverse of g(x) = x is g−1(x) = x2.
2 a31 a32
3
II. DEFINITIONS (a11, a12) · →x
A. Vector operations = 4(a21, a22) · →x5. (R)
We now define the math operations for vectors. The operations we can in terms of the angle θ between the two vectors. The formula for the dot
perform on vectors →u = (u1, u2, u3) and →v = (v1, v2, v3) are: addition, product of the vectors is →u · →v = →u →v cos θ. We say two vectors →u and
subtraction, scaling, norm (length), dot product, and cross product:
→u + →v = (u1 + v1, u2 + v2, u3 + v3)
→u − →v = (u1 − v1, u2 − v2, u3 − v3)
α→u = (αu1, αu2, αu3)
q
||→u|| = u12 + u22 + u23
→u · →v = u1v1 + u2v2 + u3v3
→u × →v =(u2v3 − u3v2, u3v1 − u1v3, u1v2 − u2v1)
The dot product and the cross product of two vectors can also be described
, (a31, a32) · →x →x produces a column vector with coefficients equal to the dot products
of rows of the matrix with the vector →x.
There are two2 fundamentally different yet equivalent ways to interpret the
matrix-vector product. In the column picture, (C), the multiplication of the
matrix A by the vector →x produces a linear combination of the columns D. Linear transformations
of the matrix: →y = A→x = x1A[:,1] + x2A[:,2], where A[:,1] and A[:,2] The matrix-vector product is used to define the notion of a linear
are the first and second columns of the matrix A. transformation, which is one of the key notions in the study of linear
In the row picture, (R), multiplication of the matrix A by the vector algebra. Multiplication by a matrix A ∈ Rm×n can be thought of as
→v are orthogonal if the angle between them is 90◦. The dot product of computing a linear transformation TA that takes n-vectors as inputs and
orthogonal vectors is zero: →u · →v = →u →v cos(90◦) = 0. produces m-vectors as outputs:
The norm of the cross product is given by →u × →v = →u →v sin θ. The
n m
cross product is not commutative: →u × →v /= →v × →u, in fact →u × →v = −→v × →u. TA : R → R .
2For
1A good textbook to (re)learn high school math is minireference.com more info see the video of Prof. Str ang’s MIT lecture: bit.ly/10vmKcL
1
Algebra
Linear algebra explained in four pages
Excerpt from the GUIDE TO LINEAR ALGEBRA
Abstract—This document will review the fundamental ideas of linear algebra. B. Matrix operations
We will learn about matrices, matrix operations, linear transformations and
discuss both the theoretical and computational aspects of linear algebra. The We denote by A the matrix as a whole and refer to its entries as aij .
tools of linear algebra open the gateway to the study of more advanced The mathematical operations defined for matrices are the following:
mathematics. A lot of knowledge buzz awaits you if you choose to follow the • addition (denoted +)
path of understanding, instead of trying to memorize a bunch of formulas.
C = A+B ⇔ cij = aij + bij .
I. INTRODUCTION
• subtraction (the inverse of addition)
Linear algebra is the math of vectors and matrices. Let n be a positive • matrix product. The product of matrices A ∈ Rm×n and B ∈ Rn×Æ
integer and let R denote the set of real numbers, then Rn is the set of all is another matrix C ∈ Rm×Æ given by the formula
n-tuples of real numbers. A vector →v∈ Rn is an n-tuple of real numbers. n
X
The notation “∈ S” is read “element of S.” For example, consider a vector C = AB ⇔ cij = aikbkj,
that has three components: k=1
2 3 2 3
→v = (v1, v2, v3) ∈ (R, R, R) ≡ R .
3 a11 a12 » – a11b11 + a12b21 a11b12 + a12b22
4a21 b11 b12
a225 = 4a21b11 + a22b21 a21b12 + a22b22 5
A matrix A ∈ Rm×n is a rectangular array of real numbers with m rows b21 b22
a31 a32 a31b11 + a32b21 a31b12 + a32b22
and n columns. For example, a 3 × 2 matrix looks like this:
2 3 2 3 • matrix inverse (denoted A−1)
a11 a12 R R • matrix transpose (denoted T):
3×2
A= 4 a21 a22 5 ∈ 4 R R 5≡R .
» 2 3
a31 a32 R R α1 α2 –T α1 β1
α3
The purpose of this document is to introduce you to the mathematical = 4α2 β25 .
β1 β2 β3
operations that we can perform on vectors and matrices and to give you a α3 β3
feel of the power of linear algebra. Many problems in science, business, Pn
• matrix trace: Tr[A] ≡ i=1 aii
and technology can be described in terms of vectors and matrices so it is
• determinant (denoted det(A) or |A|)
important that you understand how to work with these.
Note that the matrix product is not a commutative operation: AB /= BA.
Prerequisites
The only prerequisite for this tutorial is a basic understanding of high school C. Matrix-vector product
math concepts1 like numbers, variables, equations, and the fundamental The matrix-vector product is an important special case of the matrix-
arithmetic operations on real numbers: addition (denoted +), subtraction matrix product. The product of a 3 × 2 matrix A and the 2 × 1 column
(denoted −), multiplication (denoted implicitly), and division (fractions). vector →x results in a 3 × 1 vector2→y given3 by: 2 3
You should also be familiar with functions that take real numbers as 2 3 a11 a12 » – a11x1 + a12x2
y1
inputs and give real numbers as outputs, f : R → R. Recall that, by 4y25=4a21 a225 x1 = 4a21x1 + a22x25
definition, the inverse function f−1 undoes the effect of f . If you are →y = A→x ⇔
y3 a31 a32 x2
given f (x) and you want to find x, you can use the inverse function as 2 3 2 3 a31x1 + a32x2
a11 a12
follows: f−1 (f (x)) = x. For example, the function f (x) = ln(x) has the
√ = x1 a21 +x2 a225
4 5 4 (C)
inverse f−1(x) = ex, and the inverse of g(x) = x is g−1(x) = x2.
2 a31 a32
3
II. DEFINITIONS (a11, a12) · →x
A. Vector operations = 4(a21, a22) · →x5. (R)
We now define the math operations for vectors. The operations we can in terms of the angle θ between the two vectors. The formula for the dot
perform on vectors →u = (u1, u2, u3) and →v = (v1, v2, v3) are: addition, product of the vectors is →u · →v = →u →v cos θ. We say two vectors →u and
subtraction, scaling, norm (length), dot product, and cross product:
→u + →v = (u1 + v1, u2 + v2, u3 + v3)
→u − →v = (u1 − v1, u2 − v2, u3 − v3)
α→u = (αu1, αu2, αu3)
q
||→u|| = u12 + u22 + u23
→u · →v = u1v1 + u2v2 + u3v3
→u × →v =(u2v3 − u3v2, u3v1 − u1v3, u1v2 − u2v1)
The dot product and the cross product of two vectors can also be described
, (a31, a32) · →x →x produces a column vector with coefficients equal to the dot products
of rows of the matrix with the vector →x.
There are two2 fundamentally different yet equivalent ways to interpret the
matrix-vector product. In the column picture, (C), the multiplication of the
matrix A by the vector →x produces a linear combination of the columns D. Linear transformations
of the matrix: →y = A→x = x1A[:,1] + x2A[:,2], where A[:,1] and A[:,2] The matrix-vector product is used to define the notion of a linear
are the first and second columns of the matrix A. transformation, which is one of the key notions in the study of linear
In the row picture, (R), multiplication of the matrix A by the vector algebra. Multiplication by a matrix A ∈ Rm×n can be thought of as
→v are orthogonal if the angle between them is 90◦. The dot product of computing a linear transformation TA that takes n-vectors as inputs and
orthogonal vectors is zero: →u · →v = →u →v cos(90◦) = 0. produces m-vectors as outputs:
The norm of the cross product is given by →u × →v = →u →v sin θ. The
n m
cross product is not commutative: →u × →v /= →v × →u, in fact →u × →v = −→v × →u. TA : R → R .
2For
1A good textbook to (re)learn high school math is minireference.com more info see the video of Prof. Str ang’s MIT lecture: bit.ly/10vmKcL
1
