Solutions to Chapter 1
√ √
1.1 |a • b| = | − 9| = 9, while kak kbk = 6 22 ∼
= 11.489 > 9.
1.2 To verify the Cauchy–Schwarz inequality, first see that the inequality holds trivially if
a and b are zero vectors. We therefore assume that both a and b are nonzero. Let c be
the vector c = xa − yb, where x = b0 b, and y = a0 b. Clearly, c0 c ≥ 0. We express c0 c
in terms of x and y:
c0 c = (xa − yb)0 (xa − yb) = x2 a0 a − 2xya0 b + y 2 b0 b.
Since c0 c ≥ 0, and using the definitions of x and y, we see that
(b0 b)2 (a0 a) − 2(a0 b)2 (b0 b) + (a0 b)2 (b0 b) ≥ 0
and dividing by b0 b in the last inequality, we see that
(b0 b)(a0 a) − (a0 b)2 ≥ 0,
which verifies the Cauchy–Schwarz inequality.
We use the Cauchy–Schwarz inequality to deduce the triangle inequality, which can be
written in an equivalent form
ka + bk2 ≤ (kak + kbk)2 .
The expression on the left is
ka + bk2 = (a + b) • (a + b) = a • a + 2a • b + b • b
= kak2 + 2a • b + kbk2
while the expression on the right is
(kak + kbk)2 = kak2 + 2kakkbk + kbk2 .
Comparing these two formulas, we see that the triangle inequality holds if and only
if a • b ≤ kak kbk. By Cauchy–Schwarz inequality, |a • b| ≤ kakkbk, so the triangle
inequality follows as a consequence of the Cauchy–Schwarz inequality. The converse is
also true; i.e., if the triangle inequality holds, then a • b ≤ kakkbk holds for a and for −a,
from which Cauchy–Schwarz inequality follows. If equality holds, i.e., if a • b = kakkbk,
then b = ca, for some scalar c. Hence, a • b = ckak2 , and kakkbk = |c|kak2 . For nonnull
a, this implies that c = |c|, so that c ≥ 0. If b 6= 0, then b = ca, with c > 0.
1.3 Since x0 y = 0 = x0 z = y0 z, it follows that we must
√ solve the equations
√ a2 + b2 = 1, and
2 2
a − b = 0, for which the solutions are a = ±1/ 2 and b = ±1/ 2.
1.4 Let
1 2 0
1 0 −2
V=
0
.
1 1
−1 −1 1
1
,2 Solutions to Chapter 1
Using elementary column transformations C2 − 2C1 , and then, C3 − C2 , the matrix V
becomes
1 0 0
1 −2 0
V= 0
,
1 0
−1 −1 0
so that the column rank of V (or the dimension of its column space) is 2 < 3. Therefore,
v1 , v2 and v3 are linearly dependent. It is easily seen that v1 and v2 are LIN, and that
v3 = v2 − 2v1 .
1.5 It is easy to see that c1 v1 + c2 v2 + c3 v3 = 0 results in the following three equations,
and the only solution is c1 = 0, c2 = 0, and c3 = 0:
2c1 + 8c2 − 4c3 = 0
3c1 − 6c2 + 3c3 = 0
2c1 + 5c2 + c3 = 0.
1.6 Using elementary transformations, we can show that A is equivalent to the matrix
−3 3 3
0 4 4 ,
0 0 −1
so that the columns of A are LIN.
2 1 3
1.7 We can solve the system = c1 + c2 to get c1 = −1 and c2 = 1. Hence, u
3 2 5
is in Span{v1 , v2 }.
· · · , vm are linearly dependent, then there are scalars c1 , · · · , cm not
1.8 If v1 ,P Pall zero, such
m
that i=1 c i vi = 0. For any k with ck 6
= 0, we then have v k = − i6=k (ci /ck )vi ,
showing property 1 in Result 1.2.2.
Suppose without loss of generality that Pvs 1 , · · · , vs are linearly dependent and c1 , · · · , cs
are constants, not all zero, such that P i=1 ci vi = 0. Let cj = 0 for all j = s + 1, · · · , n.
n
Then c1 , · · · , cn are not all zero and i=1 ci vi = 0. Hence v1 , · · · , vn are linearly
dependent, showing property 2 in Result 1.2.2.
1.9 1. Let S = {v1 , · · · , vn } denote a set of nonzero orthogonal vectors, and let u belong to
the span of S with
u = c1 v1 + · · · + cn vn .
For a fixed i = 1, · · · , n, take the inner product of each side with vi . Since vi • vj = 0,
i 6= j,
u • vi = c1 (v1 • vi ) + · · · + cn (vn • vi ) = ci (vi • vi ).
Hence, ci = (u • vi ) (vi • vi ). To verify linear independence, set u = c1 v1 +· · ·+cn vn = 0.
This implies that ci = 0, i = 1, · · · , n, which in turn implies LIN of {v1 , · · · , vn }.
, Solutions to Chapter 1 3
2. By definition, every v ∈ V1 + · · · + Vm can be written as v1 + · · · + vm , where vi ∈ Vi ,
i = 1, . . . , m. Let wi ∈ Vi , i = 1, . . . , m, such that we also have v = w1 + · · · + wm . Then
(w1 − v1 ) + · · · + (wm − vm ) = 0. For each i, pre-multiply both sides by (wi − vi )0 . For
j 6= i, since Vi ⊥ Vj , then (wi − vi )0 (wj − vj ) = 0. As a result, (wi − vi )0 (wi − vi ) =
kwi − vi k2 = 0, giving wi = vi . Hence by definition, the sum of the Vi ’s is a direct sum.
1.10 Since {v1 , · · · , vm } is a basis of V, the vectors are LIN. So from
k−1
X yi0 vk
y1 = v1 , yk = vk − yi , k = 2, · · · , m,
i=1
kyi k2
yk
yk 6= 0, implying that zk = kyk k are well-defined and each has length 1. On the other
hand, for 1 ≤ j < k ≤ m,
k−1 j−1 k−1
X yi0 vk 0 X yi0 vk 0 X y 0 vk
yj0 yk = yj0 vk − y y i = − y yj − i
y0 y .
2 j i
i=1
kyi k2 j i=1
ky i k2 i
i=j+1
ky i k
If k = 2, then j = 1, and it is straightforward to see that y10 y2 = 0. Suppose we have
shown that for all i < j < k, yi0 yj = 0. Then the above identity shows that for all
j < k, yj0 yk = 0. By induction, the yi ’s are orthogonal to each other. Then z1 , · · · , zm
are orthonormal. Since they are LIN from Exercise 1.9, and there are m of them, they
form an orthonormal basis of V.
1.11 We have W∩(W ⊥ ∩V) ⊂ W∩W ⊥ = {0}. From W ⊂ V and W ⊥ ∩V ⊂ V, W⊕(W ⊥ ∩V) ⊂
V. On the other hand, for any v ∈ V, there are unique w ∈ W and u ∈ W ⊥ , such that
v = w + u. Since w ∈ V, then u = v − w ∈ V, and so u ∈ W ⊥ ∩ V. As a result
V ⊂ W ⊕ (W ⊥ ∩ V). Then V = W ⊕ (W ⊥ ∩ V). From the paragraph below Definition
1.2.8, dim V = dim W + dim(W ⊥ ∩ V), completing the proof of property 1. Next, by
W ⊥ ∩V ⊂ W ⊥ , (W ⊥ ∩V)⊥ ∩V ⊃ (W ⊥ )⊥ ∩V = W ∩V = W. On the other hand, if v ∈ V
and v ⊥ (W ⊥ ∩ V), then by property 1, there are unique w ∈ W and u ∈ W ⊥ ∩ V such
that v = w+u. From assumption, v ⊥ u. Meanwhile w ⊥ u. Then u0 u = u0 (v−w) = 0,
so u = 0. Then v = w ∈ W. As a result, (W ⊥ ∩V)⊥ ∩V ⊂ W. Then (W ⊥ ∩V)⊥ ∩V = W,
showing property 2.
1.12 In order for a matrix A = {aij } i, j = 1, 2, 3, to commute with the matrix B, we require
that AB = BA. Computing the product on both sides, and equating them, we see that
the conditions are a11 = a22 = a33 , a12 = a23 , and a21 = a31 = a32 = 0.
1.13 By repeated multiplication, we see that
k
k a β
A = ,
0 1
where, β = b(1 + a + · · · + ak−1 ).
1.14 It is easily verified that the product (A − B)C is equal to Ak − Bk , where, C =
Ak−1 + Ak−2 B + · · · + ABk−2 + Bk−1 .
1.15 Clearly, (A0 A)0 = A0 (A0 )0 = A0 A, and (AA0 )0 = (A0 )0 A0 = AA0 .
1.16 If A = O, then clearly A0 A = O. To show the converse, let the column vectors of A be
a1 , · · · , an . Since A0 A = {a0i aj }, if A0 A = O, then for all i = 1, · · · , n, a0i ai = 0, giving
ai = 0, and so A = O.
, 4 Solutions to Chapter 1
1.17 (a) From Result 1.3.5,
k
X k
X k
X
tr(A xi x0i ) = tr( Axi x0i ) = tr(Axi x0i )
i=1 i=1 i=1
k
X k
X
= tr(x0i Axi ) = (xi Axi ).
i=1 i=1
(b) From property 4 of Result 1.3.5, it follows that
tr(B−1 AB) = tr(BB−1 A) = tr(A).
1.18 Suppose A = {aij } and B = {bij } are both lower triangular matrices and P aii = ai and
n
bii = bi . Then aij = bij = 0 for all i < j. Now C = AB = {cij } with cij = k=1 aik bkj .
If i > j, then for every k = 1, · · · , n, either i > k or k > j, so aik bkj = 0, giving
cij = 0. Then C is lower triangular. If i = j, then aik bkj 6= 0 if and only if k = i, so
cii = aii bii = an bn . This completes the proof for the lower triangular case. The proof
for the upper triangular case is similar.
Pm Pn
1.19 Properties 1, 3, 5, and 6, of Result 1.3.5 are obvious. From tr(AB) = i=1 j=1 aij bji =
Pn Pm Pn Pm
j=1 i=1 aij bji = j=1 i=1 bji aij = tr(BA), Property 2 follows. Property 4 is a
direct consequence, since we can regard ABC as the product of AB and C, or of A and
BC. Both tr(AA0 ) and tr(A0 A) equal the sum of squares of the elements of A, which is
nonnegative, and shows property 7, of which property 8 is an immediate consequence.
1.20 If A = {akl } is lower triangular, then akl = 0 for all k < l. Now Mij = {mkl }, where
akl if k < i, l < j
a
k+1,l if k ≥ i, l < j
mkl =
ak,l+1 if k < i, l ≥ j
ak+1,l+1 if k ≥ i, l ≥ j.
If i > j, then for k < l, the second case in the display is not possible, and all the other
cases are possible, each one giving value 0. Therefore, mkl = 0, so that Mij is lower
triangular. Furthermore, for any k = j, · · · , i − 1, mkk = ak,k+1 = 0, so Mij has at least
one diagonal element equal to zero. This verifies the first fact.
From (1.3.1), the ith diagonal element of A−1 is |Mii |/|A|. If the diagonal elements of A
are d1 , · · · , dn , then the diagonal elements of Mii are d1 , · · · , di−1 , di+1 , · · · , dn . There-
fore |Mii | = d1 · · · di−1 di+1 · · · dn and |A| = d1 · · · dn . Then the ith diagonal element of
A−1 is 1/di .
1.21 Using Definition 1.3.7, |A| = −8.
1.22 It is easy to verify the given resursive expression, from which we obtain ∆n = (1 + a2 +
a4 + · · · + a2n ) = [1 − a2(n+1) ]/[1 − a4 ].
1.23 This follows directly from property 7 of Result 1.3.6.
Pn
1.24 Using Definition 1.2.7, we can show that the value of this determinant is 1 + i=1 ai .
1.25 The determinant to be calculated is |(1 − ρ)In + ρJn |. If ρ 6= 1, then from 12 of Result
1.3.6, the determinant is (1 − ρ)n |In + ρ(1 − ρ)−1 1n 10n | = (1 − ρ)n [1 + ρ(1 − ρ)−1 10n 1n ] =
(1 − ρ)n [1 + nρ(1 − ρ)−1 ] = (1 − ρ)n−1 [1 + (n − 1)ρ].
√ √
1.1 |a • b| = | − 9| = 9, while kak kbk = 6 22 ∼
= 11.489 > 9.
1.2 To verify the Cauchy–Schwarz inequality, first see that the inequality holds trivially if
a and b are zero vectors. We therefore assume that both a and b are nonzero. Let c be
the vector c = xa − yb, where x = b0 b, and y = a0 b. Clearly, c0 c ≥ 0. We express c0 c
in terms of x and y:
c0 c = (xa − yb)0 (xa − yb) = x2 a0 a − 2xya0 b + y 2 b0 b.
Since c0 c ≥ 0, and using the definitions of x and y, we see that
(b0 b)2 (a0 a) − 2(a0 b)2 (b0 b) + (a0 b)2 (b0 b) ≥ 0
and dividing by b0 b in the last inequality, we see that
(b0 b)(a0 a) − (a0 b)2 ≥ 0,
which verifies the Cauchy–Schwarz inequality.
We use the Cauchy–Schwarz inequality to deduce the triangle inequality, which can be
written in an equivalent form
ka + bk2 ≤ (kak + kbk)2 .
The expression on the left is
ka + bk2 = (a + b) • (a + b) = a • a + 2a • b + b • b
= kak2 + 2a • b + kbk2
while the expression on the right is
(kak + kbk)2 = kak2 + 2kakkbk + kbk2 .
Comparing these two formulas, we see that the triangle inequality holds if and only
if a • b ≤ kak kbk. By Cauchy–Schwarz inequality, |a • b| ≤ kakkbk, so the triangle
inequality follows as a consequence of the Cauchy–Schwarz inequality. The converse is
also true; i.e., if the triangle inequality holds, then a • b ≤ kakkbk holds for a and for −a,
from which Cauchy–Schwarz inequality follows. If equality holds, i.e., if a • b = kakkbk,
then b = ca, for some scalar c. Hence, a • b = ckak2 , and kakkbk = |c|kak2 . For nonnull
a, this implies that c = |c|, so that c ≥ 0. If b 6= 0, then b = ca, with c > 0.
1.3 Since x0 y = 0 = x0 z = y0 z, it follows that we must
√ solve the equations
√ a2 + b2 = 1, and
2 2
a − b = 0, for which the solutions are a = ±1/ 2 and b = ±1/ 2.
1.4 Let
1 2 0
1 0 −2
V=
0
.
1 1
−1 −1 1
1
,2 Solutions to Chapter 1
Using elementary column transformations C2 − 2C1 , and then, C3 − C2 , the matrix V
becomes
1 0 0
1 −2 0
V= 0
,
1 0
−1 −1 0
so that the column rank of V (or the dimension of its column space) is 2 < 3. Therefore,
v1 , v2 and v3 are linearly dependent. It is easily seen that v1 and v2 are LIN, and that
v3 = v2 − 2v1 .
1.5 It is easy to see that c1 v1 + c2 v2 + c3 v3 = 0 results in the following three equations,
and the only solution is c1 = 0, c2 = 0, and c3 = 0:
2c1 + 8c2 − 4c3 = 0
3c1 − 6c2 + 3c3 = 0
2c1 + 5c2 + c3 = 0.
1.6 Using elementary transformations, we can show that A is equivalent to the matrix
−3 3 3
0 4 4 ,
0 0 −1
so that the columns of A are LIN.
2 1 3
1.7 We can solve the system = c1 + c2 to get c1 = −1 and c2 = 1. Hence, u
3 2 5
is in Span{v1 , v2 }.
· · · , vm are linearly dependent, then there are scalars c1 , · · · , cm not
1.8 If v1 ,P Pall zero, such
m
that i=1 c i vi = 0. For any k with ck 6
= 0, we then have v k = − i6=k (ci /ck )vi ,
showing property 1 in Result 1.2.2.
Suppose without loss of generality that Pvs 1 , · · · , vs are linearly dependent and c1 , · · · , cs
are constants, not all zero, such that P i=1 ci vi = 0. Let cj = 0 for all j = s + 1, · · · , n.
n
Then c1 , · · · , cn are not all zero and i=1 ci vi = 0. Hence v1 , · · · , vn are linearly
dependent, showing property 2 in Result 1.2.2.
1.9 1. Let S = {v1 , · · · , vn } denote a set of nonzero orthogonal vectors, and let u belong to
the span of S with
u = c1 v1 + · · · + cn vn .
For a fixed i = 1, · · · , n, take the inner product of each side with vi . Since vi • vj = 0,
i 6= j,
u • vi = c1 (v1 • vi ) + · · · + cn (vn • vi ) = ci (vi • vi ).
Hence, ci = (u • vi ) (vi • vi ). To verify linear independence, set u = c1 v1 +· · ·+cn vn = 0.
This implies that ci = 0, i = 1, · · · , n, which in turn implies LIN of {v1 , · · · , vn }.
, Solutions to Chapter 1 3
2. By definition, every v ∈ V1 + · · · + Vm can be written as v1 + · · · + vm , where vi ∈ Vi ,
i = 1, . . . , m. Let wi ∈ Vi , i = 1, . . . , m, such that we also have v = w1 + · · · + wm . Then
(w1 − v1 ) + · · · + (wm − vm ) = 0. For each i, pre-multiply both sides by (wi − vi )0 . For
j 6= i, since Vi ⊥ Vj , then (wi − vi )0 (wj − vj ) = 0. As a result, (wi − vi )0 (wi − vi ) =
kwi − vi k2 = 0, giving wi = vi . Hence by definition, the sum of the Vi ’s is a direct sum.
1.10 Since {v1 , · · · , vm } is a basis of V, the vectors are LIN. So from
k−1
X yi0 vk
y1 = v1 , yk = vk − yi , k = 2, · · · , m,
i=1
kyi k2
yk
yk 6= 0, implying that zk = kyk k are well-defined and each has length 1. On the other
hand, for 1 ≤ j < k ≤ m,
k−1 j−1 k−1
X yi0 vk 0 X yi0 vk 0 X y 0 vk
yj0 yk = yj0 vk − y y i = − y yj − i
y0 y .
2 j i
i=1
kyi k2 j i=1
ky i k2 i
i=j+1
ky i k
If k = 2, then j = 1, and it is straightforward to see that y10 y2 = 0. Suppose we have
shown that for all i < j < k, yi0 yj = 0. Then the above identity shows that for all
j < k, yj0 yk = 0. By induction, the yi ’s are orthogonal to each other. Then z1 , · · · , zm
are orthonormal. Since they are LIN from Exercise 1.9, and there are m of them, they
form an orthonormal basis of V.
1.11 We have W∩(W ⊥ ∩V) ⊂ W∩W ⊥ = {0}. From W ⊂ V and W ⊥ ∩V ⊂ V, W⊕(W ⊥ ∩V) ⊂
V. On the other hand, for any v ∈ V, there are unique w ∈ W and u ∈ W ⊥ , such that
v = w + u. Since w ∈ V, then u = v − w ∈ V, and so u ∈ W ⊥ ∩ V. As a result
V ⊂ W ⊕ (W ⊥ ∩ V). Then V = W ⊕ (W ⊥ ∩ V). From the paragraph below Definition
1.2.8, dim V = dim W + dim(W ⊥ ∩ V), completing the proof of property 1. Next, by
W ⊥ ∩V ⊂ W ⊥ , (W ⊥ ∩V)⊥ ∩V ⊃ (W ⊥ )⊥ ∩V = W ∩V = W. On the other hand, if v ∈ V
and v ⊥ (W ⊥ ∩ V), then by property 1, there are unique w ∈ W and u ∈ W ⊥ ∩ V such
that v = w+u. From assumption, v ⊥ u. Meanwhile w ⊥ u. Then u0 u = u0 (v−w) = 0,
so u = 0. Then v = w ∈ W. As a result, (W ⊥ ∩V)⊥ ∩V ⊂ W. Then (W ⊥ ∩V)⊥ ∩V = W,
showing property 2.
1.12 In order for a matrix A = {aij } i, j = 1, 2, 3, to commute with the matrix B, we require
that AB = BA. Computing the product on both sides, and equating them, we see that
the conditions are a11 = a22 = a33 , a12 = a23 , and a21 = a31 = a32 = 0.
1.13 By repeated multiplication, we see that
k
k a β
A = ,
0 1
where, β = b(1 + a + · · · + ak−1 ).
1.14 It is easily verified that the product (A − B)C is equal to Ak − Bk , where, C =
Ak−1 + Ak−2 B + · · · + ABk−2 + Bk−1 .
1.15 Clearly, (A0 A)0 = A0 (A0 )0 = A0 A, and (AA0 )0 = (A0 )0 A0 = AA0 .
1.16 If A = O, then clearly A0 A = O. To show the converse, let the column vectors of A be
a1 , · · · , an . Since A0 A = {a0i aj }, if A0 A = O, then for all i = 1, · · · , n, a0i ai = 0, giving
ai = 0, and so A = O.
, 4 Solutions to Chapter 1
1.17 (a) From Result 1.3.5,
k
X k
X k
X
tr(A xi x0i ) = tr( Axi x0i ) = tr(Axi x0i )
i=1 i=1 i=1
k
X k
X
= tr(x0i Axi ) = (xi Axi ).
i=1 i=1
(b) From property 4 of Result 1.3.5, it follows that
tr(B−1 AB) = tr(BB−1 A) = tr(A).
1.18 Suppose A = {aij } and B = {bij } are both lower triangular matrices and P aii = ai and
n
bii = bi . Then aij = bij = 0 for all i < j. Now C = AB = {cij } with cij = k=1 aik bkj .
If i > j, then for every k = 1, · · · , n, either i > k or k > j, so aik bkj = 0, giving
cij = 0. Then C is lower triangular. If i = j, then aik bkj 6= 0 if and only if k = i, so
cii = aii bii = an bn . This completes the proof for the lower triangular case. The proof
for the upper triangular case is similar.
Pm Pn
1.19 Properties 1, 3, 5, and 6, of Result 1.3.5 are obvious. From tr(AB) = i=1 j=1 aij bji =
Pn Pm Pn Pm
j=1 i=1 aij bji = j=1 i=1 bji aij = tr(BA), Property 2 follows. Property 4 is a
direct consequence, since we can regard ABC as the product of AB and C, or of A and
BC. Both tr(AA0 ) and tr(A0 A) equal the sum of squares of the elements of A, which is
nonnegative, and shows property 7, of which property 8 is an immediate consequence.
1.20 If A = {akl } is lower triangular, then akl = 0 for all k < l. Now Mij = {mkl }, where
akl if k < i, l < j
a
k+1,l if k ≥ i, l < j
mkl =
ak,l+1 if k < i, l ≥ j
ak+1,l+1 if k ≥ i, l ≥ j.
If i > j, then for k < l, the second case in the display is not possible, and all the other
cases are possible, each one giving value 0. Therefore, mkl = 0, so that Mij is lower
triangular. Furthermore, for any k = j, · · · , i − 1, mkk = ak,k+1 = 0, so Mij has at least
one diagonal element equal to zero. This verifies the first fact.
From (1.3.1), the ith diagonal element of A−1 is |Mii |/|A|. If the diagonal elements of A
are d1 , · · · , dn , then the diagonal elements of Mii are d1 , · · · , di−1 , di+1 , · · · , dn . There-
fore |Mii | = d1 · · · di−1 di+1 · · · dn and |A| = d1 · · · dn . Then the ith diagonal element of
A−1 is 1/di .
1.21 Using Definition 1.3.7, |A| = −8.
1.22 It is easy to verify the given resursive expression, from which we obtain ∆n = (1 + a2 +
a4 + · · · + a2n ) = [1 − a2(n+1) ]/[1 − a4 ].
1.23 This follows directly from property 7 of Result 1.3.6.
Pn
1.24 Using Definition 1.2.7, we can show that the value of this determinant is 1 + i=1 ai .
1.25 The determinant to be calculated is |(1 − ρ)In + ρJn |. If ρ 6= 1, then from 12 of Result
1.3.6, the determinant is (1 − ρ)n |In + ρ(1 − ρ)−1 1n 10n | = (1 − ρ)n [1 + ρ(1 − ρ)−1 10n 1n ] =
(1 − ρ)n [1 + nρ(1 − ρ)−1 ] = (1 − ρ)n−1 [1 + (n − 1)ρ].
