SOLUTIONS MANUAL FOR A FIRST COURSE IN MACHINE LEARNING (2ND EDITION) EXERCISE & SOLUTIONS BY SIMON ROGERS AND MARK GIROLAMI

SOLUTIONS MANUAL FOR A FIRST COURSE IN MACHINE LEARNING
(2ND EDITION) EXERCISE & SOLUTIONS BY SIMON ROGERS AND MARK
GIROLAMI

, Chapter 1


EXERCISE: 1.1. A high positive value of w 0 and a small negative value for w1.
These reflect the high intercept on the t axis (corresponding to the
theoretical time winning time at x = 0 and the small decrese in winning time
over the years.


EXERCISE: 1.2. The following would do the job:

1 % Attributes are stored in Nx1 vector x
2 % Targets are stored in Nx1 vector t
3 xb = mean(x);
4 tb = mean(t);
5 x2b = mean(x.*x);
6 xtb = mean(x.*t);
7 w1 = (xtb − xt*xb)/(x2b−xbˆ2);
8 w0 = tb−w1*xb;
9 % Plot the data
10 plot(x,t,'b.','markersize',25);
11 % Plot the model
12 hold on;
13 plot(x,w0+w1*x,'r','linewidth',2);




EXERCISE: 1.3. We need to find wTXTXw. We’ll start with XTX. Multiplying XT by X gives:

ΣN ΣN
T n=1x2n1 n=1 x n1x n2
X X= ΣN xn2 xn1 ΣN 2
n=1 n=1 xn2



Multiplying this by w gives:
Σ ΣN
T w0 N 2
+ w1 xn1xn2
x n1
w0 Σn=1
N
n=1 x n2 x n1 +
w1
n=1
Σ
X Xw = N
x
2
n=1 n2

1

, 2 CHAPTER 1.

Finally, pre-multiplying this by wT gives:
N
Σ Σ
N
wTXTXw = w0 x2n1 + w1 xn1xn2 +
w0 n=1 n =1
Σ
N
Σ
N
2
w1 w0 xn2xn1 + w1 xn2
n =1 n=1
Σ
N Σ
N Σ
N
= w2 x2 + 2w 0w 1 xn1x n2 + w 2 x2
0 n1 1 n2
n=1 n=1 n=1

as required.
EXERCISE: 1.4. Let’s first
work out Xw:
w 0x11 + w 1x12
w 0x21 + w 1x22
Xw =
.
w 0x N 1 + w 1x N 2
Therefore
(Xw)T = [w0x11 + w1x 12, w0x 21 + w1x22, . . . , w0xN1 + w1x N2]
Finally, work out wTXT:
wTXT = [w0x11 + w1x 12, w0x21 + w1x 22, . . . , w0xN1 + w1x N2]
as required.
Σ
EXERCISE: 1.5. Starting with n xnt n. The result of this is a column vector of the same size as x
(2 × 1). Now, using the definition of X,
x11, x21, . . . , xN1
XT =
x12, x22, . . . , xN2
(which is a 2× N vector). Multiplying this by t gives a×2 1 vector that looks
like this: Σ N xn1tn
T n=1
X t = ΣN
n=1 x n2tn
Σ
which is n xnt n as required. The second EXERCISE:ample, XTXw. We already know what
T
X Xw is (EXERCISE:ercise 1.3)
Σ ΣN
T w0 N 2
+ w1 xn1xn2
x
w0 Σn=1
N
n1
n=1 x n2 x n1 +
Σ
n=1
w1 x
X Xw = N 2
n=1 n2
Now, xnxnT is the following matrix:

xnxT = x2n1 xn1xn2
2
n xn2xn1 xn2

, 3

Multiplying this by w gives:

T w0x2n1 + w1xn1xn2
xnxnw = 2
w0xn2xn1 + w1xn2
Summing over the N terms leads us to the matrix we derived previously.
EXERCISE: 1.6. Code below:

1 %% Women's 100m data
2 % Load all Olympic data
3 load olympics;
4 % Copy the necessary variables
5 x = female100(:,1); % Olympic year
6 t = female100(:,2); % Winning time
7 % Augment x
8 X = [repmat(1,size(x)) x];
9 % Get solution
10 w = inv(X'*X)*X'*t;

The fitted model
is: t = 40.9242 − 0.0151x

EXERCISE: 1.7. Plugging 2012 and 2016 into the above EXERCISE:pression
yields winning times of 10.5997 and 10.5394 respectively.
EXERCISE: 1.8. The men’s model is:
t = 36.4165 − 0.0133x
The women’s model
is: t = 40.9242 − 0.0151x
The women’s time is decreasing faster than the men’s. Therefore, the
women will be faster at the first Olympics after the x that gives identical
winning times:
40.9242 − 0.0151x = 36.4165 − 0.0133x
x = 2589

The nEXERCISE:t Olympic year after 2589 is (assuming they continue to be
held every four years) is the year 2592. The winning times are the
unrealistically fast 1.8580 seconds and 1.8628 seconds for women and men
respectively.
EXERCISE: 1.9. Code below (synthdata cv.m):

1 clear all;close all;
2 load synthdata
3
4 % Augment x
5 X = repmat(1,size(x));
6 for k = 1:4
7 X = [X x.ˆk];
8 end
9