CS7643: Deep Learning
Quiz 2 Problem Set Prep — Fully Worked Solutions
Problem set by Zsolt Kira
Solutions worked from Lesson 5 lectures and DL Book Ch. 9
Summer 2026
Part 0: Deriving the tools (nothing assumed)
Every problem in this set reduces to two facts about convolution layers. Before using them, we
derive both from first principles, following the Lesson 5 lectures (“Convolution Layers” / “Pooling
Layers,” Kira) and Goodfellow et al., Deep Learning, Ch. 9.
Tool 1: The output-size formula
A convolution layer takes a kernel of size F (per spatial dimension) and strides it across the input,
computing a dot product at each location. The lecture states this directly:
“The output size of a vanilla convolution operation is just the dimension of the im-
age minus the dimension of the kernel plus one.” — Lesson 5, “Convolution Layers”
transcript
Why minus F , plus 1? Consider one spatial dimension of length W with kernel length F ,
stride 1, no padding. The kernel must lie entirely on the input for the dot product to be defined
(the lecture calls these the valid positions: “a valid convolution is only when the kernel is fully on
the image”). The leftmost position places the kernel’s left edge at index 1; the rightmost position
places its left edge at index W − F + 1 (so its right edge sits exactly at W ). Counting the integer
positions from 1 to W − F + 1 gives
O = W − F + 1.
Adding padding. Padding P surrounds the input with P extra entries (usually zeros) on each
side. The lecture warns about exactly this trap: “padding often refers to adding one dimension. . .
P = 1. . . but actually you’re adding two to the height because you’re gonna add a padding to the
bottom as well as the top.” So the effective input length becomes W + 2P , and
O = (W + 2P ) − F + 1.
Adding stride. With stride S the kernel’s left edge visits positions 1, 1+S, 1+2S,j . . . — onlyk
every S-th position. The number of valid steps it can take beyond the first position is (W +2P
S
)−F
(the floor appears because a partial step would push the kernel off the input — the lecture: “the
1
, last time you stride the kernel, it’ll fly off the image,” so that position is not counted). Adding
back the initial position:
W − F + 2P
O= +1 (1)
S
This is applied independently per spatial dimension (width and height each get their own copy of
the formula; they need not be equal).
Output channels. Each kernel produces exactly one 2-D output map (its dot-product response
at every spatial location). Stacking the maps of all kernels gives the output depth:
“The key thing to remember: the number of channels in the output map is equal to the
number of kernels that we have.” — Lesson 5
“Same” convolution. A same convolution chooses P so the output size equals the input size
(DL Book Ch. 9 calls this same padding; the lecture: “we can pad the image to make the output
the same size”). Set O = W in Eq. (1) with S = 1:
F −1
W = W − F + 2P + 1 =⇒ 0 = −F + 2P + 1 =⇒ P =
2
(well-defined when F is odd, which is why odd kernel sizes are standard).
Tool 2: The parameter-count formula
A convolution kernel is small in width and height, but it always extends through the full depth
(all channels) of its input — each output value is a dot product over an Fh × Fw × Cin block.
Every entry of that block is a learned weight, and each kernel carries one learned bias added to its
output (“each kernel will have a bias that the output will be offset by” — Lesson 5). Crucially,
the same kernel is reused at every spatial position (weight sharing), so the spatial size of the input
contributes nothing to the parameter count. With Cout kernels (filters):
params = (Fh · Fw · Cin + 1) · Cout (2)
The lecture verifies this with its own example: a 3×3 kernel on a 3-channel input with 4 kernels
gives (3 · 3 · 3 + 1) · 4 = 28 · 4 = 112 parameters — the same formula we use throughout.
For a 1-D convolution the kernel has one spatial dimension, so the block is F × Cin and params
= (F · Cin + 1) · Cout . For a 3-D convolution it is Fd × Fh × Fw × Cin , giving (Fd Fh Fw Cin + 1) Cout .
The pattern: multiply all kernel spatial dims, times input channels, plus one bias, times the number
of filters.
Tool 3: Pooling has no parameters
Max pooling strides a window like a convolution, so its output size obeys Eq. (1), but it computes
a fixed max instead of a learned dot product:
“How many learned parameters does this layer have? The answer is none. This is just
the max operation. . . there are no parameters w that we need to learn for this layer.”
— Lesson 5, “Pooling Layers”
Pooling operates on each channel independently, so it also never changes the channel count.
2
Quiz 2 Problem Set Prep — Fully Worked Solutions
Problem set by Zsolt Kira
Solutions worked from Lesson 5 lectures and DL Book Ch. 9
Summer 2026
Part 0: Deriving the tools (nothing assumed)
Every problem in this set reduces to two facts about convolution layers. Before using them, we
derive both from first principles, following the Lesson 5 lectures (“Convolution Layers” / “Pooling
Layers,” Kira) and Goodfellow et al., Deep Learning, Ch. 9.
Tool 1: The output-size formula
A convolution layer takes a kernel of size F (per spatial dimension) and strides it across the input,
computing a dot product at each location. The lecture states this directly:
“The output size of a vanilla convolution operation is just the dimension of the im-
age minus the dimension of the kernel plus one.” — Lesson 5, “Convolution Layers”
transcript
Why minus F , plus 1? Consider one spatial dimension of length W with kernel length F ,
stride 1, no padding. The kernel must lie entirely on the input for the dot product to be defined
(the lecture calls these the valid positions: “a valid convolution is only when the kernel is fully on
the image”). The leftmost position places the kernel’s left edge at index 1; the rightmost position
places its left edge at index W − F + 1 (so its right edge sits exactly at W ). Counting the integer
positions from 1 to W − F + 1 gives
O = W − F + 1.
Adding padding. Padding P surrounds the input with P extra entries (usually zeros) on each
side. The lecture warns about exactly this trap: “padding often refers to adding one dimension. . .
P = 1. . . but actually you’re adding two to the height because you’re gonna add a padding to the
bottom as well as the top.” So the effective input length becomes W + 2P , and
O = (W + 2P ) − F + 1.
Adding stride. With stride S the kernel’s left edge visits positions 1, 1+S, 1+2S,j . . . — onlyk
every S-th position. The number of valid steps it can take beyond the first position is (W +2P
S
)−F
(the floor appears because a partial step would push the kernel off the input — the lecture: “the
1
, last time you stride the kernel, it’ll fly off the image,” so that position is not counted). Adding
back the initial position:
W − F + 2P
O= +1 (1)
S
This is applied independently per spatial dimension (width and height each get their own copy of
the formula; they need not be equal).
Output channels. Each kernel produces exactly one 2-D output map (its dot-product response
at every spatial location). Stacking the maps of all kernels gives the output depth:
“The key thing to remember: the number of channels in the output map is equal to the
number of kernels that we have.” — Lesson 5
“Same” convolution. A same convolution chooses P so the output size equals the input size
(DL Book Ch. 9 calls this same padding; the lecture: “we can pad the image to make the output
the same size”). Set O = W in Eq. (1) with S = 1:
F −1
W = W − F + 2P + 1 =⇒ 0 = −F + 2P + 1 =⇒ P =
2
(well-defined when F is odd, which is why odd kernel sizes are standard).
Tool 2: The parameter-count formula
A convolution kernel is small in width and height, but it always extends through the full depth
(all channels) of its input — each output value is a dot product over an Fh × Fw × Cin block.
Every entry of that block is a learned weight, and each kernel carries one learned bias added to its
output (“each kernel will have a bias that the output will be offset by” — Lesson 5). Crucially,
the same kernel is reused at every spatial position (weight sharing), so the spatial size of the input
contributes nothing to the parameter count. With Cout kernels (filters):
params = (Fh · Fw · Cin + 1) · Cout (2)
The lecture verifies this with its own example: a 3×3 kernel on a 3-channel input with 4 kernels
gives (3 · 3 · 3 + 1) · 4 = 28 · 4 = 112 parameters — the same formula we use throughout.
For a 1-D convolution the kernel has one spatial dimension, so the block is F × Cin and params
= (F · Cin + 1) · Cout . For a 3-D convolution it is Fd × Fh × Fw × Cin , giving (Fd Fh Fw Cin + 1) Cout .
The pattern: multiply all kernel spatial dims, times input channels, plus one bias, times the number
of filters.
Tool 3: Pooling has no parameters
Max pooling strides a window like a convolution, so its output size obeys Eq. (1), but it computes
a fixed max instead of a learned dot product:
“How many learned parameters does this layer have? The answer is none. This is just
the max operation. . . there are no parameters w that we need to learn for this layer.”
— Lesson 5, “Pooling Layers”
Pooling operates on each channel independently, so it also never changes the channel count.
2