CS 7643 / CS7643 Quiz 2 (Latest
Update ) Deep Learning
| Questions & Answers | Grade A |
100% Correct - Georgia Tech
Question:
Transfer learning
Answer:
Reuse the features on a new dataset, that were learned previously on a large-
scale dataset.
1. Train features on a large-scale dataset
2. Replace the last FC layer with one of our categories, and initialize with
random weights.
3. Continue train on the new dataset
a.) Finetune - update all parameters
b.) Freeze - update parameters only of the new FC layer - suggested if not
enough new data is available
,Question:
Transfer learning effectiveness
Answer:
Works well if:
>> Source data is large, but target data is pretty small
>> Generalizes across tasks (object recognition params can be used for object
detection)
Limitations
>> Target data/task is completely different (silhoutte, contour)
>> Target data is large --> the random initialization is better
Question:
Power law region
Answer:
If data set SIZE increases in LOG scale
than generalization ERROR decreases LINEARLY in LOG scale
, Question:
Gradient based image optimization
Answer:
1. ) Start with random/zero image
2. ) Add to the input image (I) the gradient w.r.t the score of a class (Sc) x
learning rate
3. ) Regularization (???)
Question:
Adversarial images
Answer:
Images, on which gradient-based optimization was performed, but on
incorrect class. This small change fools the network, but the picture still looks
like the original class for humans. (example image about panda)
Question:
Can a change of a single pixel change the entire class prediction?
Answer:
Yes
Update ) Deep Learning
| Questions & Answers | Grade A |
100% Correct - Georgia Tech
Question:
Transfer learning
Answer:
Reuse the features on a new dataset, that were learned previously on a large-
scale dataset.
1. Train features on a large-scale dataset
2. Replace the last FC layer with one of our categories, and initialize with
random weights.
3. Continue train on the new dataset
a.) Finetune - update all parameters
b.) Freeze - update parameters only of the new FC layer - suggested if not
enough new data is available
,Question:
Transfer learning effectiveness
Answer:
Works well if:
>> Source data is large, but target data is pretty small
>> Generalizes across tasks (object recognition params can be used for object
detection)
Limitations
>> Target data/task is completely different (silhoutte, contour)
>> Target data is large --> the random initialization is better
Question:
Power law region
Answer:
If data set SIZE increases in LOG scale
than generalization ERROR decreases LINEARLY in LOG scale
, Question:
Gradient based image optimization
Answer:
1. ) Start with random/zero image
2. ) Add to the input image (I) the gradient w.r.t the score of a class (Sc) x
learning rate
3. ) Regularization (???)
Question:
Adversarial images
Answer:
Images, on which gradient-based optimization was performed, but on
incorrect class. This small change fools the network, but the picture still looks
like the original class for humans. (example image about panda)
Question:
Can a change of a single pixel change the entire class prediction?
Answer:
Yes
