CS 7643 / CS7643 Quiz 5 (Latest
Update ) Deep Learning
| Questions & Answers | Grade A |
100% Correct - Georgia Tech
Question:
Meta-Learning (Few-Shot Learning)
Answer:
- Learning to learn
- Learn NN initialization that after it perform SGD steps on small amounts of
labeled data, you learn an effective initialization
Question:
Surrogate Tasks (Self-Supervised Learning)
Answer:
- Identify loss functions for tasks we don't care about, but allow us to learn
good feature representations
,Question:
Multi-View Pseudo-Labeling Key Details for Success
Answer:
- Pseudo-labeling without augmentation isn't very effective
--> need good data augmentation algos
- Doing this in multiple stages isn't as good as end to end
- Large unlabeled batch sizes are necessary for the labels to be good
- Confidence threshold is very important
--> too big = not getting many labels, too small = many noisy examples
- Cosine learning rate schedules
- Inference with exponential moving average of weights
--> stabilizes training and improves performance
Question:
Other Methods for Semi-Supervised Learning
Answer:
MixMatch/ReMixMatch: More complex variations prior to FIxMatch
- Temperature scaling and entropy minimization to stabilize training
- Multiple augmentations and ensemblier to improve pseudo-labels
Virtual Adversarial Training: Augmentation through adversarial examples
, Mean Teach: Student/teacher distillation consistency method with
exponential moving average
Question:
Label Propagation (Semi-supervised learning)
Answer:
- Learn feature extractors
- Use feature extractors on unlabeled examples
--> Uses clustering assumption so items close in feature space are labeled
similarly
Question:
Few-Shot Learning Baseline
Answer:
- Train classifier on base classes and Fine Tune using small amounts of labeled
data for new classes
- Pretty good baseline compared to more sophisticated approaches
Update ) Deep Learning
| Questions & Answers | Grade A |
100% Correct - Georgia Tech
Question:
Meta-Learning (Few-Shot Learning)
Answer:
- Learning to learn
- Learn NN initialization that after it perform SGD steps on small amounts of
labeled data, you learn an effective initialization
Question:
Surrogate Tasks (Self-Supervised Learning)
Answer:
- Identify loss functions for tasks we don't care about, but allow us to learn
good feature representations
,Question:
Multi-View Pseudo-Labeling Key Details for Success
Answer:
- Pseudo-labeling without augmentation isn't very effective
--> need good data augmentation algos
- Doing this in multiple stages isn't as good as end to end
- Large unlabeled batch sizes are necessary for the labels to be good
- Confidence threshold is very important
--> too big = not getting many labels, too small = many noisy examples
- Cosine learning rate schedules
- Inference with exponential moving average of weights
--> stabilizes training and improves performance
Question:
Other Methods for Semi-Supervised Learning
Answer:
MixMatch/ReMixMatch: More complex variations prior to FIxMatch
- Temperature scaling and entropy minimization to stabilize training
- Multiple augmentations and ensemblier to improve pseudo-labels
Virtual Adversarial Training: Augmentation through adversarial examples
, Mean Teach: Student/teacher distillation consistency method with
exponential moving average
Question:
Label Propagation (Semi-supervised learning)
Answer:
- Learn feature extractors
- Use feature extractors on unlabeled examples
--> Uses clustering assumption so items close in feature space are labeled
similarly
Question:
Few-Shot Learning Baseline
Answer:
- Train classifier on base classes and Fine Tune using small amounts of labeled
data for new classes
- Pretty good baseline compared to more sophisticated approaches
