MACHINE LEARNING A BAYESIAN AND
OPTIMIZATION PERSPECTIVE
COMPREHENSIVE EXAMINATION TEST
VERIFIED QUESTIONS AND ACCURATE
ANSWERS
●● What is the benefit of averaging the outputs of independently trained
networks?
Answer: It can improve accuracy, calibration, and robustness.
●● What did Frankle et al. (2020) find about averaging model weights?
Answer: Averaging weights from different models can lead to failure of
the network.
●● What is stochastic weight averaging?
Answer: A method where model weights are sampled at different time
steps and averaged together to improve performance.
●● What is the purpose of snapshot ensembles?
Answer: To store models from different time steps and average their
predictions for better performance.
, ●● How can the diversity of models in ensembling be improved?
Answer: By cyclically increasing and decreasing the learning rate.
●● What is fast geometric ensembling?
Answer: A method that explores low-energy regions around an initial
solution to provide diverse models without retraining.
●● What is dropout in neural networks?
Answer: A regularization technique that randomly sets hidden units to
zero during training to prevent overfitting.
●● How does Monte Carlo dropout work?
Answer: It computes inference with several dropout patterns and
averages the results, approximating Bayesian inference.
●● What is label smoothing?
Answer: A technique that modifies the labels during training to improve
model calibration and performance.
●● What is the relationship between wider minima and generalization?
Answer: Wider minima are thought to generalize better, as they are less
sensitive to exact weight values.
OPTIMIZATION PERSPECTIVE
COMPREHENSIVE EXAMINATION TEST
VERIFIED QUESTIONS AND ACCURATE
ANSWERS
●● What is the benefit of averaging the outputs of independently trained
networks?
Answer: It can improve accuracy, calibration, and robustness.
●● What did Frankle et al. (2020) find about averaging model weights?
Answer: Averaging weights from different models can lead to failure of
the network.
●● What is stochastic weight averaging?
Answer: A method where model weights are sampled at different time
steps and averaged together to improve performance.
●● What is the purpose of snapshot ensembles?
Answer: To store models from different time steps and average their
predictions for better performance.
, ●● How can the diversity of models in ensembling be improved?
Answer: By cyclically increasing and decreasing the learning rate.
●● What is fast geometric ensembling?
Answer: A method that explores low-energy regions around an initial
solution to provide diverse models without retraining.
●● What is dropout in neural networks?
Answer: A regularization technique that randomly sets hidden units to
zero during training to prevent overfitting.
●● How does Monte Carlo dropout work?
Answer: It computes inference with several dropout patterns and
averages the results, approximating Bayesian inference.
●● What is label smoothing?
Answer: A technique that modifies the labels during training to improve
model calibration and performance.
●● What is the relationship between wider minima and generalization?
Answer: Wider minima are thought to generalize better, as they are less
sensitive to exact weight values.
