CS7643 - QUIZ 3 WITH ALL GRADED A+ SOLUTIONS
1. optimization Eerror: E Even Eif ENN Ecan Eperfectly Emodel Ethe Eworld, Eyour Eoptimization Ealgorithm
Emay Enot Ebe Eable Eto Efind Ethe Egood Eweights Ethat Emodel Ethat Efunction
2. estimation Eerror: EEven Eif Eyou Efind Ethe Ebest Ehypothesis Ethat Ehas Ethe Ebest Eset Eof Eweights
Eor Eparameters Efor Eour ENN Ethat Eminimizes Etraining Eerror, Edoesn't Emean Eyou'll Ebe Eable Eto
Egeneralize Eto Ethe Etesting Eset. Ee.g. EOverfitting Eand Elearning Efeatures Ethat Ework Ewell Ewith Etraining
Eset Eand Enot Ewith Etesting Eset.
3. modeling Eerror: E Given Ea ENN Earchitecture, Eyour Eactual Emodel Ethat Erepresents Ethe Ereal Eworld
Emay Enot Ebe Ein Ethat
space. EThere Emay Ebe Eno Eset Eof Eweights Ethat Emodel Ereal Eworld.
4. Errors Ethat Eprevent ENN Earchitectures Efrom Emodeling Ereal Eworld: E1.
1 E/
E5
, EOptimization EError
2. Estimation EError
3. Modeling EError
5. Transfer ELearning: E A Elearning Emethod Ethat Ereuses Efeatures Elearned Eon Ea Every Elarge Edataset
Eon Ea Enew Edataset, Ethis Emethod Eminimizes Ethe Eamount Eof Elabel Edata Eneeded.
1.) Train Eon Ea Elarge Edataset
2.) Take Ea Ecustom Edataset Eand Einitialize Ethe Enetwork Ewith Eweights Efrom Ethe Emodel Eon Ea Elarg
Edataset
3.) Train Eon Ethe Ecustom Edataset Eby Eeither:
FineTune: Eupdate Eall Eparameters
Freeze: Eupdate Elayer Ethat Edidn't Ecome Efrom Eold.
6. Difficulties Eof ETransfer ELearning: E- Eit Edoesn't Ework Ewell Eif Ethe Esource Edataset Eis Every
Editterent Efrom Ethe Etarget Edataset
-it's Ebetter Eto Euse Ethe Etarget Edata
7. Modern ECNN EArchitecture: E1.) EAlexNet
2.) VGGNet
3.) Inception ENet
4.) ResNet
8. ALEXNET: E- EAlexnet Eis Ethe Ename Eof Ethe EConvolutional ENeural ENetwork Earchitecture Ethat Ewon
Ethe EILSVRC E2012 Ecompetition Eby Ea Elarge Emargin Eand Ewas Eresponsible Efor Ea Eresurgence Eof
Einterest Ein ECNNs Efor EImage ERecognition. E- EIt Econsists Eof Efive Econvolutional Elayers, Esome Eof Ewhich Eare
Efollowed Eby Emax-pooling Elayers, Eand Ethree Efully-connected Elayers E with Ea Efinal E1000-way Esoftmax.
-Alexnet Ewas Eintroduced Ein EImageNet EClassification Ewith EDeep EConvolutional ENeural ENetworks
2 E/
E5
1. optimization Eerror: E Even Eif ENN Ecan Eperfectly Emodel Ethe Eworld, Eyour Eoptimization Ealgorithm
Emay Enot Ebe Eable Eto Efind Ethe Egood Eweights Ethat Emodel Ethat Efunction
2. estimation Eerror: EEven Eif Eyou Efind Ethe Ebest Ehypothesis Ethat Ehas Ethe Ebest Eset Eof Eweights
Eor Eparameters Efor Eour ENN Ethat Eminimizes Etraining Eerror, Edoesn't Emean Eyou'll Ebe Eable Eto
Egeneralize Eto Ethe Etesting Eset. Ee.g. EOverfitting Eand Elearning Efeatures Ethat Ework Ewell Ewith Etraining
Eset Eand Enot Ewith Etesting Eset.
3. modeling Eerror: E Given Ea ENN Earchitecture, Eyour Eactual Emodel Ethat Erepresents Ethe Ereal Eworld
Emay Enot Ebe Ein Ethat
space. EThere Emay Ebe Eno Eset Eof Eweights Ethat Emodel Ereal Eworld.
4. Errors Ethat Eprevent ENN Earchitectures Efrom Emodeling Ereal Eworld: E1.
1 E/
E5
, EOptimization EError
2. Estimation EError
3. Modeling EError
5. Transfer ELearning: E A Elearning Emethod Ethat Ereuses Efeatures Elearned Eon Ea Every Elarge Edataset
Eon Ea Enew Edataset, Ethis Emethod Eminimizes Ethe Eamount Eof Elabel Edata Eneeded.
1.) Train Eon Ea Elarge Edataset
2.) Take Ea Ecustom Edataset Eand Einitialize Ethe Enetwork Ewith Eweights Efrom Ethe Emodel Eon Ea Elarg
Edataset
3.) Train Eon Ethe Ecustom Edataset Eby Eeither:
FineTune: Eupdate Eall Eparameters
Freeze: Eupdate Elayer Ethat Edidn't Ecome Efrom Eold.
6. Difficulties Eof ETransfer ELearning: E- Eit Edoesn't Ework Ewell Eif Ethe Esource Edataset Eis Every
Editterent Efrom Ethe Etarget Edataset
-it's Ebetter Eto Euse Ethe Etarget Edata
7. Modern ECNN EArchitecture: E1.) EAlexNet
2.) VGGNet
3.) Inception ENet
4.) ResNet
8. ALEXNET: E- EAlexnet Eis Ethe Ename Eof Ethe EConvolutional ENeural ENetwork Earchitecture Ethat Ewon
Ethe EILSVRC E2012 Ecompetition Eby Ea Elarge Emargin Eand Ewas Eresponsible Efor Ea Eresurgence Eof
Einterest Ein ECNNs Efor EImage ERecognition. E- EIt Econsists Eof Efive Econvolutional Elayers, Esome Eof Ewhich Eare
Efollowed Eby Emax-pooling Elayers, Eand Ethree Efully-connected Elayers E with Ea Efinal E1000-way Esoftmax.
-Alexnet Ewas Eintroduced Ein EImageNet EClassification Ewith EDeep EConvolutional ENeural ENetworks
2 E/
E5
