ISYE 6501 - RESOLUTION 2 with correct
question and answers.
,Elastic Net - correct ans:Constrain combination of absolute value of
coefficients and their squares.Choose tau and upsilon.
Ridge Regression - correct ans:Take out absolute value term from Elastic Net.
Doesn't do variable selection, but does lead to better predictive models.
LASSO - correct ans:Constraint added to standard regression equation.
Budget t applied the model.
Greedy Variable Selection - correct ans:Forward Selection
Backward Selection
Step-wise Regression
Global Variable Selection - correct ans:LASSO
Elastic Net
LASSO; How to choose t? - correct ans:Depends on:
1) # of variables you want
2) Quality of the model as you add more variables
Pros and Cons:
1) Forward Selection
2) Backward Selection
3) Step-wise Regression - correct ans:Pros:
, Good for quick initial analysis to identify variables of potential importance.
Cons:
Don't perform well on additional data.
Pros and Cons:
1) LASSO
2) Elastic Net - correct ans:Pros:
Better at prediction than greedy selection
Cons:
Slower
What's the difference between LASSO, Ridge Regression, and Elastic Net?
What's the difference between the sum of squared coefficients? - correct
ans:Quadratic term/constraint in ridge regression tends to shrinks/regularizes
the coefficients.
Shrinking adds bias, but reduces variance, resulting in better model.
Pros and Cons of Elastic Net (Only) - correct ans:Pros:
1) Variable selection benefits of LASSO
2) Predictive benefits of Ridge Regression
Cons:
1) Arbitrarily rules out some correlated variables like LASSO.
question and answers.
,Elastic Net - correct ans:Constrain combination of absolute value of
coefficients and their squares.Choose tau and upsilon.
Ridge Regression - correct ans:Take out absolute value term from Elastic Net.
Doesn't do variable selection, but does lead to better predictive models.
LASSO - correct ans:Constraint added to standard regression equation.
Budget t applied the model.
Greedy Variable Selection - correct ans:Forward Selection
Backward Selection
Step-wise Regression
Global Variable Selection - correct ans:LASSO
Elastic Net
LASSO; How to choose t? - correct ans:Depends on:
1) # of variables you want
2) Quality of the model as you add more variables
Pros and Cons:
1) Forward Selection
2) Backward Selection
3) Step-wise Regression - correct ans:Pros:
, Good for quick initial analysis to identify variables of potential importance.
Cons:
Don't perform well on additional data.
Pros and Cons:
1) LASSO
2) Elastic Net - correct ans:Pros:
Better at prediction than greedy selection
Cons:
Slower
What's the difference between LASSO, Ridge Regression, and Elastic Net?
What's the difference between the sum of squared coefficients? - correct
ans:Quadratic term/constraint in ridge regression tends to shrinks/regularizes
the coefficients.
Shrinking adds bias, but reduces variance, resulting in better model.
Pros and Cons of Elastic Net (Only) - correct ans:Pros:
1) Variable selection benefits of LASSO
2) Predictive benefits of Ridge Regression
Cons:
1) Arbitrarily rules out some correlated variables like LASSO.
