ISYE 6501 - Midterm 1 with actual remedy

ISYE 6501 - RESOLUTION 2 with correct
question and answers.

,Elastic Net - correct ans:Constrain combination of absolute value of
coefficients and their squares.Choose tau and upsilon.



Ridge Regression - correct ans:Take out absolute value term from Elastic Net.



Doesn't do variable selection, but does lead to better predictive models.



LASSO - correct ans:Constraint added to standard regression equation.



Budget t applied the model.



Greedy Variable Selection - correct ans:Forward Selection

Backward Selection

Step-wise Regression



Global Variable Selection - correct ans:LASSO

Elastic Net



LASSO; How to choose t? - correct ans:Depends on:

1) # of variables you want

2) Quality of the model as you add more variables



Pros and Cons:

1) Forward Selection

2) Backward Selection

3) Step-wise Regression - correct ans:Pros:

, Good for quick initial analysis to identify variables of potential importance.



Cons:

Don't perform well on additional data.



Pros and Cons:

1) LASSO

2) Elastic Net - correct ans:Pros:

Better at prediction than greedy selection



Cons:

Slower



What's the difference between LASSO, Ridge Regression, and Elastic Net?



What's the difference between the sum of squared coefficients? - correct
ans:Quadratic term/constraint in ridge regression tends to shrinks/regularizes
the coefficients.



Shrinking adds bias, but reduces variance, resulting in better model.



Pros and Cons of Elastic Net (Only) - correct ans:Pros:

1) Variable selection benefits of LASSO

2) Predictive benefits of Ridge Regression



Cons:

1) Arbitrarily rules out some correlated variables like LASSO.