ISYE 6501 - Midterm 2 Exam 2024

ISYE 6501 - Midterm 2 Exam 2024 when might overfitting occur - answerwhen the # of factors is close to or larger than the # of data points causing the model to potentially fit too closely to random effects Why are simple models better than complex ones - answerless data is required; less chance of insignificant factors and easier to interpret what is forward selection - answerwe select the best new factor and see if it's good enough (R^2, AIC, or p-value) add it to our model and fit the model with the current set of factors. Then at the end we remove factors that are lower than a certain threshold what is backward elimination - answerwe start with all factors and find the worst on a supplied threshold (p = 0.15). If it is worse we remove it and start the process over. We do that until we have the number of factors that we want and then we move the factors lower than a second threshold (p = .05) and fit the model with all set of factors what is stepwise regression - answerit is a combination of forward selection and backward elimination. We can either start with all factors or no factors and at each step we remove or add a factor. As we go through the procedure after adding each new factor and at the end we eliminate right away factors that no longer appear. what type of algorithms are stepwise selection? - answerGreedy algorithms - at each step they take one thing that looks best what is LASSO - answera variable selection method where the coefficients are determined by both minimizing the squared error and the sum of their absolute value not being over a certain threshold t How do you choose t in LASSO - answeruse the lasso approach with different values of t and see which gives the best trade off