OMSA Midterm 2 Exam with complete solutions

Overfitting - Answer- Number of factors is too close to or larger than number of data points -- fitting to both real effects and random effects. Comes from including too many variables! Ways to avoid overfitting - Answer- - Need number of factors to be same order of magnitude as the number of points - Need enough factors to get good fit from real effects and random effects Simplicity - Answer- Simple models are better than complex. When fewer factors exist, less data collection is required -- less chance for including factors that are not significant. Another reason for variable selection! DOE - Answer- systematic method to determine the relationship between factors affecting a process and the output of that process. must make sure either: 1) 2 data sets have same mix 2) break down data into smaller tests that test all factors, not just one. forward selection - Answer- go step by step either narrowing or building a model -- begin with no factors. only allow new factors with p-value 0.1 or lower and removing any factors above 0.05. Greedy models - Answer- forward selection, backward selection, stepwise regression means: at each step it does the one thing that looks best withouth taking future options into consideration scaled variable selection models - Answer- lasso, elastic net, (and ridge even though not variable selection) backward selection - Answer- start with model includes all factors and at each step find worst factor and remove it from the model. continue till there's no factor bad enough to remove and model doesn't have any more factors we want. step-wise regression - Answer- combination of forward and backward elimination -- beginning with either all factors or no factors. after adding each new factor (and at end) -- eliminate right away any factors that no longer appear any good. future options are not considered. could use AIC or BIC, or model's R squared to pick