ISYE 6501 FINAL COMPLETE EXAM 2025

Support vector machine - -a supervised learning, classification model. Uses extremes, or identified points in the data from which margin vectors are placed against. The hyperplane between these vectors is the classifier Svm pros/cons - -pros: it works really well with a clear margin of separation It is effective in high dimensional spaces. It is effective in cases where the number of dimensions is greater than the number of samples. It uses a subset of training points in the decision function (called support vectors), so it is also memory efficient. Cons: not good for very large data sets Not good for when the data set has more noise i.e. Target classes are overlapping Doesn't directly provide probability estimates. K-nearest neighbor (k-nn) - -an unsupervised classification algorithm. Looks at the x number of closest points to the new one and classifies as whichever is most common. K-nearest neighbor (k-nn) pros/cons - -pros: no assumptions about data Easy to understand/interpret Varsatile Cons: computationally expensive because algorithm stores all training data Sensitive to irrelevant features and scale of data K-fold cross validation - -validation technique where data is divided into x number of data subsets. Each subset is then used as a for testing while the rest are used for training. The algorithm then rotates through each subset and averages the results K fold cross validation pros/cons - -pros: validates performance of model Can create balance across predicted features classes Cons: doesn't work well with time series data The aggregate scores of your model could miss some important extreme values or overpower them so theyre harder to pick up on K-means clustering - -unsupervised learning heuristic that sets x starts by assigning x number of cluster centers, then clusters all data points into each of them based on distance. The center point of each cluster is then calculated and all data points are again re clustered. Repeat process until no-data points change clusters. Ideal number of clusters can be identified via elbow diagram. ISYE 6501 ISYE 6501 K-means pros and cons - -pros: simple to implement Scales well to large data sets Easily adaptable Cons: choosing k manually can bias it towards initial values Sensitive to outliers Grubbs outlier test - -a formula that uses an outlier's value, the mean of the data, and the standard deviation to determine whether or not the data point is within the confidence interval for a normal distribution or should be thrown out Cusum - -change detection model that keeps a running total of the amount that observations vary above the expected value. The running total exceeds a preset threshold value, it indicates there has been a change. Cusum pros/cons - -pros: best way to detect the small shifts of process mean especially 0.5 to 2 sd from the target mean Easy to identify visually the shifts in process mean Cons: cumbersome to establish and maintain Tough to interpret the patterns. Choosing c and t values is a pro and con as it can cause bias but creates more flexibility Exponential smoothing - -technique regarding time series data in which older observations are assigned exponentially decreasing weights, so more emphasis is given to recent observations. Can include trends, seasonality, and cyclic patterns to account for expected differences in observations over time Exponential smoothing pros and cons - -pros: easy to learn and apply Can produce accurate forecasts Can account for trends/seasonality/cyclic effects Works well when mean/variance/etc are expected to remain relatively constant Cons: forecasts can sometimes have lag Arima (auto regressive integrated moving average) - -a time series analysis method used for forecasting that combines three components: differences in differences to find stationary change when data metrics aren't stationary. Autoregression, where predicting current value is based on previous time period values Moving averages where we go back and incorporate q time periods' previous errors Garch (generalized autoregressive conditional Heteroskedasticity) - -time series analytic method that estimates/forecasts variance. Helps determine how much a forecast may be higher or lower than the true value. Useful for estimating risks on investment portfolios. Linear regression - -a regression technique that describes relationships between independent and dependent variables as linear functions ISYE 6501 ISYE 6501 Aic (akaike information criterion) - -model selection technique that balances model fit and complexity. Penalizes models with too much complexity in an attempt to avoid overfitting. Bic (bayesian information criterion) - -model election technique that balances model fit and complexity. Generally penalizes complexity more than aic. Box-cox transformation - -logarithmic transformation technique used to eliminate heteroskedasticity (unequal variance) across a data set to make it fit a normal distribution. This will help a model avoid being biased towards parts of the data with greater variance. Pca (principal component analysis) - -feature extraction technique that can reduce the dimensionality of the feature space in a model, creating new components ordered by importance, while also transforming the data around new orthogonal axis in order to remove correlation. Pca pros and cons - -pros: removes correlated features Can improve algorithm performance by omitting correlated variables Reduces over fitting Improves visualization Cons: harder to interpret effects of individual independent variables Requires data standardization Cart (classification and regression trees) - -regression technique that creates different models for different subset of data. This is useful when different factors may have different ways they interact that makes them more effective to model separately. For example when modeling likeliness of response to advertisement, may be useful to divide population into age group and marital status. Branching - -the process of deciding how to split the data, and which factors to include in a cart model. Can go either backwards, removing leaves that don't decrease variance more than a certain threshold, or forward by adding leaves until they either don't improve the model enough or don't contain enough data. Random forests - -method where a multitude of carts are produced, using randomized subset of data and randomized factors. For regression you then use the average predicted response across the trees, and for classification you use the most common predicted response/ Random forest pros/cons - -pros: tends to give better estimates overall because while each tree might be overfitting in one place or another, they don't necessarily over-fit the same way Average over all trees corrects overfitting Cons: harder to explain the output of these models. ISYE 6501 ISYE 6501 Doesn't help explain how the variables interact, or how certain sequence of branches is helpful or meaningful like we could a single tree. Best we can do is some aggregate measure rather than specific insights. Can't give us a single regression or classification model. Logistic regression - -a nonlinear regression model that relates a set of explanatory variables to a probability of a positive or negative response. Auc (area under the roc curve) - -the area under the roc curve is the probability that a classifier will be more confident that a randomly chosen positive example is actually positive than that a randomly chosen negative example is positive. Confusion matrices - -a measurement of how well a classification model works by breaking down responses into true positive (tp), true negative (tn), false positive (fp), and false negatives (fn) Sensitivity - -sensitivity = tp / (tp + fn) This is the fraction of category members that are correctly classified Specificity - -specificity = tn / (tn + fp) Fraction of non category members that are correctly classified Forward selection - -start with a model that has no factors. At each step we find the best new factor to add to the model And put it in as long as it's a good enough improvement. When there's no factor that's good enough to add, or if we've added as many factors as we want to have, we stop. Backward elimination - -we start with a model that includes all factors and at each step, we find the worst factor and remove it from the model. We continue until there's no factor bad enough to remove, and the model doesn't have any more factors than we want. Stepwise regression - -multiple different forms, but essentially a combination of forward selection and backwards elimination. Since in each step these models look at only the best current option and don't take future possibilities into account it is known as the greedy algorithm Lasso approach - -we add a constraint to the standard regression equation. The goal is still to minimize sse given the regression a budget t to use on coefficients. It'll use that budget on the most important coefficients which means all the rest of the factors will have zero coefficient and so those factors won't be part of the model. Elastic net - -elastic net is effectively a combination of lasso and ridge regressions that trades some bias in order to reduce variance and ultimately reduce total prediction ISYE 6501 ISYE 6501 error. Constrains a combination of the absolute value of the coefficients and their squares. Elastic net pros/cons - -pros: variable selection benefits of lasso Predictive benefits of ridge regression Cons: arbitrarily rules out some correlated variables Underestimate coefficients of very predictive variables A/b testing - -analytic method used to pick the best out of several alternatives. Best used when data can be collected quickly, from a representative population, and the amount of data is small relative to the whole population. Factorial design tests - -design of experiment method used to test a multitude of combinations. E.g 2 fonts x 2 wordings x 2 backgrounds to compare efficacy Multi-armed bandit - -design of experiment approach that starts with no information and k alternatives. Over time, while performing tests, the probability of which alternative is best is updated and we become more likely to chose the better alternatives