GLOSSARY FOR ISYE 6501 INTRODUCTION TO ANALYTICS MODELING
(Organized by topics; for full alphabetical glossary, see other file)
TOPIC LESSONS PAGE
BASIC MACHINE LEARNING 2.1-2.2, 2.4-2.6, 2.8, 4.1, 4.3- 2
4.6, 6.1-6.3, 16.4
CONFUSION MATRICES 10.5-10.6 3
DATA 2.3, 2.7, 5.1-5.3, 9.2-9.5, 14.1- 5
14.3
DESIGN OF EXPERIMENTS 12.1-12.4 7
GAME THEORY 16.5-16.5a 8
MODEL QUALITY 3.1-3.4, 8.2, 8.4 9
NON-PARAMETRIC TESTS 16.1 10
OPTIMIZATION 15.1-15.8, 16.3 11
PROBABILITY-BASED MODELS 13.5-13.8, 16.2 15
PROBABILITY DISTRIBUTIONS 13.1-13.4 16
REGRESSION 8.1, 8.3-8.6, 9.1, 10.1-10.4, 10.7 18
TIME SERIES MODELS 7.1-7.6 20
VARIABLE SELECTION 11.1-11.3 22
OTHER TOPICS 1.1, 4.2, and other assorted 22
lessons
Definitions in this document are meant to be in the context of ISYE 6501 only. Some of these terms 1
have other definitions beyond the scope of this course. Many of these terms have precise
mathematical definitions not included here (or even glossed over here), because they are beyond
the scope of the course.
,BASIC MACHINE LEARNING LESSONS 2.1-2.2, 2.4-2.6, 2.8, 4.1, 4.3-4.6, 6.1-6.3, 16.4
Algorithm Step-by-step procedure designed to carry out a task.
Change detection Identifying when a significant change has taken place in a process.
Classification The separation of data into two or more categories, or (a point’s
classification) the category a data point is put into.
Classifier A boundary that separates the data into two or more categories. Also
(more generally) an algorithm that performs classification.
Cluster A group of points identified as near/similar to each other.
Cluster center In some clustering algorithms (like -means clustering), the central
point (often the centroid) of a cluster of data points.
Clustering Separation of data points into groups (“clusters”) based on
nearness/similarity to each other. A common form of unsupervised
learning.
CUSUM Change detection method that compares observed distribution mean
with a threshold level of change. Short for “cumulative sum”.
Deep learning Neural network-type model with many hidden layers.
Dimension A feature of the data points (for example, height or credit score). (Note
that there is also a mathematical definition for this word.)
EM algorithm Expectation-maximization algorithm.
Expectation-maximization General description of an algorithm with two steps (often iterated),
algorithm (EM algorithm) one that finds the function for the expected likelihood of getting the
response given current parameters, and one that finds new
parameter values to maximize that probability.
Heuristic Algorithm that is not guaranteed to find the absolute best (optimal)
solution.
-means algorithm Clustering algorithm that defines clusters of data points, each
corresponding to one of cluster centers selected by the algorithm.
-Nearest-Neighbor (KNN) Classification algorithm that defines a data point’s category as a
function of the nearest data points to it.
Kernel A type of function that computes the similarity between two inputs;
thanks to what’s (really!) sometimes known as the “kernel trick”,
nonlinear classifiers can be found almost as easily as linear ones.
Learning Finding/discovering patterns (or rules) in data, often that can be
applied to new data.
Machine Apparatus that can do something; in “machine learning”, it often refers
Definitions in this document are meant to be in the context of ISYE 6501 only. Some of these terms 2
have other definitions beyond the scope of this course. Many of these terms have precise
mathematical definitions not included here (or even glossed over here), because they are beyond
the scope of the course.
, to both an algorithm and the computer it’s run on. (Fun fact: before
computers were developed, the term “computers” referred to people
who did calculations quickly in their heads or on paper!)
Margin For a single point, the distance between the point and the classification
boundary; for a set of points, the minimum distance between a point
in the set and the classification boundary. Also called the separation.
Machine learning Use of computer algorithms to learn and discover patterns or structure
in data, without being programmed specifically for them.
Misclassified Put into the wrong category by a classifier.
Neural network A machine learning model that itself is modeled after the workings of
neurons in the brain.
Supervised learning Machine learning where the “correct” answer is known for each data
point in the training set.
Support vector In SVM models, the closest point to the classifier, among those in a
category. (Note that there is a more-technical mathematical definition
too.)
Support vector machine (SVM) Classification algorithm that uses a boundary to separate the data into
two or more categories (“classes”).
SVM Support vector machine.
Unsupervised learning Machine learning where the “correct” answer is not known for the
data points in the training set.
Voronoi diagram Graphical representation of splitting a plane with two or more special
points into regions with one special point each, where each region’s
points are closer to the region’s special point than to any other special
point.
CONFUSION MATRICES LESSONS 10.5-10.6
Accuracy Fraction of data points correctly classified by a model; equal to
+
+ + + .
Confusion matrix Visualization of classification model performance.
Diagnostic odds ratio Ratio of the odds that a data point in a certain category is correctly
classified by a model, to the odds that a data point not in that category
is incorrectly classified by the model; equal to ⁄
=
×
.
⁄ ×
Fall out Fraction of data points not in a certain category that are incorrectly
Definitions in this document are meant to be in the context of ISYE 6501 only. Some of these terms 3
have other definitions beyond the scope of this course. Many of these terms have precise
mathematical definitions not included here (or even glossed over here), because they are beyond
the scope of the course.
(Organized by topics; for full alphabetical glossary, see other file)
TOPIC LESSONS PAGE
BASIC MACHINE LEARNING 2.1-2.2, 2.4-2.6, 2.8, 4.1, 4.3- 2
4.6, 6.1-6.3, 16.4
CONFUSION MATRICES 10.5-10.6 3
DATA 2.3, 2.7, 5.1-5.3, 9.2-9.5, 14.1- 5
14.3
DESIGN OF EXPERIMENTS 12.1-12.4 7
GAME THEORY 16.5-16.5a 8
MODEL QUALITY 3.1-3.4, 8.2, 8.4 9
NON-PARAMETRIC TESTS 16.1 10
OPTIMIZATION 15.1-15.8, 16.3 11
PROBABILITY-BASED MODELS 13.5-13.8, 16.2 15
PROBABILITY DISTRIBUTIONS 13.1-13.4 16
REGRESSION 8.1, 8.3-8.6, 9.1, 10.1-10.4, 10.7 18
TIME SERIES MODELS 7.1-7.6 20
VARIABLE SELECTION 11.1-11.3 22
OTHER TOPICS 1.1, 4.2, and other assorted 22
lessons
Definitions in this document are meant to be in the context of ISYE 6501 only. Some of these terms 1
have other definitions beyond the scope of this course. Many of these terms have precise
mathematical definitions not included here (or even glossed over here), because they are beyond
the scope of the course.
,BASIC MACHINE LEARNING LESSONS 2.1-2.2, 2.4-2.6, 2.8, 4.1, 4.3-4.6, 6.1-6.3, 16.4
Algorithm Step-by-step procedure designed to carry out a task.
Change detection Identifying when a significant change has taken place in a process.
Classification The separation of data into two or more categories, or (a point’s
classification) the category a data point is put into.
Classifier A boundary that separates the data into two or more categories. Also
(more generally) an algorithm that performs classification.
Cluster A group of points identified as near/similar to each other.
Cluster center In some clustering algorithms (like -means clustering), the central
point (often the centroid) of a cluster of data points.
Clustering Separation of data points into groups (“clusters”) based on
nearness/similarity to each other. A common form of unsupervised
learning.
CUSUM Change detection method that compares observed distribution mean
with a threshold level of change. Short for “cumulative sum”.
Deep learning Neural network-type model with many hidden layers.
Dimension A feature of the data points (for example, height or credit score). (Note
that there is also a mathematical definition for this word.)
EM algorithm Expectation-maximization algorithm.
Expectation-maximization General description of an algorithm with two steps (often iterated),
algorithm (EM algorithm) one that finds the function for the expected likelihood of getting the
response given current parameters, and one that finds new
parameter values to maximize that probability.
Heuristic Algorithm that is not guaranteed to find the absolute best (optimal)
solution.
-means algorithm Clustering algorithm that defines clusters of data points, each
corresponding to one of cluster centers selected by the algorithm.
-Nearest-Neighbor (KNN) Classification algorithm that defines a data point’s category as a
function of the nearest data points to it.
Kernel A type of function that computes the similarity between two inputs;
thanks to what’s (really!) sometimes known as the “kernel trick”,
nonlinear classifiers can be found almost as easily as linear ones.
Learning Finding/discovering patterns (or rules) in data, often that can be
applied to new data.
Machine Apparatus that can do something; in “machine learning”, it often refers
Definitions in this document are meant to be in the context of ISYE 6501 only. Some of these terms 2
have other definitions beyond the scope of this course. Many of these terms have precise
mathematical definitions not included here (or even glossed over here), because they are beyond
the scope of the course.
, to both an algorithm and the computer it’s run on. (Fun fact: before
computers were developed, the term “computers” referred to people
who did calculations quickly in their heads or on paper!)
Margin For a single point, the distance between the point and the classification
boundary; for a set of points, the minimum distance between a point
in the set and the classification boundary. Also called the separation.
Machine learning Use of computer algorithms to learn and discover patterns or structure
in data, without being programmed specifically for them.
Misclassified Put into the wrong category by a classifier.
Neural network A machine learning model that itself is modeled after the workings of
neurons in the brain.
Supervised learning Machine learning where the “correct” answer is known for each data
point in the training set.
Support vector In SVM models, the closest point to the classifier, among those in a
category. (Note that there is a more-technical mathematical definition
too.)
Support vector machine (SVM) Classification algorithm that uses a boundary to separate the data into
two or more categories (“classes”).
SVM Support vector machine.
Unsupervised learning Machine learning where the “correct” answer is not known for the
data points in the training set.
Voronoi diagram Graphical representation of splitting a plane with two or more special
points into regions with one special point each, where each region’s
points are closer to the region’s special point than to any other special
point.
CONFUSION MATRICES LESSONS 10.5-10.6
Accuracy Fraction of data points correctly classified by a model; equal to
+
+ + + .
Confusion matrix Visualization of classification model performance.
Diagnostic odds ratio Ratio of the odds that a data point in a certain category is correctly
classified by a model, to the odds that a data point not in that category
is incorrectly classified by the model; equal to ⁄
=
×
.
⁄ ×
Fall out Fraction of data points not in a certain category that are incorrectly
Definitions in this document are meant to be in the context of ISYE 6501 only. Some of these terms 3
have other definitions beyond the scope of this course. Many of these terms have precise
mathematical definitions not included here (or even glossed over here), because they are beyond
the scope of the course.
