ISyE 6525: High Dimensional Data
Analytics - Final Exam with
questions and correct answers and
rationale updated 2026 graded A+
Section 1: Foundations and the Curse of Dimensionality (Questions 1-15)
1. What is the primary characteristic that defines "high-dimensional data" in
the context of ISyE 6525?
A) The number of observations (n) is very large.
B) The number of features (p) is very large.
C) The data is stored in a distributed system.
D) The data requires more than 1 terabyte of storage.
Answer: B
Rationale: High-dimensional data is defined by a large number of features or
variables (p), often with p being comparable to or even larger than the number of
observations (n). This is the "large p, small n" paradigm.
2. The "curse of dimensionality," a term coined by Richard Bellman, refers to:
A) The increased computational cost of analyzing data with many features.
B) The exponential increase in the volume of the feature space as dimensionality
increases.
C) The difficulty in visualizing data with more than three dimensions.
D) The tendency for models to become more accurate as more features are added.
Answer: B
Rationale: The curse of dimensionality primarily refers to the fact that the volume of
the space increases so fast with dimensionality that the available data becomes
sparse. This sparsity makes it difficult for any model to find reliable patterns.
3. In a high-dimensional space, what happens to the distance between any two
randomly selected points as the number of dimensions (p) approaches infinity?
A) The distances become more distinct and spread out.
B) The distances all converge to a similar value.
C) The distances become increasingly random with no discernible pattern.
D) The distances decrease to zero.
Answer: B
,Rationale: This is a classic result of the curse of dimensionality. The ratio of the
maximum to minimum distance between points approaches 1, meaning that all
points become nearly equidistant from each other, making distance-based methods
like k-NN ineffective.
4. Which of the following is a direct consequence of the curse of dimensionality
in supervised learning?
A) The bias of a model decreases as dimensionality increases.
B) The variance of a model decreases as dimensionality increases.
C) The risk of overfitting increases because the data becomes sparse.
D) The computational time for training always decreases.
Answer: C
Rationale: With high dimensionality, the feature space is vast and the data points are
sparse. This makes it easy for a flexible model to find spurious patterns that fit the
training data perfectly but do not generalize, leading to overfitting.
5. The "large p, small n" problem is a common challenge in fields like genomics
and text analysis because:
A) The number of samples (n) is typically in the millions.
B) The number of features (p) can be in the thousands or millions, while the number
of samples (n) is often limited.
C) The data is always categorical.
D) The features are always independent of each other.
Answer: B
Rationale: In genomics, for example, you might have gene expression data for
thousands of genes (p) but only for a few hundred patients (n). This is the classic
"large p, small n" scenario.
6. Which of the following methods is NOT typically used to mitigate the curse
of dimensionality?
A) Feature selection
B) Dimensionality reduction
C) Adding more polynomial features
D) Regularization
Answer: C
Rationale: Adding more features (like polynomial features) exacerbates the curse of
dimensionality. The other three options are standard techniques to reduce
dimensionality or control model complexity.
7. What is the "intrinsic dimensionality" of a dataset?
A) The number of features (p) in the dataset.
B) The number of observations (n) in the dataset.
C) The minimum number of parameters needed to represent the data without
significant loss of information.
, D) The number of principal components required to explain 100% of the variance.
Answer: C
Rationale: Intrinsic dimensionality is the true, underlying dimensionality of the data,
which is often much lower than the ambient dimensionality (the number of measured
features). Manifold learning techniques aim to discover this.
8. A dataset of images, where each image is 100x100 pixels, has an ambient
dimensionality of:
A) 100
B) 200
C) 10,000
D) 1,000,000
Answer: C
Rationale: Each image is a point in a 100 * 100 = 10,000-dimensional space. The
intrinsic dimensionality, however, may be much lower (e.g., determined by the
number of objects, poses, lighting conditions, etc.).
9. The phenomenon where the distance to the nearest neighbor becomes
almost the same as the distance to the farthest neighbor in high dimensions is
known as:
A) Distance concentration
B) The empty space phenomenon
C) Hubness
D) The Johnson-Lindenstrauss lemma
Answer: A
Rationale: Distance concentration is the formal term for the effect where the
contrast between distances disappears as dimensionality grows.
10. Which of the following is a key assumption made by many classical
statistical methods that is violated in the high-dimensional setting?
A) The data is normally distributed.
B) The features are independent.
C) The number of observations is greater than the number of features (n > p).
D) The data is homoscedastic.
Answer: C
Rationale: Many classical methods, like Ordinary Least Squares (OLS) regression,
assume that the design matrix is of full column rank, which requires n ≥ p. In high
dimensions, n < p is common, making OLS ill-posed.
11. The Johnson-Lindenstrauss lemma states that:
A) A set of points in high-dimensional space can be embedded into a lower-
dimensional space while approximately preserving the distances between them.
B) Any high-dimensional dataset can be perfectly reconstructed from its principal
components.
Analytics - Final Exam with
questions and correct answers and
rationale updated 2026 graded A+
Section 1: Foundations and the Curse of Dimensionality (Questions 1-15)
1. What is the primary characteristic that defines "high-dimensional data" in
the context of ISyE 6525?
A) The number of observations (n) is very large.
B) The number of features (p) is very large.
C) The data is stored in a distributed system.
D) The data requires more than 1 terabyte of storage.
Answer: B
Rationale: High-dimensional data is defined by a large number of features or
variables (p), often with p being comparable to or even larger than the number of
observations (n). This is the "large p, small n" paradigm.
2. The "curse of dimensionality," a term coined by Richard Bellman, refers to:
A) The increased computational cost of analyzing data with many features.
B) The exponential increase in the volume of the feature space as dimensionality
increases.
C) The difficulty in visualizing data with more than three dimensions.
D) The tendency for models to become more accurate as more features are added.
Answer: B
Rationale: The curse of dimensionality primarily refers to the fact that the volume of
the space increases so fast with dimensionality that the available data becomes
sparse. This sparsity makes it difficult for any model to find reliable patterns.
3. In a high-dimensional space, what happens to the distance between any two
randomly selected points as the number of dimensions (p) approaches infinity?
A) The distances become more distinct and spread out.
B) The distances all converge to a similar value.
C) The distances become increasingly random with no discernible pattern.
D) The distances decrease to zero.
Answer: B
,Rationale: This is a classic result of the curse of dimensionality. The ratio of the
maximum to minimum distance between points approaches 1, meaning that all
points become nearly equidistant from each other, making distance-based methods
like k-NN ineffective.
4. Which of the following is a direct consequence of the curse of dimensionality
in supervised learning?
A) The bias of a model decreases as dimensionality increases.
B) The variance of a model decreases as dimensionality increases.
C) The risk of overfitting increases because the data becomes sparse.
D) The computational time for training always decreases.
Answer: C
Rationale: With high dimensionality, the feature space is vast and the data points are
sparse. This makes it easy for a flexible model to find spurious patterns that fit the
training data perfectly but do not generalize, leading to overfitting.
5. The "large p, small n" problem is a common challenge in fields like genomics
and text analysis because:
A) The number of samples (n) is typically in the millions.
B) The number of features (p) can be in the thousands or millions, while the number
of samples (n) is often limited.
C) The data is always categorical.
D) The features are always independent of each other.
Answer: B
Rationale: In genomics, for example, you might have gene expression data for
thousands of genes (p) but only for a few hundred patients (n). This is the classic
"large p, small n" scenario.
6. Which of the following methods is NOT typically used to mitigate the curse
of dimensionality?
A) Feature selection
B) Dimensionality reduction
C) Adding more polynomial features
D) Regularization
Answer: C
Rationale: Adding more features (like polynomial features) exacerbates the curse of
dimensionality. The other three options are standard techniques to reduce
dimensionality or control model complexity.
7. What is the "intrinsic dimensionality" of a dataset?
A) The number of features (p) in the dataset.
B) The number of observations (n) in the dataset.
C) The minimum number of parameters needed to represent the data without
significant loss of information.
, D) The number of principal components required to explain 100% of the variance.
Answer: C
Rationale: Intrinsic dimensionality is the true, underlying dimensionality of the data,
which is often much lower than the ambient dimensionality (the number of measured
features). Manifold learning techniques aim to discover this.
8. A dataset of images, where each image is 100x100 pixels, has an ambient
dimensionality of:
A) 100
B) 200
C) 10,000
D) 1,000,000
Answer: C
Rationale: Each image is a point in a 100 * 100 = 10,000-dimensional space. The
intrinsic dimensionality, however, may be much lower (e.g., determined by the
number of objects, poses, lighting conditions, etc.).
9. The phenomenon where the distance to the nearest neighbor becomes
almost the same as the distance to the farthest neighbor in high dimensions is
known as:
A) Distance concentration
B) The empty space phenomenon
C) Hubness
D) The Johnson-Lindenstrauss lemma
Answer: A
Rationale: Distance concentration is the formal term for the effect where the
contrast between distances disappears as dimensionality grows.
10. Which of the following is a key assumption made by many classical
statistical methods that is violated in the high-dimensional setting?
A) The data is normally distributed.
B) The features are independent.
C) The number of observations is greater than the number of features (n > p).
D) The data is homoscedastic.
Answer: C
Rationale: Many classical methods, like Ordinary Least Squares (OLS) regression,
assume that the design matrix is of full column rank, which requires n ≥ p. In high
dimensions, n < p is common, making OLS ill-posed.
11. The Johnson-Lindenstrauss lemma states that:
A) A set of points in high-dimensional space can be embedded into a lower-
dimensional space while approximately preserving the distances between them.
B) Any high-dimensional dataset can be perfectly reconstructed from its principal
components.
