PCA Questions and Answers

PCA Questions and Answers Question: PCA stands for Principal _________ Analysis. Component Difficulty: Easy Explanation: PCA stands for Principal Component Analysis. It's a dimension reduction technique that enables you to identify the most significant underlying structure of your data. Question: PCA aims to find a new set of dimensions such that all the dimensions are _________ and the variance is maximized. Orthogonal Difficulty: Medium Explanation: In PCA, the goal is to find a set of new dimensions that maximize variance and are orthogonal to (independent of) each other. This is achieved by projecting the data onto new axes, which are called principal components. Previous Play Next Rewind 10 seconds Move forward 10 seconds Unmute 0:00 / 0:15 Full screen Brainpower Read More Question: The first principal component explains the _________ amount of variance in the data. Largest Difficulty: Medium Explanation: The first principal component is the direction in the data that explains the largest amount of variance. Each subsequent component explains the maximum variance possible under the constraint that it is orthogonal to the preceding components. Question: PCA assumes that the data variables have a _________ distribution. Gaussian or Normal Difficulty: Medium Explanation: PCA is a method that relies on linear algebra and is generally more suited for data that has a linear structure. It assumes that the data variables have a Gaussian (or Normal) distribution. Question: The matrix that contains all the principal components is called the _________ matrix. Eigenvector Difficulty: Hard Explanation: An eigenvector is a direction. In the context of PCA, it is a direction in the space of input features. The collection of all eigenvectors creates a matrix that we can use to transform our data. Question: PCA is often used before applying a machine learning algorithm to _________ the dimensionality of the input data. Reduce Difficulty: Medium Explanation: PCA is a method that reduces the dimensionality of the data while retaining most of the important information. It is often used as a pre-processing step before applying a machine learning algorithm, as it can reduce computational cost and alleviate the problem of overfitting. Question: The _________ of a matrix are used in PCA to form the new feature dimensions. Eigenvectors Difficulty: Medium Explanation: PCA involves computing the eigenvectors of the covariance matrix of the data. These eigenvectors form the new axes or dimensions of the data, which are used to project the original data points into the new feature space. Question: In PCA, the amount of variance that each PC explains is quantified by the corresponding _________. Eigenvalue Difficulty: Hard Explanation: In PCA, each principal component (PC) has a corresponding eigenvalue, which is a measure of the amount of variance in the data that is explained by that PC. Higher eigenvalues correspond to higher amounts of explained variance. Question: The principal components of PCA are selected based on their corresponding _________ values. Eigenvalue Difficulty: Medium Explanation: The eigenvalues corresponding to each of the principal components determine the amount of variance captured by each component. The components are sorted according to the eigenvalues, in decreasing order, and the top few components (those with the largest eigenvalues) are selected. Question: PCA tends to perform poorly when the data variables have _________ relationships. Non-linear Difficulty: Hard Explanation: PCA is a linear method and assumes linear relationships among variables. It tends to perform poorly when the relationships among variables are non-linear. For data with non-linear relationships, other methods like Kernel PCA or non-linear dimension reduction methods might be more suitable. Question: One of the main uses of PCA in machine learning is for _________ reduction, which helps to alleviate the curse of dimensionality. Feature Difficulty: Medium Explanation: PCA is commonly used for feature reduction in machine learning. By transforming the data to a lower-dimensional space, PCA can help alleviate the curse of dimensionality, which is the problem that arises when dealing with high-dimensional data, such as increased computational complexity and overfitting. Question: PCA is a form of _________ learning, since it learns the components based solely on the input data, without using any class label information. Unsupervised Difficulty: Medium Explanation: PCA is an unsupervised method because it doesn't use any class label information. Instead, it transforms the input data based solely on the variance and structure within the data itself. Question: In PCA, the dimensionality of the data is reduced by projecting onto the _________. Principal Components Difficulty: Medium Explanation: In PCA, the dimensionality of the data is reduced by projecting the data onto the axes defined by the principal components. These principal components are vectors that define a new coordinate system for the data. Question: In PCA, the principal components are derived from the _________ of the data matrix. Covariance Difficulty: Hard Explanation: In PCA, the principal components are the eigenvectors of the covariance matrix of the data. The covariance matrix provides information about the variance of the data and the correlations between different dimensions. Question: When using PCA, data is often _________ before calculating the covariance matrix to ensure each feature has the same scale. Standardized Difficulty: Medium Explanation: Prior to performing PCA, data is often standardized. This is because PCA is sensitive to the variances of the initial variables. If there are large differences in the scales (ranges) of the variables, those with larger scales will dominate over those with small scales. Question: The axes or principal components in PCA are chosen to maximize the _________ of the projected data. Variance Difficulty: Medium Explanation: The goal of PCA is to identify the axes or principal components in the feature space that account for the maximum variance in the data. By projecting the data onto these axes, PCA reduces the dimensionality of the data while retaining as much of the data's variation as possible. Question: The number of principal components that one can obtain from a PCA equals to the number of _________ in the original data. Features Difficulty: Hard Explanation: The number of principal components that can be obtained from PCA equals to the number of features in the original data. However, often only the first few principal components that explain the most variance are used. Question: When visualizing high-dimensional data, one common use of PCA is to project the data down to two or three dimensions, which are the first two or three _________. Principal Components Difficulty: Medium Explanation: PCA is often used to visualize high-dimensional data. By projecting the data onto the first two or three principal components, one can create a 2D or 3D plot of the data. This allows one to visually inspect the structure and distribution of the data. Question: PCA is sensitive to the _________ of the variables, hence it is common to normalize the data before applying PCA. Scale Difficulty: Medium Explanation: PCA is sensitive to the scale of the variables in the dataset. If variables are of different scales, it may cause PCA to assign more importance to variables with a larger scale. Therefore, data is usually normalized or standardized before applying PCA. Question: If two variables are perfectly _________, then they will lie along the same principal component. Correlated Difficulty: Medium Explanation: If two variables are perfectly correlated, they essentially contain the same information. PCA will then reduce them to a single principal component, removing the redundancy. Question: PCA is based on the _________ matrix of the data when the data is zero centered. Covariance Difficulty: Medium Explanation: PCA is a technique that relies on the computation of the covariance matrix of the data. It is a method that transforms the original variables to a new set of variables, which are a linear combination of the original variables. Question: PCA can be affected by _________ in the data, which are extreme values that deviate significantly from the other observations. Outliers Difficulty: Hard Explanation: PCA can be significantly affected by outliers in the data. This is because PCA is based on the covariance matrix of the data and the calculation of the covariance is sensitive to outliers. Therefore, it is often recommended to handle outliers before applying PCA. Question: A potential drawback of using PCA in predictive modeling is that the principal components may be hard to _________ in terms of the original features. Interpret Difficulty: Hard Explanation: One of the downsides of PCA is that the principal components may not have a clear interpretation. They are linear combinations of the original features, and it may be hard to relate them back to the original features. Question: While PCA aims to find orthogonal directions of maximum variance, the related technique called _________ Component Analysis aims to find non-orthogonal directions. Independent Difficulty: Hard Explanation: Independent Component Analysis (ICA) is a technique that, unlike PCA, aims to find non-orthogonal (independent but not necessarily orthogonal) directions in the data. ICA is based on the statistical concept of independent components, rather than uncorrelated directions.