ARTIBA Artificial Intelligence Engineer Certification
Practice Exam
1. A bank trains a loan-default classifier on historical records. Only 2% of
borrowers default. The model achieves 98% accuracy by predicting no
default for everyone. Which evaluation approach is most informative?
A) Training accuracy alone
B) Precision-recall analysis for the default class
C) Number of training epochs
D) Mean squared error of customer ages
Answer:
B) Precision-recall analysis for the default class
Rationale:
Accuracy can be misleading for imbalanced classification. Precision and recall
measure how effectively the model identifies rare defaults while controlling false
alarms.
2. A hospital trains a model to predict readmission using a field
indicating whether a readmission follow-up appointment was scheduled
after discharge. The field is unavailable at prediction time. What is the
main problem?
A) Underfitting
B) Excessive regularization
C) Target leakage
D) Class imbalance
Answer:
C) Target leakage
Rationale:
Information unavailable when a prediction is made must not enter training features.
Leakage produces unrealistically favorable evaluation results.
,3. A retail company trains a demand forecasting model using randomly
shuffled daily records. Performance is excellent, but production
forecasts are poor. What should be changed?
A) Use chronological training and validation splits
B) Increase batch size
C) Remove all seasonal features
D) Replace regression with clustering
Answer:
A) Use chronological training and validation splits
Rationale:
Random splits can expose a forecasting model to future patterns. Time-ordered
validation better approximates production forecasting.
4. A model performs well on training data but substantially worse on
unseen test data. What is the most likely issue?
A) High bias and low variance
B) Perfect calibration
C) Insufficient model capacity
D) Overfitting
Answer:
D) Overfitting
Rationale:
A large train-test performance gap suggests the model has learned training-specific
patterns rather than generalizable relationships.
,5. A data scientist applies standardization to an entire dataset before
dividing it into training and test sets. Why is this problematic?
A) Standardization cannot be used with regression
B) Test-set statistics leak into model development
C) Standardization always removes outliers
D) The model becomes unsupervised
Answer:
B) Test-set statistics leak into model development
Rationale:
The scaler must be fitted using training data only and then applied unchanged to
validation and test data.
6. A fraud classifier predicts probabilities of 0.9 for transactions that turn
out to be fraudulent only 60% of the time. Which property needs
improvement?
A) Feature dimensionality
B) Training throughput
C) Probability calibration
D) Model serialization
Answer:
C) Probability calibration
Rationale:
A calibrated classifier's predicted probabilities should approximately match observed
event frequencies.
, 7. A logistics team needs to group delivery locations without predefined
labels. Which approach is appropriate?
A) Clustering
B) Supervised classification
C) Logistic regression on target labels
D) Policy-gradient reinforcement learning
Answer:
A) Clustering
Rationale:
Clustering discovers structure among unlabeled observations.
8. A customer segmentation project uses K-means on variables
measured in dollars, kilometers, and percentages. The clusters are
dominated by the dollar-valued variable. What is the best first
correction?
A) Increase the number of clusters indefinitely
B) Remove all numeric variables
C) Replace Euclidean distance with accuracy
D) Scale features appropriately
Answer:
D) Scale features appropriately
Rationale:
Distance-based methods are sensitive to feature magnitude. Scaling prevents
large-unit variables from dominating distances.
Practice Exam
1. A bank trains a loan-default classifier on historical records. Only 2% of
borrowers default. The model achieves 98% accuracy by predicting no
default for everyone. Which evaluation approach is most informative?
A) Training accuracy alone
B) Precision-recall analysis for the default class
C) Number of training epochs
D) Mean squared error of customer ages
Answer:
B) Precision-recall analysis for the default class
Rationale:
Accuracy can be misleading for imbalanced classification. Precision and recall
measure how effectively the model identifies rare defaults while controlling false
alarms.
2. A hospital trains a model to predict readmission using a field
indicating whether a readmission follow-up appointment was scheduled
after discharge. The field is unavailable at prediction time. What is the
main problem?
A) Underfitting
B) Excessive regularization
C) Target leakage
D) Class imbalance
Answer:
C) Target leakage
Rationale:
Information unavailable when a prediction is made must not enter training features.
Leakage produces unrealistically favorable evaluation results.
,3. A retail company trains a demand forecasting model using randomly
shuffled daily records. Performance is excellent, but production
forecasts are poor. What should be changed?
A) Use chronological training and validation splits
B) Increase batch size
C) Remove all seasonal features
D) Replace regression with clustering
Answer:
A) Use chronological training and validation splits
Rationale:
Random splits can expose a forecasting model to future patterns. Time-ordered
validation better approximates production forecasting.
4. A model performs well on training data but substantially worse on
unseen test data. What is the most likely issue?
A) High bias and low variance
B) Perfect calibration
C) Insufficient model capacity
D) Overfitting
Answer:
D) Overfitting
Rationale:
A large train-test performance gap suggests the model has learned training-specific
patterns rather than generalizable relationships.
,5. A data scientist applies standardization to an entire dataset before
dividing it into training and test sets. Why is this problematic?
A) Standardization cannot be used with regression
B) Test-set statistics leak into model development
C) Standardization always removes outliers
D) The model becomes unsupervised
Answer:
B) Test-set statistics leak into model development
Rationale:
The scaler must be fitted using training data only and then applied unchanged to
validation and test data.
6. A fraud classifier predicts probabilities of 0.9 for transactions that turn
out to be fraudulent only 60% of the time. Which property needs
improvement?
A) Feature dimensionality
B) Training throughput
C) Probability calibration
D) Model serialization
Answer:
C) Probability calibration
Rationale:
A calibrated classifier's predicted probabilities should approximately match observed
event frequencies.
, 7. A logistics team needs to group delivery locations without predefined
labels. Which approach is appropriate?
A) Clustering
B) Supervised classification
C) Logistic regression on target labels
D) Policy-gradient reinforcement learning
Answer:
A) Clustering
Rationale:
Clustering discovers structure among unlabeled observations.
8. A customer segmentation project uses K-means on variables
measured in dollars, kilometers, and percentages. The clusters are
dominated by the dollar-valued variable. What is the best first
correction?
A) Increase the number of clusters indefinitely
B) Remove all numeric variables
C) Replace Euclidean distance with accuracy
D) Scale features appropriately
Answer:
D) Scale features appropriately
Rationale:
Distance-based methods are sensitive to feature magnitude. Scaling prevents
large-unit variables from dominating distances.
