A fraud model reports 99% accuracy when only 1% of transactions are fraudulent. How would you evaluate it?
Interactive Practice Mode
Formulate your thoughts or take notes before revealing the model solution.
Saved automatically
Model Answer & Deep-Dive Explanation
A classifier that always predicts legitimate already achieves 99% accuracy, so inspect the confusion matrix, fraud-class precision and recall, and precision-recall behavior across thresholds. Estimate the costs of missed fraud and unnecessary investigations, then select a threshold that fits operational capacity. Use a time-based evaluation if production predicts future transactions. Check calibration when scores guide decisions, and examine important customer segments. Report results at the chosen threshold rather than presenting one impressive aggregate metric without its business consequences.
Related Questions in AI & Machine Learning
Explain Retrieval-Augmented Generation (RAG) Architecture vs Model Fine-Tuning
Advanced level • 1858 views
Explain the Transformer Architecture and Multi-Head Self-Attention in LLMs
Advanced level • 1427 views
How do you detect and mitigate Overfitting vs Underfitting in deep neural networks?
Intermediate level • 1185 views
Explain the Self-Attention mechanism in Transformer architectures and how its computational complexity scales with sequence length.
Advanced level • 154 views
test
Medium level • 4 views