ML Modeling: Design and Evaluate a Click-Through Rate Prediction Model

Question

## Problem

You are asked to build a CTR (click-through rate) prediction model for a content recommendation system. Walk through the full ML modeling process:

**1. Problem framing**
- Binary classification: will user click on item? (positive = click)
- Training signal: implicit feedback (clicks), with heavy class imbalance (~1% CTR)

**2. Feature engineering**
- User features: historical CTR, session recency, demographics
- Item features: category, age, historical CTR
- Context features: device, time-of-day, position bias

**3. Model options**
- Logistic Regression (baseline, interpretable)
- Gradient Boosted Trees (GBDT) for tabular features
- Deep factorization machines or two-tower neural model for large sparse IDs

**4. Evaluation**
- Metrics: AUC-ROC, log-loss, calibration
- Why accuracy is a poor metric at 1% CTR

## Follow-ups

1. How do you handle position bias in training data (items shown higher get more clicks)?
2. How do you evaluate the model offline before an A/B test?
3. The model's calibration drifts after 2 weeks — what causes this and how do you fix it?

ML Modeling: Design and Evaluate a Click-Through Rate Prediction Model

Question Details

Full Details

Topics

More from Reddit