Model Distribution: Design a System for Distributing ML Models to Edge Devices

Question

## Round 1 - System Design

## Problem

Design a system that manages and distributes machine learning models to a fleet of edge devices (e.g., mobile phones, IoT sensors). The system must:
- Allow data scientists to upload new model versions
- Roll out models to device segments (e.g., 10% canary -> 50% -> 100%)
- Track which model version each device is running
- Support rollback if error rate spikes

## Key Components to Cover

- **Model registry**: versioning, metadata, storage (e.g., S3 + database)
- **Device registry**: device -> current version mapping, last heartbeat
- **Rollout controller**: segment targeting, gradual percentage rollout, auto-rollback triggers
- **Update delivery**: push vs. pull model; delta updates for large models
- **Monitoring**: per-version error rates, latency, adoption metrics

## Follow-ups

1. How do you handle devices that are offline for weeks and miss multiple version jumps?
2. What consistency guarantees does the device registry need? Is eventual consistency acceptable?
3. How do you sign and verify model artifacts to prevent tampering on-device?
4. If model files are 500 MB, how do you minimize bandwidth cost during rollout?

Model Distribution: Design a System for Distributing ML Models to Edge Devices

Question Details

Full Details

Topics

More from Anthropic