Chat Messages: Design a Scalable Chat Message Storage and Retrieval System

Question

## Round 1 - System Design

## Problem

Design the backend storage and retrieval system for a chat application. Requirements:
- Users can send messages in 1:1 and group conversations
- Retrieve messages for a conversation paginated by time (newest-first)
- Support message search by keyword within a conversation
- Message delivery guarantees: at-least-once; UI deduplicates by `message_id`
- Scale: 100M DAU, 10B messages/day

## Key Design Decisions

- **Data model**: `(conversation_id, message_id, sender_id, content, ts)` — partition by `conversation_id`
- **Storage**: Cassandra or HBase (write-heavy, time-series, partition by conv)
- **Pagination**: cursor-based using `(ts, message_id)` as the cursor
- **Search**: async index to Elasticsearch; acceptable lag of seconds
- **Fanout**: for group chats, write to each member's inbox vs. single conv store + pull

## Follow-ups

1. How do you handle message ordering across multiple senders in the same conversation?
2. What are the trade-offs between push fanout vs. pull-on-read for group messages?
3. How do you implement end-to-end encryption and what does that mean for search?

Chat Messages: Design a Scalable Chat Message Storage and Retrieval System

Question Details

Full Details

Topics

More from Reddit