## Problem You are building a data deduplication pipeline. Given a list of text records, identify and remove near-duplicates. Two records are near-duplicates if their normalized Jaccard similarity on character 3-grams exceeds a threshold `t`. Return the deduplicated list, keeping the first occurrence of each cluster. ```python def deduplicate( records: list[str], threshold: float = 0.8 ) -> list[str]: pass def jaccard_3gram(a: str, b: str) -> float: """Return Jaccard similarity of 3-gram sets.""" pass ``` ``` Input: records = ["hello world", "Hello World!", "foo bar", "hello world "] threshold = 0.7 Output: ["hello world", "foo bar"] # After normalization, records 0,1,3 cluster together -> keep first. Input: records = ["abc", "xyz"] Output: ["abc", "xyz"] ``` ## Follow-ups 1. Naive O(n^2) comparison is too slow for large datasets. How does MinHash + LSH reduce this to near-linear? 2. How do you choose the 3-gram size? What are the tradeoffs of 2-grams vs 4-grams? 3. If records are structured (e.g., JSON objects), how would you define similarity across fields? 4. How would you implement this as an incremental pipeline where new records stream in daily?

InterviewDB Question · Paris

Deduplication Pipeline: Remove Near-Duplicate Records Using Hashing

phone

Question Details

Problem You are building a data deduplication pipeline. Given a list of text records, identify and remove near-duplicates. Two records are near-duplicates if their normalized Jaccard similarity on character 3-grams exceeds a threshold t. Return the deduplicated list, keeping the first occurrence of each cluster. Follow-ups Naive O(n^2) comparison is too slow for large datasets. How does MinHash + LSH reduce this to near-linear? How do you choose the 3-gram size? What are the tradeoffs of 2-grams…

Full Details

🔒

Unlock all Perplexity questions

Full insider details, leaked discussions, and candidate experiences.

Get full access — from $50/mo

About This Question

This is a reported interview question from a perplexity interview during the phone round.

It covers the following topics: Coding, Mle, Phone .

Topics

Coding Mle Phone

About Perplexity Interview Reports

This question was reported by a candidate who interviewed at Perplexity. LeakCode aggregates interview reports from 10+ sources, including 1Point3Acres, Glassdoor, LeetCode Discuss, Blind, Reddit, Indeed, and Nowcoder. Each report is translated where necessary, deduplicated against existing entries, and tagged by company, role, round type, and reporting date.

Use this question as one calibration data point, not a memorization target. Companies typically rotate their question pools every 2-4 months; the exact wording of a 2024 question may differ from what you encounter today. The underlying pattern, difficulty level, and follow-up depth at Perplexity are the higher-signal extractions to take from this report.

For broader preparation context, the Perplexity interview process typically includes a recruiter screen, one or two technical phone screens, and a 4-5 round on-site loop covering coding, system design (at L4+ levels), and behavioral. Reports tagged on LeakCode show the round-by-round distribution and typical difficulty calibration. To browse questions filtered by round type and seniority, use the company hub linked above.

How To Practice This Type of Question

Solve similar problems on LeetCode under timed conditions (25-35 minutes per medium difficulty). The goal is pattern recognition: recognize the underlying technique (sliding window, two-pointer, BFS, memoized recursion, etc.) within 60-90 seconds of reading. Strong candidates verbalize their hypothesis out loud before coding, then iterate based on feedback. Weak candidates dive into implementation immediately, lose time on the wrong approach, and run out of time for follow-ups.

Companies update their question pools every 2-4 months. The exact wording of any given question may have been retired by the time you interview. Focus your prep on the pattern, not the specific problem. The patterns that appear in Perplexity reports consistently are the ones worth investing in; one-off niche problems are not.

During Your Perplexity Round

Apply the standard interview round template: clarify requirements (2-3 minutes), state your approach out loud and confirm direction with the interviewer (3-5 minutes), code with narration (15-25 minutes), test with concrete examples including edge cases (5 minutes), discuss optimization or trade-offs if time permits (5 minutes). This template is universally accepted across FAANG and adjacent companies; deviating from it produces weaker interviewer feedback signal.

The single most predictive failure mode in Perplexity reports tagged "no hire": not asking clarifying questions. Interviewers are explicitly trained to weight this. Strong candidates ask 3-5 clarifying questions even on problems that look obvious; weak candidates dive into code immediately. The clarifying-question check is often the first signal recorded in the interviewer's written notes.