How to Pass the Data Engineering System Design Interview: Notes from the Interviewer’s Side
A repeatable structure for the 45-minute design round, how to handle “we need real-time”, and the failure-mode follow-ups that decide the result.
The system design round is the interview candidates fear most, and I understand why. The question is vague on purpose (“Design the analytics platform for a food delivery app”), there’s no single correct answer, and you have 45 minutes and a whiteboard.
I’ve been on the other side of that whiteboard many times, interviewing data engineers and DevOps engineers. What I can tell you is that the result is rarely decided by which tools you draw. It’s decided by how you get to the design and what you say when I try to break it.
This article covers both: the structure I hope to see, the “real-time” question that comes up in almost every design round, and the follow-ups I use to find the edges of a candidate’s experience.
What the interviewer is actually grading
When I write my notes after a design round, they’re usually about four things:
- Did they clarify before designing? Requirements, consumers, freshness, correctness.
- Did they size the problem? Even rough numbers change the design.
- Can they explain trade-offs? Every box should have a “because”.
- Do they think about failure? Late data, duplicates, backfills, schema changes, on-call.
Tool knowledge matters, but it’s the easiest part to learn later. Judgment is the part that’s hard to teach.
A structure that works
Here’s the sequence I recommend. It’s not the only one, but having any fixed structure keeps you from freezing or rambling.
- Clarify consumers and requirements (5 to 8 minutes). Who uses the data and for what? Finance reporting needs exact numbers but daily is fine. An operations dashboard needs minutes-fresh data, approximate is OK. ML features need point-in-time correctness. Ask about retention, PII and compliance.
- Size it. Back-of-the-envelope, out loud.
- Sources and ingestion. Database changes via CDC, app events via a stream, third-party APIs via scheduled extractors. For each: how does it handle duplicates, late data and schema changes?
- Storage and modeling. Raw immutable landing zone, a cleaned layer, then dimensional marts. Name the main fact tables and their grain.
- Processing. Batch for finance-grade facts, streaming only where it’s required. Orchestration, idempotent tasks, backfill strategy.
- Serving. BI on the marts, a low-latency store for product features, reverse ETL if operational tools need data.
- Reliability and operations. Data quality checks, freshness monitoring, lineage, access control, cost.
- Evolution. “At 10x volume I would change…”, “With a team of three I would drop…”.
What sizing sounds like
Candidates often skip sizing because they fear getting the numbers wrong. The numbers don’t need to be right; they need to be stated, so the design can follow from them. For example:
“Let’s assume 2 million orders a day, and around 30 status and courier-location events per order. That’s 60 million events a day, about 700 per second on average. Dinner peaks might be five times that, about 3,500 events per second. At roughly 1 KB per event, that’s about 60 GB of raw data per day.”
These are invented assumptions for the exercise, and that’s fine, as long as you say so and ask the interviewer whether they’re realistic. What matters is the conclusion you draw: even at a dinner peak of about 3,500 events per second, this is comfortably handled by a managed streaming service or even micro-batches into a warehouse. It doesn’t need a custom distributed system.
The question that always comes: “We need real-time”
At some point, often in the clarify phase, the interviewer (or the scenario) says the business wants “real-time analytics”. This is a test.
Weak answer
“Then we use Kafka and Flink, because streaming is real-time.”
Strong answer
“What decision gets made faster if the data is five minutes old instead of an hour old? If a person looks at the dashboard twice a day, hourly batch is enough. If a system acts on each event, like blocking a fraudulent payment, we need streaming. Each step down in latency adds cost and operational work, so I’d pick the slowest option that still meets the requirement, and probably only for the few metrics that need it.”
A comparison you can sketch quickly:
- Batch: latency of hours; reprocessing = rerun the job; hard parts = scheduling and backfills; cost = compute while running.
- Micro-batch: seconds to minutes; replay from source offsets; small files and per-run overhead; frequent or always-on compute.
- Streaming: sub-second to seconds; replay with state (harder); state, watermarks and ordering; always-on compute plus state.
Two points that make this answer stronger:
Get itcraftsman’s stories in your inbox
Join Medium for free to get updates from this writer.
Know the history of the debate. Running a batch layer and a speed layer with separate code (the “Lambda architecture”) means implementing the same logic twice. Jay Kreps’ Questioning the Lambda Architecture (2014) proposed a single stream-processing code path that reprocesses by replaying the log, which became known as the Kappa architecture. You don’t need to name either, but the idea of “one code path where possible” is exactly what I want to hear.
Know that warehouses moved closer to streaming. Many requirements that sound like streaming can now be met with streaming ingestion plus incremental transforms. Examples: Snowpipe Streaming and Dynamic Tables with a target lag in Snowflake, or the BigQuery Storage Write API. Saying “I’d check whether minute-level freshness in the warehouse is enough before introducing a stream processor” shows cost awareness.
Follow-up I ask: “What breaks first if you go from hourly batch to five-minute micro-batches?” Good answers: many small files and growing metadata, fixed per-run overhead (startup, a MERGE that scans the target) suddenly dominating, twelve times more runs for the orchestrator, and more late data per run because each window is less complete.
The failure-mode follow-ups
Once a design is on the board, I start breaking it. These are the questions that separate people who have run pipelines in production from people who have read about them.
“An event arrives two days late. What happens to yesterday’s numbers?”
A weak answer is “it gets added to today”. A strong answer names a policy. In batch: reprocess a lookback window and deduplicate on an event id, and decide whether published reports get restated. In streaming: an event-time watermark defines how long the job waits for late data before it finalises a window and drops state.
In Spark Structured Streaming, that looks like this:
from pyspark.sql.functions import window, col
orders_per_minute = (
order_events
.withWatermark("event_ts", "30 minutes")
.groupBy(window(col("event_ts"), "1 minute"), col("city"))
.count()
)The Structured Streaming guide explains how late events update older windows, and how the watermark bounds the state the engine keeps. The trade-off to say out loud: a longer watermark means more complete results but more state and later final numbers. Events later than the watermark need a separate path, often the nightly batch that reconciles against the source.
“How do you guarantee no double counting?”
I’m listening for “exactly-once effect”, not “Kafka has exactly-once”. Delivery across a network is at-least-once in practice, so correctness comes from stable record IDs plus idempotent writes: MERGE by key, overwrite of deterministic partitions, or transactions that commit the input offset together with the output. A candidate who adds “and the ID must come from the source, not from a uuid() generated during processing, or a retry creates new IDs" has clearly debugged this before.
“You’re reading changes from the production Postgres via CDC. What can go wrong?”
My favourite answer mentions replication slots. With log-based CDC (Debezium or a managed connector reading PostgreSQL’s logical replication), the slot keeps the write-ahead log until the consumer confirms it. The PostgreSQL documentation warns that slots prevent removal of required WAL even when no consumer is connected, so if a slot is no longer needed, it should be dropped. In plain words: if your CDC consumer stops and nobody notices, the database disk fills up. Candidates who say “I’d monitor slot lag and alert on it” get a tick from me.
“The revenue logic was wrong for a year. How do you backfill?”
Write the corrected history to a separate table in idempotent chunks, validate old against new, tell finance before the numbers change, then swap atomically. “Rerun the job in a loop over the production table” is the answer I don’t want to hear.
Common mistakes I see
- Drawing tools in the first minute, before anyone has said who uses the data.
- No numbers at all. “It’s big data” is not sizing.
- Every box without a reason. “Snowflake, because the team knows SQL and we don’t want to run infrastructure” beats “Snowflake”.
- Monologuing. Check in every few minutes: “Should I go deeper into ingestion or modeling?” The interviewer often has a part they want to explore, and they’ll tell you if you ask.
- Forgetting operations. Who gets paged at 3 a.m., and how do they know which table is wrong?
How to practise
Pick three scenarios: an e-commerce analytics platform, a clickstream pipeline, and a CDC-based replica of an operational database. For each, set a 45-minute timer and talk through the eight steps out loud, ideally to another person. Then ask yourself the four failure questions above: late data, duplicates, CDC breakage, backfill. If any of them makes you hesitate, that’s the part to study.
The design you draw matters less than you think. The reasoning you say out loud is what I write down.
If you want structured practice: my Data Engineering Interview Kit 2026 has a system design chapter (framework, batch vs streaming, CDC, exactly-once, backfills, Kafka, data contracts, cost, GDPR deletion) and 10 worked design and modeling exercises, answered step by step. The free sample with 20 complete questions, including the design framework and the batch-vs-streaming question, is here: https://payhip.com/b/18A5n. The full kit is available as Basic and Pro; the code LAUNCH25 gives 25% off until October 18, 2026, 21:00 CEST.
About me: I’m Toni, an independent data engineer and DevOps freelancer (IT-Craftsman). I have delivered data engineering and DevOps projects for large companies, and I conduct hiring interviews for data engineers and DevOps engineers at a technology consulting firm.
