Distilled Prep

Interview prep, distilled from
engineering blogs.

Engineering blogs of major tech companies, distilled into interview preparation. Every question links back to the post that inspired it, and teaches the reasoning — not the trivia.

Meta
0 questions
Google
0 questions
Airbnb
0 questions
Stripe
1 question
Uber
2 questions
DoorDash
1 question
Netflix
1 question

Recently added

difficulty 4/5 Data scienceexperimentationcausal-inferencemarketplace-dynamics

A city launches a new driver incentive that guarantees a minimum hourly earning during peak periods. Design the evaluation plan to decide whether the program should be expanded to other cities.

Practice against the follow-up probes
  • What are your objectives and guardrails across riders, drivers, and the platform — and which one is the decision metric?
  • Would you randomize at the rider level, driver level, use switchbacks, geo experiments, difference-in-differences, or synthetic control? Why?
  • Drivers can see the incentive and migrate across zones or shift their hours in anticipation. How does that contaminate your design?
  • How do you estimate incrementality — supply that exists because of the program — rather than just measuring that supply went up?
  • The program has a fixed budget. How does that change what "success" means?
Show answer guide

What the interviewer is probing

Whether the candidate recognizes this as an interference problem before choosing a method: driver-level randomization is the intuitive answer and it's wrong here, because treated and control drivers compete in one marketplace and the incentive changes behavior of both. Strong candidates frame the business decision first (expand or not, at what cost per incremental supply hour), pick a design that matches the interference structure, and are honest about what each design cannot identify.

Strong answer outline

  • Frame the decision: expansion is a cost-effectiveness question — dollars per incremental supply-hour during peak, and downstream effects on rider wait times, completed trips, and driver retention. Guardrails: driver earnings dispersion, off-peak supply cannibalization, rider price effects.
  • Rule out naive designs aloud: driver-level randomization suffers spillovers (control drivers face more competition, inflating apparent effects); pre/post comparison confounds seasonality and city trends.
  • Prefer designs matching the interference structure: geo-randomization at the city or zone-cluster level if enough units exist; switchbacks (time-based alternation) if the effect onset/decay is fast; synthetic control or diff-in-diffs against comparable cities when only one launch city exists — which is the realistic case here.
  • Name anticipation and learning effects: announce dates create pre-period contamination; drivers take time to respond, so define a burn-in and an effect window.
  • Incrementality: compare supply-hours against the counterfactual, then decompose — new drivers activated vs. existing drivers shifting hours from off-peak (cannibalization) vs. hours shifted from neighboring zones (displacement). Only the first is cleanly incremental.
  • End with a decision rule: expand if cost per incremental peak supply-hour beats the alternative levers (e.g. surge subsidy) at comparable guardrail impact, validated in 2-3 structurally different cities before general rollout.

The underlying concept

The Stable Unit Treatment Value Assumption (SUTVA) — that one unit's treatment doesn't affect another unit's outcome — fails almost everywhere in a marketplace, because participants compete for shared demand and supply. When SUTVA fails, the choice of randomization unit is the real design decision: you must randomize at the level that contains the interference (zones, cities, time blocks), even though this slashes your sample size and statistical power. That power-vs-validity trade-off, and the discipline of measuring incrementality against a counterfactual rather than a before/after, is the core of applied marketplace causal inference.

Source

Distilled Prep canon — curated from Uber's public work on marketplace experimentation and causally-informed optimization.

difficulty 4/5 Data scienceML engforecastingmarketplace-dynamics

You're forecasting rider demand per city zone in 15-minute intervals to position drivers ahead of demand. A new city launches next month with no historical data. Your global model performs well in mature cities but the launch team needs zone-level forecasts from day one. How do you approach this, and how does your approach change over the city's first 90 days?

Show answer guide

What the interviewer is probing

Whether the candidate can reason about cold-start under real operational pressure: transfer from similar cities, use of covariates that exist before launch (population, POI density, events), honest uncertainty communication, and a plan for graduating from priors to learned patterns. Strong candidates also ask what decision the forecast feeds — positioning tolerance determines how much error is acceptable.

Strong answer outline

  • Clarify the decision consumer first: driver positioning tolerates different error than surge pricing; this sets the accuracy bar and the cost of over- vs under-forecasting (asymmetric loss).
  • Day 0: transfer learning — pool data from demographically/geographically similar cities; features available pre-launch (census, POI, weather, calendar) drive a global model's predictions for the new city.
  • Early weeks: hierarchical/Bayesian shrinkage — city-level estimates borrow strength from the global prior, weight shifting to local data as it accumulates; simple exponential blending is an acceptable pragmatic answer.
  • Instrumentation: define when the city "graduates" (e.g. local-only model beats blended on rolling backtest).
  • Traps to name: launch-period data is unrepresentative (promos, novelty); feedback loops (positioning drivers changes observed demand — you observe fulfilled demand, not true demand).

The underlying concept

Cold-start forecasting is fundamentally a bias-variance trade: with no local data, a global prior is high-bias but low-variance; local data is unbiased but high-variance early on. Hierarchical models formalize the blend, letting the data decide how quickly to trust local signal. The second deep idea is censored demand: a marketplace only observes demand it could serve, so naive training on fulfilled trips systematically underestimates demand exactly where supply was short — the places you most need to forecast well.

Source

Hand-written seed example in the style of Uber's marketplace forecasting posts.

difficulty 3/5 Data sciencepaymentscausal-inferenceexperimentation

Stripe launches a feature that automatically retries failed subscription payments on an optimized schedule. Determine whether it creates incremental recovered revenue without harming customers.

Practice against the follow-up probes
  • Many failed payments would have succeeded on the merchant's existing retry logic anyway. How do you avoid claiming credit for those?
  • What harm could this feature cause, and what guardrail metrics catch it?
  • Merchants differ wildly — subscription size, industry, existing dunning tools. How does heterogeneity change your analysis and your rollout?
  • The feature interacts with merchants' own retry configurations. How do you attribute recovery between the two systems?
  • What would the merchant-facing report claim, and how do you make sure it's honest?
Show answer guide

What the interviewer is probing

Whether the candidate instinctively reaches for a counterfactual rather than a raw recovery rate — the seductive wrong answer is "we recovered $X of failed payments," most of which would have been recovered anyway. Also probing harm-awareness: retries are not free (card network fees, decline-rate penalties, customers charged after intending to cancel) and strong candidates surface these unprompted. This is Stripe's flavor of DS: financial correctness and merchant trust as first-class constraints.

Strong answer outline

  • Define incrementality precisely: revenue recovered by this feature that would NOT have been recovered by the merchant's baseline process within the same window. The estimand is a difference against a counterfactual, not a recovery total.
  • Design: merchant-level randomization (the feature operates on merchant configuration, and payment-level randomization within a merchant leaks through shared retry budgets and issuer behavior). Stratify by merchant size, industry, and presence of existing dunning tools.
  • Measurement window matters: a retry can merely accelerate recovery that would have happened; compare cumulative recovery curves over 30-60 days, not point-in-time rates.
  • Guardrails: involuntary churn, disputes/chargebacks, refund rate, support contacts, card-network decline penalties, and customer complaints about post-cancellation charges. Any of these moving is a launch blocker regardless of recovered revenue.
  • Heterogeneity: report treatment effects by stratum; expect the feature to help small merchants (no dunning sophistication) far more than large ones, which shapes both rollout priority and pricing.
  • Merchant reporting: report incremental recovery with uncertainty, not gross recovery — overstated claims destroy trust when merchants audit.

The underlying concept

Attribution without a counterfactual systematically overstates impact — the same failure mode as "email campaigns drive purchases from people who would have bought anyway." The general antidote is to define the estimand first (incremental effect vs. baseline process), pick the randomization unit where the mechanism operates (here, the merchant), and measure over a horizon long enough to distinguish acceleration of an outcome from creation of one. In payments, the additional twist is that every intervention carries direct costs and trust costs, so guardrails aren't an afterthought — they're half the evaluation.

Source

Distilled Prep canon — curated from Stripe's public work on payments reliability and revenue recovery.

difficulty 3/5 Data scienceexperimentationrecommendationml-monitoring

Viewing hours rise after a homepage redesign, but title completion and next-month retention fall. What would you conclude, and what would you investigate before recommending launch or rollback?

Practice against the follow-up probes
  • Which of these three metrics should carry the decision, and why?
  • What mechanisms could push hours up while pushing completion down?
  • How would you tell "more low-intent viewing" apart from "worse recommendations"?
  • What segments would you cut first, and what would each cut tell you?
  • Retention is slow and noisy. How do you make a launch decision on a reasonable timeline anyway?
Show answer guide

What the interviewer is probing

Metric hierarchy judgment: does the candidate recognize that engagement volume is a proxy that has come apart from the outcomes it proxies for (satisfaction, retention), and that when proxies and true outcomes disagree, the true outcome wins? Also probing mechanistic thinking — a strong candidate generates concrete hypotheses (autoplay inflation, accidental starts, discovery of shallow content) and maps each to an observable signature, rather than proposing generic "dig into the data."

Strong answer outline

  • State the metric hierarchy immediately: retention is closest to the business outcome; completion is a satisfaction proxy; raw hours is the weakest of the three and the easiest to inflate mechanically. A design that trades retention for hours is a regression, full stop.
  • Generate mechanisms with signatures: (a) autoplay/preview inflation — hours up via many short sessions, starts-per-session up, completion down; (b) accidental or low-intent starts — high abandon rate in first 5 minutes; (c) discovery shifted toward bingeable-but-shallow content — title mix shift, diversity down; (d) the redesign buried search or My List — navigation funnel changes for returning users with intent.
  • Check measurement artifacts before behavior: did the redesign change what counts as a "view start" or how hours are logged? Instrumentation changes masquerade as behavior changes constantly.
  • Segment cuts, each tied to a hypothesis: device (autoplay behavior differs), profile maturity (novelty effects hit tenured users differently), household sharing, content type, market.
  • Handle the retention timeline honestly: use leading indicators validated against retention historically (completion, successful- session rate, days-active), run the experiment longer for a novelty washout, and pre-register the decision rule: launch only if retention is flat-or-up once leading indicators stabilize.
  • Recommendation shape: likely rollback or iterate — but a strong answer specifies what evidence would change that call.

The underlying concept

This is Goodhart's law meeting proxy metrics: when a measure becomes the target, it stops being a good measure. Engagement volume correlates with satisfaction until you optimize for it directly, at which point systems find ways to generate engagement that satisfaction doesn't back — autoplay, clickbait thumbnails, low-intent sessions. Mature experiment programs therefore maintain a metric hierarchy (true north outcomes, validated proxies, diagnostics) and treat proxy-outcome divergence as an alarm, not a mixed result. The practical skill is mapping each candidate mechanism to a signature you can actually observe in the data.

Source

Distilled Prep canon — curated from Netflix's public work on experimentation and personalization measurement.

difficulty 3/5 Software engdistributed-systemsapi-designreliability

A customer taps "Place order" and their app times out waiting for your response, so it retries. Meanwhile your first attempt actually succeeded — the order reached the kitchen. Walk me through how you design the order placement API so retries are safe, and then tell me what happens when the deduplication store itself is briefly unavailable.

Show answer guide

What the interviewer is probing

Whether the candidate reaches for idempotency keys as a design principle rather than a buzzword: where the key is generated, what gets stored, the atomicity requirement between "check" and "act", and TTL trade-offs. The second part tests degraded-mode reasoning — do they fail open (risk duplicates) or fail closed (risk lost orders), and can they connect that choice to the business cost of each failure?

Strong answer outline

  • Client generates the idempotency key (per logical order attempt, not per HTTP request) so all retries of one intent share a key.
  • Server: atomic check-and-set (e.g. unique constraint or conditional write), not read-then-write — the race between two concurrent retries is the classic bug.
  • Store the response with the key so retries return the original result, not just a "duplicate" error.
  • TTL discussion: too short re-enables duplicates, too long bloats storage; tie it to realistic client retry windows.
  • Degraded mode: articulate fail-open vs fail-closed and pick using domain cost — a duplicate food order costs a refund; a dropped one costs a customer. Reasonable answers differ; unreasoned answers don't.

The underlying concept

Exactly-once delivery doesn't exist between distributed parties; what systems actually implement is at-least-once delivery plus idempotent processing, which is indistinguishable from exactly-once from the caller's perspective. The idempotency key turns a side-effecting operation into something safely retryable by giving the server a durable memory of intents it has already honored. Every payment and ordering system you've used is built on this pattern.

Source

Hand-written seed example in the style of DoorDash's platform reliability posts.