Rewind by
December 1st

A system design story in five tracks. You're the team shipping a personalized year in music to every listener, all on the same morning.

An example design of Spotify Wrapped, inspired by the Spotify Engineering blog.

→ or Space next · N speaker notes

The brief from product, in one sentence

"Give every listener a story about their year."

500,000,000 listeners

×30 plays a day

×365 days

0

play events behind one feature

It ships on one morning, and it has to be right, because people will screenshot it.

How would you build it?

Tracklist

Five problems, in the order they bite

Interrupt any time Each fix causes the next problem

Track 01 Just run a query?

It works for one listener.

SELECT artist_id, COUNT(*) AS plays
FROM   plays
WHERE  user_id = 'u_4821'
  AND  played_at >= '2026-01-01'
GROUP BY artist_id
ORDER BY plays DESC LIMIT 5;

-- the extra step: rank fans inside each artist
PERCENT_RANK() OVER (
  PARTITION BY artist_id
  ORDER BY plays DESC)

Run it 500M times and you scan the same 5.5T rows over and over, on the database that serves the app.
So it becomes a batch job. A scheduler such as Airflow starts it each night, and many workers share the stored play events, split by listener.
Then product wants "You're in the top 0.5% of this artist's fans."
Ranking fans compares listeners with each other, so the data has to be split again, by artist.

What happens to the worker that gets the biggest artist?

Track 01 The 9-hour straggler

One artist, one worker, everyone waits.

Guess the job time before I press run. Then tell me how you'd fix it.

Job time –Worker 5 got the biggest artist, with 2,000× the listeners of a typical one.

Salting: add a suffix, artist_id#0..7, so the hot key spreads over 8 workers. Each works out a partial answer, then one small step merges the 8.

4:00. Take guesses, click "Run job", and let the room watch seven workers go idle. Collect fixes before you salt. Someone will say "add workers"; it doesn't help, because one key can't be split across machines.

Why it happens: a grouped job sends every row with the same key to the same worker. One giant key means one overloaded worker, and the job is only as fast as its slowest worker.
Salting: glue a random number 0 to 7 onto the hot key. It is now 8 different keys, so the rows spread over 8 workers. Each worker computes a partial result. A second, tiny step strips the suffix and merges the 8 partial results into one.
When it works: only when partial results can be merged: counts, sums, min and max, histograms. For the fan rank, each worker builds a small histogram (how many listeners had 1 play, 2 plays, and so on). Merge 8 histograms and you can read off where the top 0.5% starts.
In practice: salt only the hot keys, found by sampling the data first; salting everything adds a step for no gain. Some engines can detect and split skewed keys on their own.
Cost: an extra stage and more complicated code, so do it only where the skew hurts.

Track 02 Game round · hands up, then flip

Does this count as a play?

None of these is a compute problem. They're definitions, and the definition is the hardest code to change later.

Track 02 Rules as a pipeline

Every event passes three gates.

1 · dedupe event_id

2 · duration ≥ min

3 · before cutoff

Minimum play length30 s

0Counted plays

0Duplicates dropped

0Too short

0Late, held for correction run

Which two gates could you safely swap? Which swap breaks the count?

2:30. Where these rules live: defined once in a shared, versioned place, and applied early in the pipeline by a cleaning job, so every later job counts plays the same way. Not in the app, which only reports facts.

Drag the slider from 30 to 0 and back: the "Counted" number is a product decision. Then ask about gate order.

Gates 1 and 2 can swap. The duration check looks only at the event itself, so both copies of a duplicate get the same verdict. The final count is the same either way. Filtering first even leaves fewer IDs to remember; the only price is that the "too short" tally counts duplicates.
Dedupe must come before gate 3. The cutoff depends on when a copy arrived, and a retry can arrive weeks after the original. If dedupe came last, the original would be counted now, and the late copy would be held for the correction run and counted again there.
The rule: dedupe before any step that counts, or that can treat two copies of one event differently. Wasted compute on duplicates is a small cost; double counting is the real problem.

Track 02 The event is a contract

Old app versions send v1 events for years.

play_event v1 (2024)

{
  "event_id": "e_7f3a…",
  "user_id": "u_4821",
  "track_id": "t_9912",
  "ts": 1764531900,
  "ms_played": 192000
}

play_event v2 (2026)

{
  "schema": 2,
  "event_id": "e_c01d…",
  "user_id": "u_4821",
  "track_id": "t_9912",
  "ts": 1793112000,
  "ms_played": 192000,
  "content_type": "podcast",
  "offline_synced_at": 1794900000
}

Add fields as optional, with a default the reader knows (missing content_type means music).
Never rename a field or change what it means. Add a new one.
Readers ignore fields they don't know, so an old job keeps working when a new field appears.

Track 03 Code you can trust

November 20th. Podcast minutes are double.

Support ticket: "Rewind preview says I listened to 1,840 podcast minutes. I listened to maybe 900."

The query, unchanged since March

SELECT p.user_id,
       SUM(p.ms_played) / 60000 AS podcast_min
FROM   plays p
JOIN   episodes e ON e.episode_id = p.episode_id
GROUP BY p.user_id;

Release note, Nov 18, podcasts team: "Episodes are now available in several languages."

Nobody touched the query. Why is it wrong now?

episodes.episode_id	language	added
ep_301	en	March
ep_301	es	Nov 18

One play of ep_301 now matches two rows, so its minutes are added twice.

3:00. Reveal the ticket, then the query. Let people look, then show the release note as the hint and ask the question on the slide.

The hidden assumption: the query joins plays to the episodes table to know which plays are podcasts. It quietly assumes episodes has exactly one row per episode_id.
What changed: the podcasts team added translations by inserting one episodes row per language. Now ep_301 has two rows.
Why that doubles: a join returns one output row for every matching pair. One play of ep_301 matches both the English and the Spanish row, so the play appears twice, and SUM adds its minutes twice. 920 real minutes become 1,840.
Why it is nasty: no error, no failed job, and only listeners of translated podcasts are affected. Our code did not change; someone else's table did. This is often called a join fan-out.

Bridge: the job already ran for everyone. How do we fix it, and how do we prove the fix?

Track 03 Before a listener does

Three cheap checks. Which would have caught it?

A · Test the query

plays:    ep_301, 30 min
episodes: ep_301
expect podcast_min == 30

A unit test with hand-written rows.

B · Check the table first

SELECT episode_id FROM episodes
GROUP BY episode_id
HAVING COUNT(*) > 1

Any row back: stop the job.

C · Check the result after

podcast_min + music_min
   == total_min
-- total comes from raw plays

The parts must add up to the whole.

Our code never changed. The data did. So test what you assume about the data, not only the code.

Which table do you join to without ever checking it?

Track 03 Re-run, don't repair

Fix the code, re-run the year, compare.

1 · Fix

Join to a table with one row per episode.

And keep check B, so the job stops if this ever happens again.

2 · Re-run

Same raw play log, same cutoff date. A new output next to the old one.

rewind_v1 stays. rewind_v2 appears. Nothing is overwritten.

3 · Compare

u_4821	v1	v2
podcast min	1,840	920
music min	12,400	12,400
top artist	same	same

listeners see: rewind_v1

What in that comparison would make you refuse to switch?

Track 04 A pile of cards

Every card is a small plugin with one rule.

type CardGenerator interface {
    // does this listener get the card?
    IsEligible(s YearSummary) bool
    // what does it say?
    Build(s YearSummary) Card
}

Top artistneeds plays ≥ 50

Podcast of the yearneeds podcast minutes ≥ 300

Welcomeneeds plays < 50

A new card is a new type with these two methods, not an edit to one giant function.
Each card decides for itself which listeners get it.

Track 04 Who gets which card?

Guess the cards, then pick a listener.

Track 04 Compute when?

At 9:00 AM, or the night before?

On demand · work the servers do

Everyone taps at once, and each tap builds a Rewind from scratch.

Precompute · work the servers do

The cards are already built. Each tap is one lookup.

key	stored value
u_4821	top_artist, minutes, fan_rank
u_0007	welcome

Store each listener's finished cards under their user ID. The price: you also store cards for people who never open them.

When would on demand be the right call?

Track 05 Delete my data

A listener asks us to forget them.

Remember track 03: we never edit the raw play log. That is why re-running the year was safe.

Before I press it: where else does this listener's data live?

4:00. Let the room name places the data lives, then press delete. Three stores clear; the raw log cannot.

Why the log is never edited: every result can be rebuilt from it, so a bug is fixed by re-running (track 03). You can always explain where a number came from. And files that never change are simple and cheap to copy, cache and back up.
The cost: "never edit" also means "never delete one person". The rule that saved us now blocks us.
The way out on the slide: lock each listener's events with that listener's own key when writing them. To forget someone, throw away their key. The bytes stay in the log, but nobody can ever read them. This is called crypto-shredding.
Other options: rewrite old log files now and then, leaving deleted listeners out (slow and expensive at this size), or keep raw events only for a limited time.
The fourth box: you can only delete copies you know about. Keeping track of which data was built from which is called lineage.

Recap

One play, all the way to a card.

The real thing Spotify's 2025 Wrapped Archive, simplified

What Spotify built last year

One Wrapped feature: short written reports about each listener's most remarkable days of the year.

~350Meligible listeners

~1.4Breports, all written before launch day

4 daysof writing, at thousands of requests per second

Everything is built ahead of time: "Wrapped doesn't ramp, it spikes!" Track 04
One database row per listener, found by user ID. Track 04
A timezone bug gave wrong results. They fixed the pipeline, deleted the bad reports and ran them again. Track 03

Simplified from Inside the Archive: The Tech Behind Your 2025 Wrapped Highlights, Spotify Engineering, March 2026.

2:30. This is one feature of the real Wrapped, as Spotify described it in March 2026. Walk the diagram left to right:

Data pipeline: a distributed batch job picks up to five "remarkable days" per listener (biggest listening day, biggest discovery day, and so on).
Object storage, then a queue: each listener's days are saved as files, then one message per listener goes on a queue so the next stage can work through them at its own pace.
Report writer: a language model, fine-tuned for this job, writes each report.
Key-value database: one row per listener keyed by user ID, one column per day.
App: reads the finished reports. Nothing is generated when you tap.
On the side: larger "judge" models graded a random sample of about 165,000 reports for accuracy, safety and tone.

Then reveal the three lines: the same lessons as our tracks, from a real launch. Be clear that this is a summary of one blog post, not inside knowledge.

Lessons learned

What to take back to your own systems

Find the giant key. When work is split by a key, the one key far bigger than the rest decides how long everything takes.

Definitions are code. Write business rules down in one place, give them a version, and test them.

Test the data, not only the code. And keep raw inputs, so a bug becomes a re-run and a comparison.

Small pieces, built early. Independent parts turn edge cases into test cases. Work done the night before turns a launch into lookups.

Know where your data goes. Every copy is one you must be able to find again, and every rule that helps today has a price later.

Thank you for
your attention

References

Spotify Unwrapped: How we brought you a decade of dataSpotify Engineering, 2020. One stored result per listener, one re-runnable job per story.
How Spotify Optimized the Largest Dataflow Job Ever for Wrapped 2020Spotify Engineering, 2021. Huge joins, and what skewed keys do to them.
Spotify Unwrapped 2019: How We Built an In-App Experience Just for YouSpotify Engineering, 2020. The app side of the story.
Inside the Archive: The Tech Behind Your 2025 Wrapped HighlightsSpotify Engineering, 2026. The most recent post on Wrapped.
Designing Data-Intensive ApplicationsMartin Kleppmann. The chapters on batch processing and on changing data formats over time.

Rewind byDecember 1st

"Give every listener a story about their year."

Five problems, in the order they bite

It works for one listener.

One artist, one worker, everyone waits.

Does this count as a play?

Every event passes three gates.

Old app versions send v1 events for years.

November 20th. Podcast minutes are double.

Three cheap checks. Which would have caught it?

Fix the code, re-run the year, compare.

Every card is a small plugin with one rule.

Guess the cards, then pick a listener.

At 9:00 AM, or the night before?

A listener asks us to forget them.

One play, all the way to a card.

What Spotify built last year

What to take back to your own systems

Thank you foryour attention

Rewind by
December 1st

Thank you for
your attention