---
title: Music streaming DJ
sidebarTitle: Music DJ
description: How a net-new, low-risk DJ feature in a music streaming app goes live almost immediately and improves from listening behavior.
---

# Music streaming DJ

  This is an example to illustrate the concepts of the [Langfuse
  Academy](/academy).

## Context

A music streaming app adds a DJ feature in beta. The DJ keeps a queue going based on what the user has played before and what the algorithm thinks the user will like. Every few songs it says a short word about what's coming up and why it chose it. Listeners can also optionally interact with the DJ to steer it, by clicking a microphone button and talking to it.

Two characteristics drive how to approach the AI engineering setup:

1. The **risk of a bad output is low**. The worst case is a skipped song or an awkward DJ comment, and the feature is labeled beta.
2. This is a new feature, so **there is no historical data to start from**, and the team will need to learn from live listener behavior.

Because of this, it makes sense to go live as soon as possible and [iterate based on live feedback from the beginning](/academy/ai-engineering-loop#you-dont-have-to-close-the-full-loop-on-day-one).

## Tracing the DJ

There are two different kinds of [traces](/academy/tracing#anatomy-of-a-trace) that will together [form a session](/academy/tracing#traces-vs-sessions):

| Trace name          | Details                                                                                                                                                                                                                                     |
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `plan-next-set`     | Plans the next set of tracks and writes the commentary, triggered by the DJ itself every few songs or by a DJ request.<br />**input:** listening context and any instructions<br />**output:** next tracks queued, plus the commentary line |
| `handle-dj-request` | Handles a voice request from the listener, triggered by the microphone button.<br />**input:** voice clip and the current queue<br />**output:** a short reply, plus instructions for the `plan-next-set` run it triggers                   |

A listening session and both trace types up close:

**Example session**

**Session listen_7f3e, user u_8841**

- [Trace] `plan-next-set` (2.8s)
  - "Kicking off with two favorites from your week."
- [Trace] `plan-next-set` (2.4s)
  - "Staying in this lane with some mellow electronica."
- [Trace] `handle-dj-request` (1.6s)
  - "Play something calmer."
- [Trace] `plan-next-set` (triggered by dj-request, 1.5s)
  - "Calming it down, here is some ambient piano."

**Example plan-next-set trace**

**The observations inside a plan-next-set trace**

- [Trace] `plan-next-set` (session: listen_7f3e, user: u_8841, 1.5s)
  - Input: taste: electronica, downtempo; recent: Tycho "Awake", Bonobo "Kerala"; instruction: "calmer"
  - Output: 4 tracks queued; commentary: "Calming it down, here is some ambient piano."
  - [Tool] `select-tracks` (0.6s)
    - Input: taste: electronica, downtempo; recent: Tycho "Awake", Bonobo "Kerala"; instruction: "calmer"
    - Output: Nils Frahm "Says", Ólafur Arnalds "Saman", Max Richter "On the Nature of Daylight", Joep Beving "Sleeping Lotus"
  - [Gen] `write-commentary` (gpt-4.1-mini, 120 tok, $0.0002, 0.9s)
    - Input: the 4 selected tracks; instruction: "calmer"
    - Output: "Calming it down, here is some ambient piano."

**Example handle-dj-request trace**

**The observations inside a handle-dj-request trace**

- [Trace] `handle-dj-request` (session: listen_7f3e, user: u_8841, 1.6s)
  - Input: voice clip (2s)
  - Output: reply: "Got it, calming things down."; instruction: "calmer"
  - [Tool] `transcribe-request` (0.4s)
    - Input: voice clip (2s)
    - Output: "Play something calmer."
  - [Gen] `interpret-request` (gpt-4.1-mini, 210 tok, $0.0004, 1.2s)
    - Input: "Play something calmer."; current queue
    - Output: reply: "Got it, calming things down."; instruction: "calmer"
  - [Event] `trigger-plan-next-set` (@ 1.6s)
    - Input: instruction: "calmer"

## Capturing user behavior

In order to learn from users, we will capture [user signals](/academy/monitoring/capturing-signals) on the applicable traces: behavioral signals from the player, conversation signals from the voice requests, and one explicit rating the app already has.

**Evaluators**

- `track_skipped` — Player event, binary plus comment, on the plan-next-set trace
  - The listener skips a DJ-picked track shortly after it starts
  - No evaluator needed, the player writes the score. It always lands on the `plan-next-set` trace whose set was playing at that moment, and the comment holds which track was skipped, so a set with several misses collects several scores. Beware: a skip is a weak negative. It says as much about taste and mood as about the DJ.
- `track_liked` — Player event, binary, on the plan-next-set trace
  - The listener likes or saves a DJ-picked track
  - The like button already exists in the app, so capturing it costs nothing. It is the only [explicit rating](/academy/monitoring/capturing-signals#explicit-ratings) here: rare and unambiguous, read as a trend per set rather than as a target on its own.
- `dj_replaced` — Player event, binary, on the session
  - The listener turns the DJ off but keeps listening
  - Simply ending a session says little: the listener may just have arrived somewhere. Continuing to listen without the DJ is what makes it a signal, the listener chose music over the DJ.
- `message_type` — LLM-as-a-judge, categorical, on the handle-dj-request trace
  - Classifies each voice request as `steering`, `correction`, `repeated_instruction`, `positive_reaction`, or `negative_reaction`
  - `steering` is normal use of the feature and stays out of the failure counts. `correction` means the DJ's last action missed. `repeated_instruction` means the DJ failed to follow an instruction it already got: a compliance failure rather than a taste miss, and it [asks for a different fix](/academy/evaluate/choosing-what-to-evaluate#tie-every-metric-to-a-decision). It is also the only signal here with LLM costs to run, and voice requests are a small slice of the traffic, so the [evaluator budget](/academy/evaluate/choosing-what-to-evaluate#mind-the-budget) stays small. `correction` and `repeated_instruction` are the negatives worth [opening one by one](/academy/monitoring/capturing-signals#how-to-interpret-signals).

These signals are acted upon with inherent biases in mind: the DJ only gets signal on tracks it plays, so a genre it never tries stays [unlabeled](/academy/monitoring/capturing-signals#biases). And beta listeners and time of day influence skip rates independent of DJ quality: a commute plays differently from a focus session.

In principle, the team could already iterate with only this in place. Tracing and [monitoring](/academy/monitoring#metrics-and-signals) form a small loop of their own. In the beginning, this is probably enough for the team to quickly improve the DJ feature.

**The AI engineering loop**

- **Trace** — every set and every request
- **Monitor** — skips, likes, dj_replaced, message_type
- **Build datasets** — not yet
- **Experiment** — not yet
- **Evaluate** — not yet

As the setup matures, and the team wants to have more structured testing in place, they can start building datasets and experiments on top of these signals.

## Structured testing

With only the live signals, testing a change means shipping it and watching the scores. The team can add two more deliberate ways of testing: [experiments on datasets](/academy/experiments#how-experiments-are-used), and A/B tests on live users.

### Experiments on datasets

In this use case, testing end to end is very hard offline: whether a session was good only shows in live listening behavior, and taste differs per user, so no expected output holds for everyone. [A single step](/academy/datasets#what-makes-a-good-dataset) like `select-tracks` can be tested, with the expectation describing the direction of the set rather than exact tracks:

**Dataset: selection-directions**

Listening contexts paired with good and bad directions for the next set. Tests only the select-tracks step.

| Input | Expected output |
| --- | --- |
| taste: indie folk; recent: Big Thief 'Vampire Empire', Fleet Foxes 'Mykonos'; instruction: none | Good directions: adjacent indie folk, soft rock. Bad directions: high-energy EDM, hip hop. |
| taste: electronica, house; recent: an upbeat house set; instruction: 'something calmer' | Good directions: downtempo, ambient electronica. Bad directions: more upbeat house, ignoring the instruction. |

**Evaluators**

- `direction_match` — LLM-as-a-judge, binary, per dataset item
  - Does the selected set go in one of the good directions and avoid the bad ones?
  - A [reference-based](/academy/evaluate#reference-based-vs-reference-free) evaluator: the expected output gives it the directions to grade against. It runs on the `select-tracks` step only, not the whole DJ, so a run is cheap and a failure points directly at selection.

### A/B tests on live users

Some changes are hard to grade offline, like the tone of voice of the commentary. Since the risk is low, the team can give a small group of listeners the new version and compare the signal scores between the groups, like the skip rate and the `message_type` distribution. Skip rate alone is a risky target: a DJ optimized never to be skipped learns to play safe favorites on repeat. Pair it with a signal closer to [what the product is for](/academy/monitoring/capturing-signals#which-signal), like time listened per session or how often the DJ stays on. If the new version does better, they can roll it out to everyone.

With this in place, the full loop is running:

**The AI engineering loop**

- **Trace** — every set and every request
- **Monitor** — skips, likes, dj_replaced, message_type, A/B comparisons
- **Build datasets** — from what we see while monitoring production
- **Experiment** — selection algorithm, DJ prompt, ...
- **Evaluate** — grade step outputs against their expectations

## Conclusion

This is an example of a feature that is low risk and has no historical data to learn from. The best thing you can do in such cases is get traces in as soon as possible and start monitoring them. Everything else, datasets, experiments, A/B tests, can be built on top of that over time.

Check out the [other examples](/academy/examples) or the [academy](/academy) to learn more.

<!-- agent-instructions -->

---

## Agent Instructions

This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/academy/examples/music-streaming-dj.md`.

### Querying these docs

If the answer is not on this page, query the documentation instead of guessing:

- **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question:

  ```bash
  curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?"
  ```

- **Index of every page**: <https://langfuse.com/llms.txt>, with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt).

### Before writing Langfuse code

- **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results.
- **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application.
- **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples.
- **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx @langfuse/cli api <resource> <action>`) to read or write traces, prompts, datasets, and scores from the terminal.

Found an error in these docs? Please open an issue at <https://github.com/langfuse/langfuse-docs/issues>.
