--- title: Music streaming DJ sidebarTitle: Music DJ description: How a net-new, low-risk DJ feature in a music streaming app goes live almost immediately and improves from listening behavior. --- # Music streaming DJ This is an example to illustrate the concepts of the [Langfuse Academy](/academy). ## Context A music streaming app adds a DJ feature in beta. The DJ keeps a queue going based on what the user has played before and what the algorithm thinks the user will like. Every few songs it says a short word about what's coming up and why it chose it. Listeners can also optionally interact with the DJ to steer it, by clicking a microphone button and talking to it. Two characteristics drive how to approach the AI engineering setup: 1. The **risk of a bad output is low**. The worst case is a skipped song or an awkward DJ comment, and the feature is labeled beta. 2. This is a new feature, so **there is no historical data to start from**, and the team will need to learn from live listener behavior. Because of this, it makes sense to go live as soon as possible and [iterate based on live feedback from the beginning](/academy/ai-engineering-loop#you-dont-have-to-close-the-full-loop-on-day-one). ## Tracing the DJ There are two different kinds of [traces](/academy/tracing#anatomy-of-a-trace) that will together [form a session](/academy/tracing#traces-vs-sessions): | Trace name | Details | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `plan-next-set` | Plans the next set of tracks and writes the commentary, triggered by the DJ itself every few songs or by a DJ request.
**input:** listening context and any instructions
**output:** next tracks queued, plus the commentary line | | `handle-dj-request` | Handles a voice request from the listener, triggered by the microphone button.
**input:** voice clip and the current queue
**output:** a short reply, plus instructions for the `plan-next-set` run it triggers | A listening session and both trace types up close: **Session listen_7f3e · user u_8841** - [Trace] `plan-next-set` (2.8s) - "Kicking off with two favorites from your week." - [Trace] `plan-next-set` (2.4s) - "Staying in this lane with some mellow electronica." - [Trace] `handle-dj-request` (1.6s) - "Play something calmer." - [Trace] `plan-next-set` (triggered by dj-request, 1.5s) - "Calming it down, here is some ambient piano." **The observations inside a plan-next-set trace** - [Trace] `plan-next-set` (session: listen_7f3e, user: u_8841, 1.5s) - Input: taste: electronica, downtempo · recent: Tycho "Awake", Bonobo "Kerala" · instruction: "calmer" - Output: 4 tracks queued · commentary: "Calming it down, here is some ambient piano." - [Tool] `select-tracks` (0.6s) - Input: taste: electronica, downtempo · recent: Tycho "Awake", Bonobo "Kerala" · instruction: "calmer" - Output: Nils Frahm "Says" · Ólafur Arnalds "Saman" · Max Richter "On the Nature of Daylight" · Joep Beving "Sleeping Lotus" - [Gen] `write-commentary` (gpt-4.1-mini, 120 tok, $0.0002, 0.9s) - Input: the 4 selected tracks · instruction: "calmer" - Output: "Calming it down, here is some ambient piano." **The observations inside a handle-dj-request trace** - [Trace] `handle-dj-request` (session: listen_7f3e, user: u_8841, 1.6s) - Input: voice clip (2s) - Output: reply: "Got it, calming things down." · instruction: "calmer" - [Tool] `transcribe-request` (0.4s) - Input: voice clip (2s) - Output: "Play something calmer." - [Gen] `interpret-request` (gpt-4.1-mini, 210 tok, $0.0004, 1.2s) - Input: "Play something calmer." · current queue - Output: reply: "Got it, calming things down." · instruction: "calmer" - [Event] `trigger-plan-next-set` (@ 1.6s) - Input: instruction: "calmer" ## Capturing user behavior In order to learn from users, we will log some [interesting behavior](/academy/monitoring#user-feedback) on the applicable traces. **Evaluators** - `track_skipped` — Player event, binary plus comment, on the plan-next-set trace - The listener skips a DJ-picked track shortly after it starts - No evaluator needed, the player writes the score. It always lands on the `plan-next-set` trace whose set was playing at that moment, and the comment holds which track was skipped, so a set with several misses collects several scores. - `dj_replaced` — Player event, binary, on the session - The listener turns the DJ off but keeps listening - Simply ending a session says little: the listener may just have arrived somewhere. Continuing to listen without the DJ is what makes it a signal, the listener chose music over the DJ. - `message_type` — LLM-as-a-judge, categorical, on the handle-dj-request trace - Classifies each voice request as `steering`, `correction`, `repeated_instruction`, `positive_reaction`, or `negative_reaction` - `steering` is normal use of the feature and stays out of the failure counts. `correction` means the DJ's last action missed. `repeated_instruction` means the DJ failed to follow an instruction it already got: a compliance failure rather than a taste miss, and it [asks for a different fix](/academy/evaluate/choosing-what-to-evaluate#tie-every-metric-to-a-decision). It is also the only signal here with LLM costs to run, and voice requests are a small slice of the traffic, so the [evaluator budget](/academy/evaluate/choosing-what-to-evaluate#mind-the-budget) stays small. In principle, the team could already iterate with only this in place. Tracing and [monitoring](/academy/monitoring#metrics-and-signals) form a small loop of their own. In the beginning, this is probably enough for the team to quickly improve the DJ feature. **The AI engineering loop** - **Trace** — every set and every request - **Monitor** — skips, dj_replaced, message_type - **Build datasets** — not yet - **Experiment** — not yet - **Evaluate** — not yet As the setup matures, and the team wants to have more structured testing in place, they can start building datasets and experiments on top of these signals. ## Structured testing With only the live signals, testing a change means shipping it and watching the scores. The team can add two more deliberate ways of testing: [experiments on datasets](/academy/experiments#how-experiments-are-used), and A/B tests on live users. ### Experiments on datasets In this use case, testing end to end is very hard offline: whether a session was good only shows in live listening behavior, and taste differs per user, so no expected output holds for everyone. [A single step](/academy/datasets#what-makes-a-good-dataset) like `select-tracks` can be tested, with the expectation describing the direction of the set rather than exact tracks: **Dataset: selection-directions** Listening contexts paired with good and bad directions for the next set. Tests only the select-tracks step. | Input | Expected output | | --- | --- | | taste: indie folk · recent: Big Thief 'Vampire Empire', Fleet Foxes 'Mykonos' · instruction: none | Good directions: adjacent indie folk, soft rock. Bad directions: high-energy EDM, hip hop. | | taste: electronica, house · recent: an upbeat house set · instruction: 'something calmer' | Good directions: downtempo, ambient electronica. Bad directions: more upbeat house, ignoring the instruction. | **Evaluators** - `direction_match` — LLM-as-a-judge, binary, per dataset item - Does the selected set go in one of the good directions and avoid the bad ones? - A [reference-based](/academy/evaluate#reference-based-vs-reference-free) evaluator: the expected output gives it the directions to grade against. It runs on the `select-tracks` step only, not the whole DJ, so a run is cheap and a failure points directly at selection. ### A/B tests on live users Some changes are hard to grade offline, like the tone of voice of the commentary. Since the risk is low, the team can give a small group of listeners the new version and compare the signal scores between the groups, like the skip rate and the `message_type` distribution. If the new version does better, they can roll it out to everyone. With this in place, the full loop is running: **The AI engineering loop** - **Trace** — every set and every request - **Monitor** — skips, dj_replaced, message_type, A/B comparisons - **Build datasets** — from what we see while monitoring production - **Experiment** — selection algorithm, DJ prompt, ... - **Evaluate** — grade step outputs against their expectations ## Conclusion This is an example of a feature that is low risk and has no historical data to learn from. The best thing you can do in such cases is get traces in as soon as possible and start monitoring them. Everything else, datasets, experiments, A/B tests, can be built on top of that over time. Check out the [other examples](/academy/examples) or the [academy](/academy) to learn more. --- ## Agent Instructions This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/academy/examples/music-streaming-dj.md`. ### Querying these docs If the answer is not on this page, query the documentation instead of guessing: - **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question: ```bash curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?" ``` - **Index of every page**: , with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt). ### Before writing Langfuse code - **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results. - **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application. - **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples. - **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx langfuse-cli api `) to read or write traces, prompts, datasets, and scores from the terminal. Found an error in these docs? Please open an issue at .