--- title: "Launch YC" date: 2023/07/19 description: Cross-post of our Launch YC (W23) post which explains why we're building Langfuse. tag: announcement ogImage: /images/blog/product-analytics-for-llm-apps/yc-team.jpeg author: Clemens --- _**TLDR:** Langfuse is building open source product analytics (think 'Mixpanel') for LLM apps. We help companies to track and analyze quality, cost and latency across product releases and use cases._ ![Langfuse](/images/blog/product-analytics-for-llm-apps/yc-team.jpeg) Hi everyone, we're Max, Marc and Clemens. We were part of the Winter 23 batch and work on Langfuse, where we help teams make sense of how their LLM applications perform. ## 🤯 Problem LLMs represent a new paradigm in software. Single LLM calls are probabilistic and add substantial latency and cost. Applications use LLMs in new ways via advanced prompting, embedding-based retrieval, chains, and agents with tools. Teams building production-grade LLM applications have new **product analytics** and **monitoring** needs: - **Quality of outputs** is difficult to measure. Outputs can e.g. be inaccurate, unhelpful, poorly formatted, hallucinated or error. - **Cost of compute** is a priority again given high inference costs. - **Latency of responses** matters for synchronous use cases. - **Debugging is challenging** due to increasingly complex LLM applications (chains, agents, tool usage). - **Understanding user behavior** is difficult given open-ended user prompts and conversational interactions. ## 🧠 Solution ![Metrics](/images/blog/product-analytics-for-llm-apps/metrics.png) Langfuse derives actionable insights from production data. Our customers use Langfuse to answer questions such as: _"How helpful are my LLM app's outputs? What is my LLM API spend by customer? What do latencies look like across geographies and steps of LLM chains? Did the quality of the application improve in newer versions? What was the impact of switching from zero-shotting GPT4 to using few-shotted Llama calls?"_ #### Metrics - **Quality** is measured through user feedback, model-based scoring and human-in-the-loop scored samples. Quality is assessed over time as well as across prompt versions, LLMs and users. - **Cost and Latency** are accurately measured and broken down by user, session, geography, feature, model and prompt version. #### Insights - Monitor quality/cost/latency **tradeoffs** by release to facilitate product and engineering decisions. - **Cluster use cases** by employing a classifier to understand what users are doing. - Break down **LLM usage by customer** for usage-based billing and profitability analysis. #### Integrations - [Python](/docs/sdk/python) and [Typescript](/docs/sdk/typescript/guide) SDKs to easily monitor complex LLM apps - [Frontend SDK](/docs/sdk/typescript/guide) to directly capture feedback from users as a quality signal _Langfuse can be self-hosted or used with a generous free tier in our managed cloud version._ ## 🚧 Debugging UI ![Debugging UI](/images/blog/product-analytics-for-llm-apps/debugging-ui.png) Based on the ingested data, Langfuse helps developers debug complex LLM apps in production: - Inspect LLM app executions in a nested UI for chains, agents and tool usage. - Segment by user feedback to find the root cause of quality problems. ## 🙏 Asks 1. Star us on [GitHub](https://github.com/langfuse/langfuse) + follow along on [Twitter](https://twitter.com/langfuse) & [LinkedIn](https://www.linkedin.com/company/langfuse). 2. If you run an LLM app, go ahead and reach out to us, we'd love to see how we can be helpful. 3. Please forward Langfuse to teams building commercial LLM applications. --- ## References 1. [Launch YC](https://www.ycombinator.com/launches/J2s-langfuse-open-source-product-analytics-for-llm-apps) 2. [LinkedIn post](https://www.linkedin.com/feed/update/urn:li:activity:7087446200353783808/) 3. [Twitter thread](https://twitter.com/MarcKlingen/status/1681687868534652928) --- ## Agent Instructions This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/blog/product-analytics-for-LLM-apps.md`. ### Querying these docs If the answer is not on this page, query the documentation instead of guessing: - **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question: ```bash curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?" ``` - **Index of every page**: , with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt). ### Before writing Langfuse code - **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results. - **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application. - **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples. - **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx langfuse-cli api `) to read or write traces, prompts, datasets, and scores from the terminal. Found an error in these docs? Please open an issue at .