--- title: "Native LlamaIndex Integration" date: 2024/03/06 description: "Context retrieval is a mainstay in LLM engineering. This latest integration brings Langfuse's observability to LlamaIndex applications for simple tracing, monitoring and evaluation of RAG applications." tag: integration ogImage: /images/changelog/2024-02-27-llama-index-integration.png author: Clemens --- We’re launching our latest major **integration with the LlamaIndex framework**. LlamaIndex is a darling of our community. This integration has been our users’ [most requested](https://github.com/orgs/langfuse/discussions/828) feature for a while. We're thrilled to publicly release the integration after getting great feedback from our initial beta users. Thanks again to everyone who contributed to this. ## 🦙 LlamaIndex, RAG and 🪢 Langfuse [LlamaIndex](https://www.llamaindex.ai/) is a framework for augmenting LLMs with private data. The framework is extremely popular with developers (it is approaching [30k stars on GitHub](https://github.com/run-llama/llama_index)) and is used across the space, from hobbyists to enterprises. LLMs are trained on huge datasets. Popular models such as OpenAI's GPT, Anthropic's Claude or Mistral are general purpose in nature. That is, they are not trained for a specific use case or context but to provide answers to _any_ input that is provided to them. Retrieval-augmented-generation (RAG) tries to steer Large Language Models towards a _specific_ context again. By allowing the addition of private data sources and domain specific context to LLM apps, they can [significantly enhance the quality of an LLM’s response](https://www.pinecone.io/blog/rag-study/). **RAG has proven to be a pragmatic and efficient way of working with LLMs**. It is specifically suited to builders on the application layer, trying to tackle problems in a specific domain or context. It is already a much used technique in Langfuse user’s projects and we’re thrilled to more natively support its most popular framework. LlamaIndex completes our roster of major integrations next to existing integrations with [OpenAI](/integrations/model-providers/openai-py) and LLM framework [LangChain](/integrations/frameworks/langchain) (see [full list of integrations](/integrations)). We are committed to support the most popular ways developers build on top of LLMs and specifically to open source application frameworks. Thanks again to the team at LlamaIndex for their guidance which helped make this integration feature rich and stable in record time. ## Integrating Langfuse with LlamaIndex Since this post was written, we have released a new version of the LlamaIndex integration. Please check out the [integration docs](/integrations/frameworks/llamaindex) for the latest version. Langfuse integrates with LlamaIndex via its global **callback manager**. This makes getting set up as easy as adding the following few lines to your LlamaIndex app: ```python from llama_index.core import Settings from llama_index.core.callbacks import CallbackManager from langfuse.llama_index import LlamaIndexCallbackHandler langfuse_callback_handler = LlamaIndexCallbackHandler( public_key="pk-lf-...", secret_key="sk-lf-...", base_url="https://cloud.langfuse.com" ) Settings.callback_manager = CallbackManager([langfuse_callback_handler]) ``` That’s it. Now your LlamaIndex app will automatically send detailed traces to Langfuse. While easy to set up, the integration makes extensive use of [Langfuse Tracing](/docs/tracing) to capture sessions, users, tags, versions, and additional metadata. Also, the integration is [fully interoperable](/integrations/frameworks/llamaindex#interoperability-with-langfuse-sdks) with the Langfuse Python SDK. Users can naturally continue layering other Langfuse features such as [evaluations](/docs/scores/model-based-evals), [datasets](/docs/datasets/overview) or [prompt management](/docs/prompts/get-started). --- ## Agent Instructions This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/blog/llama-index-integration.md`. ### Querying these docs If the answer is not on this page, query the documentation instead of guessing: - **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question: ```bash curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?" ``` - **Index of every page**: , with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt). ### Before writing Langfuse code - **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results. - **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application. - **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples. - **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx langfuse-cli api `) to read or write traces, prompts, datasets, and scores from the terminal. Found an error in these docs? Please open an issue at .