15 Questions about Langfuse Answered

1. Is Langfuse Open Source?

The answer is yes, Langfuse is open source. Its entire code-base is available on the Langfuse GitHub Repository. The vast majority of Langfuse’s code is permissively licensed under the MIT license.

Langfuse follows an open-core licensing model. The core of Langfuse, its tracing, integrations, API, data model and exports are MIT-licensed. Some features on Langfuse’s periphery such as the model-based evaluation service or the Prompt Playground are commercially licensed and can only be used in Langfuse Cloud or need to be purchased to be used when Self-Hosting. Find more information on this here.

2. Can I use Langfuse for Prompt Management?

Yes, Langfuse comes with built-in, powerful Prompt Management capabilities. Langfuse allows you to effectively manage and version your prompts.

The main benefits of using Prompt Management with your LLM apps are:

• Decoupling: deploy new prompts without redeploying your application.
• Non-technical users can create and update prompts via Langfuse Console.
• Quickly rollback to a previous version of a prompt.

Find out more about how to start using Langfuse Prompt Managment here.

For Langchain Users: You can use Langfuse to add Prompt Management to Langchain. Prompt Management works both for Langchain Python and Langchain JS

3. How can I run LLM Evaluations?

You can use Langfuse for evaluations of your LLM applications. Langfuse offers several powerful evaluation capabilities:

• Model-based evaluations: Automate evaluations of all incoming traces. Use pre-built templates for common criteria like hallucinations, toxicity, relevance, correctness
  • Create custom evaluation templates tailored to your specific needs
  • Apply evaluations to filtered subsets of your traces
  • Run evaluations asynchronously within Langfuse’s infrastructure
• Evaluation Libraries: You can use Langfuse’s Python SDK to run evaluations using popular eval libraries such as:
  • OpenAI Evals
  • Langchain Evaluators
  • RAGAS (for RAG applications)
  • UpTrain evals

In addition to automated and programmatic evaluations, Langfuse supports:

• Collecting user feedback
• Manually labelling observations within the Langfuse UI (Human annotation, data labelling, manual evaluation)
• You can always export your scores to use them for analysis or downstream use cases

4. Is Langfuse an Alternative to Langsmith?

The answer is yes, Langfuse is an open source alternative to Langsmith. Langsmith is a closed source project maintained by the team behind Langchain. Langfuse is an independent open source product.

The Consultancy Astral Insights has written up a report comparing Langsmith and Langfuse.

5. Can I use the Vercel AI SDK with Langfuse?

The answer is yes, you can use the Vercel AI SDK with Langfuse. It is a popular library amongst builders and we are happy to provide first class support for the Vercel AI SDK. Please find integration documentation and an example in our Vercel AI SDK integrations page.

6. Which models and frameworks can I use Langfuse with?

You can use any model or framework with Langfuse. We maintain a large number of integrations into Langfuse that you can use either with our SDKs or integrations with other popular libraries such as Llama Index, Dify.AI, Langchain and others.

As a fallback, you can always use our API to ingest data into Langfuse while using your preferred language. The API specification includes a downloadable OpenAPI spec to allow you to create types (download button at the top of the API reference.

7. Can Langfuse provide a DPA to its customers?

Yes, Langfuse can provide customers with a DPA. We are happy to enter into Data Protection Agreements with customers on our paid plans. Please navigate to our Security Center to review our DPA template. Please download it, sign it and send a signed version to [email protected]. We will then counter-sign the DPA and send it back to you.

8. Is Langfuse SOC2 Type2 certified?

The answer is yes, Langfuse is SOC2 Type2 certified. The audit report is available upon request from the Langfuse Team. Please navigate to our Security Center to learn more.

9. Is Langfuse ISO27001 audited?

The answer is yes, Langfuse has been audited for ISO27001. The audit report is available upon request from the Langfuse Team. Please head to our Security Center to learn more.

10. Is Langfuse GDPR compliant?

Yes, Langfuse is GDPR compliant. Langfuse is a German company, based in Berlin, Germany and we take privacy a data security very seriously. You can use Langfuse in compliance with GDPR. You will need to a) use our EU Cloud Data Region and enter into a DPA with us or b) self-host Langfuse.

You can find more information in our Security Center.

11. How to add observability/tracing to Ollama?

You can add observability or tracing to Ollama by using Langfuse with one of its many integrations. The easiest way to get started adding observability or tracing to Ollama is most likely to use the Llama Index or Langchain integrations with Langfuse.

12. Alternatives to Promptlayer? Is Langfuse an Alternative to Promptlayer?

The answer is yes, Langfuse is an open source alternative to Promptlayer. You can use Langfuse to manage your prompts, monitor your application, track analytics and run evaluations. Langfuse is an open source project while Prompt Layer is closed source.

13. How can I use the LLM Playground?

You can use the LLM Playground on Langfuse Cloud. You will need to add LLM API keys to your project’s settings before you are able to start using the Prompt Playground.

14. How can I use RAGAS evaluations?

The answer is, you can use RAGAS Evaluations with Langfuse. We have written a tutorial that shows how to run RAGAS evaluations on your historical LLM data. Additionally, you can use the RAGAS evaluation prompts and use them with Langfuse’s automated eval service to run RAGAS evaluations on any new LLM trace.

15. Can I use Langchain, Langserve and Langgraph with Langfuse Observability and Tracing?

Yes, you can use Langchain, Langserve and Langgraph with Langfuse to gain observability, tracing and evals. Langfuse integrates with both Langchain Python and Langchain JS. Langfuse has maintained a first class integration with Langchain since day one.