Experimentation (releases & versions)
You can track the effect of changes to your LLM app on metrics in Langfuse. This allows you to:
- Run experiments (A/B tests) in production and measure the impact on costs, latencies and quality.
- Example: "What is the impact of switching to a new model?"
- Explain changes to metrics over time.
- Example: "Why did latency in this chain increase?"
release tracks the overall version of your application. Commonly it is set to the semantic version or git commit hash of your application.
from langfuse import Langfuse langfuse = Langfuse( ENV_PUBLIC_KEY, ENV_SECRET_KEY, ENV_HOST, release="<release_tag>" )
The SDKs will look for a
LANGFUSE_RELEASE environment variable. Use it to configure the release e.g. in your CI/CD pipeline.
LANGFUSE_RELEASE = "<release_tag>" # <- github sha or other identifier
If no other
release is set, the Langfuse SDKs default to a set of known release environment variables.
version parameter can be added to
traces and all observation types (
event). Thereby, you can track the effect of a new
version on the metrics of an object with a specific
name using Langfuse analytics.
from langfuse.model import InitialGeneration langfuse.generation(InitialGeneration( name="guess-countries", version="1.0", ))
langfuse.event() also take an optional
Version parameter in Langfuse interface