AWS Bedrock Integration

AWS Bedrock (AWS (opens in a new tab)) is a fully managed AWS service that lets you use foundation models and custom models to generate text, images, and audio.

When using Langfuse with AWS Bedrock, you can easily capture detailed traces and metrics for every request, giving you insights into the performance and behavior of your application.

Integration Options

There are a few ways through which you can capture traces and metrics for AWS Bedrock:

1. via an application framework that is integrated with Langfuse:

   • Langchain
   • Llama Index
   • Haystack

2. via a Proxy such as LiteLLM

3. via wrapping the Bedrock SDK with the Langfuse Decorator (We are currently working on adding more documentation and a cookbook for this.)

4. via the low-level Python SDK to have full control over what is captured

Can I monitor AWS Bedrock cost and token usage in Langfuse?

Yes, you can monitor cost and token usage of your Bedrock calls in Langfuse. The native integrations with LLM application frameworks and the LiteLLM proxy will automatically report token usage to Langfuse.

If you use the Langfuse decorator or the low-level Python SDK, you can report token usage and (optionally) also cost information directly.

You can define custom price information via the Langfuse dashboard or UI (see docs) to adjust to the exact pricing of your models on AWS Bedrock.

Resources

• Metadocs, Monitoring your Langchain app's cost using Bedrock with Langfuse (opens in a new tab), featuring Langchain integration and custom model price definitions for Bedrock models.