This is a Jupyter notebook

Observability for Fireworks AI with Langfuse

This guide shows you how to integrate Fireworks AI with Langfuse. Fireworks AI’s API endpoints are fully compatible with the OpenAI SDK, allowing you to trace and monitor your AI applications seamlessly.

What is Fireworks AI? Fireworks AI is a platform that provides API access to state-of-the-art open-source and proprietary AI models with OpenAI-compatible endpoints.

What is Langfuse? Langfuse is an open source LLM engineering platform that helps teams trace API calls, monitor performance, and debug issues in their AI applications.

Step 1: Install Dependencies

%pip install openai langfuse

Step 2: Set Up Environment Variables

import os
 
# Get keys for your project from the project settings page
# https://cloud.langfuse.com
 
os.environ["LANGFUSE_PUBLIC_KEY"] = "pk-lf-..." 
os.environ["LANGFUSE_SECRET_KEY"] = "sk-lf-..."
os.environ["LANGFUSE_HOST"] = "https://cloud.langfuse.com" # 🇪🇺 EU region
# os.environ["LANGFUSE_HOST"] = "https://us.cloud.langfuse.com" # 🇺🇸 US region
 
# Set your Fireworks API details
os.environ["FIREWORKS_AI_API_BASE"] = "https://api.fireworks.ai/inference/v1"
os.environ["FIREWORKS_AI_API_KEY"] = "fw_..."

Step 3: Use Langfuse OpenAI Drop-in Replacement

from langfuse.openai import openai
 
client = openai.OpenAI(
  api_key=os.environ.get("FIREWORKS_AI_API_KEY"),
  base_url=os.environ.get("FIREWORKS_AI_API_BASE")
)

Step 4: Run an Example

response = client.chat.completions.create(
  model="accounts/fireworks/models/llama-v3p1-8b-instruct",
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Why is open source important?"},
  ],
  name = "Fireworks-AI-Trace" # name of the trace
)
print(response.choices[0].message.content)

Open source is a crucial aspect of the software development and technology ecosystem, and its importance cannot be overstated. Here are some reasons why open source is important:

1. **Collaboration and Community**: Open source allows developers from all over the world to collaborate on a project, share ideas, and contribute code. This leads to a large community of developers working together to improve the software, making it more robust and reliable.
2. **Transparency and Accountability**: Open source code is freely available for anyone to review, modify, and distribute. This transparency ensures that developers can see exactly how the software works, making it easier to identify and fix bugs or security vulnerabilities.
3. **Innovation and Competition**: Open source software (OSS) can foster innovation and competition, as developers can create new features, plugins, or extensions that build upon existing OSS. This leads to a more diverse and vibrant ecosystem.
4. **Cost Savings**: Open source software is often free or low-cost, which can be a significant advantage for individuals, organizations, or governments with limited budgets.
5. **Security**: Open source code can be reviewed by a large community of developers, making it easier to identify and fix security vulnerabilities before they become major issues.
6. **Customization and Flexibility**: Open source software can be customized and modified to meet specific needs, making it an attractive option for organizations with unique requirements.
7. **Patent and Licensing Issues**: Open source software is often released under permissive licenses, which reduces the risk of patent and licensing disputes.
8. **Scalability and Interoperability**: Open source software can be easily integrated with other OSS and proprietary software, making it easier to build complex systems and applications.
9. **Education and Research**: Open source software provides a platform for students, researchers, and developers to learn and experiment with new technologies, contributing to the advancement of computer science and technology.
10. **Democratization of Technology**: Open source software can democratize access to technology, enabling individuals and organizations to participate in the development and use of software, regardless of their financial or geographical constraints.

Overall, open source is essential for promoting collaboration, innovation, and transparency in software development, and it has become an integral part of the technology landscape.

Step 5: Enhance Tracing (Optional)

You can enhance your Fireworks AI traces:

Add metadata, tags, log levels and user IDs to traces
Group traces by sessions
@observe() decorator to trace additional application logic
Use Langfuse Prompt Management and link prompts to traces
Add score to traces

Visit the OpenAI SDK cookbook to see more examples on passing additional parameters. Find out more about Langfuse Evaluations and Prompt Management in the Langfuse documentation.