Use Milvus with Langfuse

Thanks to the team at Milvus for developing this guide. These docs are adapted from their write up, which you can read here.

What is Milvus?

Milvus is an open-source vector database that powers AI applications with vector embeddings and similarity search. It offers tools for efficient storage and retrieval of high-dimensional vectors, making it ideal for AI and machine learning applications.

Trace your queries with the Langfuse LlamaIndex integration

In this quickstart, we'll show you how to set up a LlamaIndex application using Milvus Lite as the vector store. We'll also show you how to use the Langfuse LlamaIndex integration to trace your application.

Quick Start Guide

Step 1: Create a Langfuse Account

Visit Langfuse and create an account.
Create a new project and copy your Langfuse API keys.

Step 2: Install Required Packages

Make sure you have both llama-index and langfuse installed.

$ pip install llama-index langfuse openinference-instrumentation-llama-index llama-index-vector-stores-milvus --upgrade

Step 3: Initialize Langfuse

Visit Langfuse to create an account. Create a new project and copy your Langfuse API keys. This example uses OpenAI for embeddings and chat completions, so you also need to specify your OpenAI key in the environment variable.

import os

# Get keys for your project from the project settings page
os.environ.setdefault("LANGFUSE_SECRET_KEY", "sk-...")
os.environ.setdefault("LANGFUSE_PUBLIC_KEY", "pk-...")
os.environ.setdefault("LANGFUSE_BASE_URL", "https://cloud.langfuse.com") # 🇪🇺 EU region
# Other Langfuse data regions include 🇺🇸 US: https://us.cloud.langfuse.com, 🇯🇵 Japan: https://jp.cloud.langfuse.com and ⚕️ HIPAA: https://hipaa.cloud.langfuse.com

# Your OpenAI key
os.environ.setdefault("OPENAI_API_KEY", "sk-...")

Step 4: Set Up LlamaIndex Instrumentation

Langfuse traces LlamaIndex via the OpenInference instrumentation, which exports OpenTelemetry spans to Langfuse.

from langfuse import get_client
from openinference.instrumentation.llama_index import LlamaIndexInstrumentor

langfuse = get_client()

# Instrument LlamaIndex — all subsequent index and query operations are traced
LlamaIndexInstrumentor().instrument()

Step 5: Index Using Milvus Lite

from llama_index.core import Document
from llama_index.core import VectorStoreIndex
from llama_index.core import StorageContext
from llama_index.vector_stores.milvus import MilvusVectorStore

# Create documents
doc1 = Document(text="Your document text here.")
doc2 = Document(text="Another document text here.")

# Set up Milvus vector store
vector_store = MilvusVectorStore(
    uri="tmp/milvus_demo.db", dim=1536, overwrite=False
)
storage_context = StorageContext.from_defaults(vector_store=vector_store)

# Create index
index = VectorStoreIndex.from_documents(
    [doc1, doc2], storage_context=storage_context
)

Step 6: Query and Chat

# Query
response = index.as_query_engine().query("Your query here")
print(response)

# Chat
response = index.as_chat_engine().chat("Your chat message here")
print(response)

Step 7: Explore Traces in Langfuse

You can now see traces of your index and query in your Langfuse project.

Example traces in Langfuse (public links):

Was this page helpful?

On this page