Observe the memory layer with EverOS and Langfuse

Memory is the one layer of the agent stack that observability usually can't see. This integration makes an agent's memory operations visible in Langfuse: what got stored, what a question recalled, how confident that recall was, when two memories were reconciled into one, and what all of it cost.

EverOS emits OpenTelemetry natively as of 1.2.0. There is no wrapper and no instrumentation code to write: enable it in your config and the traces appear.

What is EverOS?

EverOS is an open-source (Apache-2.0), local-first memory runtime for AI agents. Conversations and agent trajectories are extracted by an LLM into user profiles, episodic memories, agent cases and reusable agent skills; stored as human-readable Markdown; indexed in SQLite + LanceDB; retrieved via hybrid BM25 + vector recall with reranking; and consolidated over time by an offline reflection engine. It runs as an HTTP service (pip install everos).

What is Langfuse?

Langfuse is the open-source platform for LLM observability, evaluation, and prompt management. It ingests OpenTelemetry traces natively, so any system that speaks OTLP, including EverOS, shows up next to the rest of your agent's traces, with model-usage and cost views, scores, and dashboards.

See it in Langfuse

A write. The memory EverOS distilled from the conversation is the output of the everos.extract generation. Boundary detection, extraction and each reflection strategy carry their own tokens and cost, so the price of remembering one thing is itemised rather than lumped together.

A read: the question, what came back, and the recall-quality scores on the retriever observation.

The scores themselves, each tagged with the retrieval method that produced it.

Every figure above is a real trace, not a mock-up. To get an interactive one in your own project, replay the recording the EverOS repo ships (below) — no EverOS install required.

What you get

• The memory lifecycle as trace trees. flush → boundary → extract → persist, with the reflection strategies fanning out behind it, and search → recall → rank with the query embedding nested inside recall. Every stage timed and costed.
• The cost of remembering. LLM and embedding calls are generation and embedding observations carrying model and token usage, so Langfuse computes cost in its model-usage views. EverOS sends no cost of its own.
• Recall quality over time. Each search attaches two numbers: how relevant the best hit looked, and whether that cleared a threshold. These are useful to plot and trend, but they are proxies, not ground-truth relevance. For real relevance evaluation, run an LLM-as-judge over the query and recalled content in the trace.
• Conflicting memories getting resolved. When two memories say different things about the same subject — a plan that changed, a preference that was updated — reflection merges them and retires the outdated one, which then stops coming back from search. This is the part of a memory system that usually happens invisibly, and it is the hardest to debug when it goes wrong. Here it is an observation of its own, inside the write that triggered it.
• Filtering by conversation and by user. Write traces are tagged with the conversation they belong to, so Langfuse groups a whole conversation into one session. Search traces record whose memory was searched, so you can filter searches by user.

How it maps

Each EverOS API call becomes one Langfuse trace, with the server-side pipeline stages as typed child observations:

Reflection runs after the request that triggered it has already returned, but its spans still land inside that request's trace instead of a separate one. So a write and everything it set off in the background stay in one place, and you can tell which write caused which piece of work.

Recall quality is pushed to the Langfuse scores API, attached to the search observation. Calibrated and uncalibrated retrieval methods report under different score names, because their values are on different scales and averaging them together would be meaningless. The span and score reference in the EverOS repo lists the exact names and what each one means.

Quick Start

pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
export LANGFUSE_PUBLIC_KEY="pk-lf-..."
export LANGFUSE_SECRET_KEY="sk-lf-..."
export LANGFUSE_HOST="https://cloud.langfuse.com"  # 🇺🇸 US: https://us.cloud.langfuse.com
python replay.py

pip install "everos[otel]"

[observability]
enabled             = true
langfuse_public_key = "pk-lf-..."
langfuse_secret_key = "sk-lf-..."
langfuse_host       = "https://cloud.langfuse.com"  # 🇺🇸 US: https://us.cloud.langfuse.com
# capture_content   = true   # opt-in: also record query and extracted memory text

everos server start
python demo.py     # from examples/langfuse/

Resources

• EverOS on GitHub
• Integration example and span/score reference
• Raven, the self-improving agent harness built on EverOS
• Langfuse OpenTelemetry docs

Interoperability with the Python SDK

You can use this integration together with the Langfuse SDKs to add additional attributes to the observation.

from langfuse import observe, propagate_attributes, get_client

langfuse = get_client()

@observe()
def my_llm_pipeline(input):
    # Add additional attributes (user_id, session_id, metadata, version, tags) to all spans created within this execution scope
    with propagate_attributes(
        user_id="user_123",
        session_id="session_abc",
        tags=["agent", "my-observation"],
        metadata={"email": "user@langfuse.com"},
        version="1.0.0"
    ):

        # YOUR APPLICATION CODE HERE
        result = call_llm(input)

        return result

# Run the function
my_llm_pipeline("Hi")

from langfuse import get_client, propagate_attributes

langfuse = get_client()

with langfuse.start_as_current_observation(
    as_type="span",
    name="my-observation",
    trace_context={"trace_id": "abcdef1234567890abcdef1234567890"},  # Must be 32 hex chars
) as observation:

    # Add additional attributes (user_id, session_id, metadata, version, tags)
    # to all observations created within this execution scope
    with propagate_attributes(
        user_id="user_123",
        session_id="session_abc",
        metadata={"experiment": "variant_a", "env": "prod"},
        version="1.0",
    ):
        # YOUR APPLICATION CODE HERE
        result = call_llm("some input")

# Flush events in short-lived applications
langfuse.flush()

Troubleshooting

export LANGFUSE_DEBUG="True"

Next Steps

Once you have instrumented your code, you can manage, evaluate and debug your application: