Langfuse
open sourceOpen-source, self-hostable tracing, prompt management, and evaluation for model applications.
Directory
LLM Observability and Evaluation Tools Compared
Model applications fail differently from ordinary software. Nothing throws. A prompt change that improves one case quietly degrades four others, and you find out from a user weeks later. Conventional monitoring shows you a healthy 200 response containing a wrong answer.
Two capabilities address this. Tracing records what actually happened in a run — every call, tool result, token count, and cost — so a bad output can be traced back to its cause. Evaluation scores output against a dataset so you can tell whether a change helped before it ships.
This is the category teams skip while shipping fast and regret at the first regression they cannot explain. Cost attribution alone frequently pays for it: knowing which feature spends your budget is hard to reconstruct after the fact and trivial to capture up front.
6 tools · 4 open source · 3 self-hostable
At a glance
| Tool | Best for | Pricing | Runs |
|---|---|---|---|
| Langfuseopen source | Teams who need traces containing customer data to stay inside their own infrastructure. | Open source | Hosted or self-hosted |
| LangSmith | LangChain and LangGraph users, who get the deepest integration for the least work. | Free tier | Hosted |
| Braintrust | Teams treating prompt changes as changes that need tests, with evals wired into CI. | Free tier | Hosted |
| Heliconeopen source | Getting cost and latency visibility across a codebase quickly, especially one you do not want to instrument. | Free tier | Hosted or self-hosted |
| Phoenixopen source | Teams standardised on OpenTelemetry, and anyone debugging retrieval quality interactively. | Open source | Hosted or self-hosted |
| W&B Weaveopen source | Organisations already on Weights & Biases who want one place for all model work. | Free tier | Hosted |
Tracing, datasets, and evaluation from the LangChain team; works with or without LangChain.
Eval-first platform for scoring model output against datasets and catching regressions before release.
Helicone
open sourceOne-line proxy that logs every request with cost and latency attribution per user or feature.
Phoenix
open sourceOpen-source OpenTelemetry-based tracing and evaluation from Arize, runnable in a notebook.
W&B Weave
open sourceTracing and evaluation from Weights & Biases, aimed at teams already tracking experiments there.
How to choose
Tracing or evaluation first
Some tools lead with tracing and add evals; others are eval-first with tracing attached. If you are debugging today, start with tracing. If you are about to change a prompt that is already in production, start with evals.
How it gets your data
A proxy needs a one-line base-URL change and captures everything. An SDK needs instrumentation and captures more structure. OpenTelemetry-based tools slot into tracing you may already run.
Self-hosting
Traces contain your prompts and your users' inputs. If that data cannot leave your infrastructure, the shortlist is the tools you can run yourself — several here are open source and designed for it.
Whether evals run in CI
An eval you have to remember to run is an eval that stops being run. The tools worth adopting expose a command that fails a build on regression.
Frequently asked
What is LLM observability?
Recording what a model application actually did — prompts, responses, tool calls, latency, token counts and cost — so behaviour can be inspected after the fact. Unlike ordinary monitoring, a failure here is usually a plausible-looking wrong answer rather than an error.
What is an LLM eval?
A test that scores model output against a dataset of inputs and expected properties, using assertions, another model as judge, or human review. It is how you tell whether a prompt or model change was an improvement.
Can I self-host LLM observability?
Yes. Several tools here are open source and built to be self-hosted, which matters because traces contain your prompts and your users' data.
How do I track LLM cost per feature?
Tag requests with the feature, user, or tenant they belong to, and use a tool that aggregates cost by tag. Doing this at the gateway or proxy layer captures every call without touching application code.
Tooling choices rarely dominate a bill; the model does. Compare model prices per million tokens or price your own workload before optimising anything here.