[ OBSERVE / LLM_OBSERVABILITY ]

What is LLM observability?

LLM observability is the practice of capturing what a model was asked, what it returned, what it cost, and how well it performed, so a failure can be diagnosed after the fact. Centurian extends that record into signed evidence a third party can verify without trusting your dashboard.

A call, a sequence, and a record

Three layers get discussed as if they were one. LLM observability watches a call: prompt in, completion out, tokens counted, latency measured. Agent observability watches a sequence, because an autonomous agent chose a tool, passed it an argument, read the result, and decided what to do next, and the reasoning lives in the ordering rather than in any single completion.

The third layer is the record itself, and it is the one most stacks skip. A trace answers a developer’s question this week. An audit trail answers a regulator’s question in two years, asked by someone who was not there and has no reason to take your word for it. Those are different artifacts with different retention, different mutability, and different verification requirements.

Why sampling is the structural problem

Observability tooling samples on purpose. Storing every trace forever is expensive and diagnosis rarely needs the full population, so a sensible default drops most of it. Record-keeping obligations do not work that way: they attach to a system because of what it does, not because someone happens to be debugging it. A stack sampling 1% of traces has instrumented 1% of its evidence, and the other 99% is not recoverable later. This is the gap that turns a well-instrumented engineering org into an unprepared one the first time an auditor asks for a specific agent’s specific action on a specific date.

Bitemporal, signed, cross-vendor

Centurian writes every agent action with both valid time (when it happened) and transaction time (when the system learned of it), which is what lets you answer “what did we know, and when did we know it” rather than only “what is true now.” The export is signed, so a third party can verify nothing changed after the fact. And registration runs through one MCP front door regardless of whether the agent was built on Salesforce Agentforce, AWS Bedrock, Microsoft Foundry, Google Vertex, Copilot Studio, or custom code, which is why the roll-up is one list instead of one list per platform. Every cost record carries a source label (OBSERVED, IMPORTED, SELF_REPORTED, or ESTIMATED) so no number is presented without saying how it was obtained. See per-agent cost tracking.

Where existing tools fit

If you are shipping an LLM application and need to know why p95 latency moved or which prompt version regressed, a dedicated LLM observability tool is the right instrument and Centurian is not competing for that job. The question Centurian answers sits one seat over: the person accountable for AI activity across the business, who did not deploy most of it, and who needs to produce proof rather than a chart. See the comparison with AgentOps and Langfuse for where the boundary falls.

FAQ

What is LLM observability?

+
LLM observability is the practice of capturing what a model was asked, what it returned, what it cost, and how well it performed, so a failure can be diagnosed after the fact rather than reproduced from scratch. In practice it means traces of prompts and completions, token and latency metrics, and some quality signal attached to the output.

How is LLM observability different from agent observability?

+
LLM observability watches a call. Agent observability watches a sequence: the agent chose a tool, passed it an argument, got a result, and decided what to do next. A trace of the model call alone cannot tell you why the agent took the third step, because the reasoning that produced it lives in the sequence rather than in any single completion. Multi-step autonomy is what makes the distinction matter.

What are the best LLM observability tools?

+
The established options are built for the team shipping the model: they instrument your own application, in your own stack, and answer engineering questions about latency, cost, and output quality. Centurian is not a replacement for that and does not claim to be. It answers a different question, from a different seat: what did every agent across every vendor do, and can you prove it to someone who does not trust your dashboard.

Is a trace the same as an audit trail?

+
No, and the difference decides whether the record survives a challenge. A trace is written for a developer debugging a failure, and it is usually mutable, retained for weeks, and scoped to one application. An audit trail is written for someone who was not there and may not trust you: it needs both valid time and transaction time, retention measured against a regulation, and a signature that lets a third party verify nothing changed after the fact.

Does observability satisfy the EU AI Act?

+
Not on its own. Observability tooling is built to be sampled, truncated, and expired, because its job is diagnosis and storing everything forever is wasteful. Record-keeping obligations run the other direction: they attach to systems in scope regardless of whether anyone is currently debugging them. A stack that samples 1% of traces has instrumented 1% of the evidence, and the missing 99% cannot be reconstructed later.

Which LLM observability platform scales across an organization?

+
The scaling constraint is usually organizational rather than technical. Per-application observability scales fine per application and produces one silo per team, which is why an org running agents on Salesforce Agentforce, AWS Bedrock, and custom code ends up with three partial pictures and no combined one. Centurian's registration layer is cross-vendor by design, so the roll-up exists at the org level rather than being assembled by hand each quarter.
Get early access →

First agent free, forever · No credit card