The Question Most Teams Cannot Answer in Under a Minute

Ask a typical enterprise team running AI agents in production a simple question: what are your agents doing right now?

The answer almost always starts with “well, it depends what you mean by” — and that hedging is the entire problem. The questions any operator should be able to answer in under a minute:

• How many agent runs are in flight right now?
• Which agents have called Anthropic, OpenAI, or Bedrock in the last hour?
• What is the median latency per model per team in the last 24 hours?
• Which prompts triggered our PII scanner today, and which agents sent them?
• Is the cost trend for this team’s agents above or below the 7-day rolling average?
• Has any agent exceeded its rate limit in the last week, and if so, which evaluator caught it?
• Did anything anomalous happen overnight that needs human review?

A team that cannot answer those quickly is not just inconvenienced. They are operating in the dark. And operating in the dark is fine when your agents do safe, narrow, deterministic things. It becomes dangerous the moment your agents touch real users, real data, real money — which, in May 2026, almost all of them do.

This is the observability gap that live agent observability closes.

“Live” Is the Word That Matters

Observability is not new. Every enterprise has logs, metrics, and traces somewhere. Most of those somewheres are post-hoc: data lands in a warehouse, gets indexed, becomes queryable an hour or a day later. That cadence was acceptable when systems made decisions on the timescale of human approvals. AI agents do not.

An agent in a production loop makes hundreds of decisions per minute. Each decision involves a model call, a tool invocation, a state update. By the time a post-hoc dashboard tells you something went wrong, the agent has made another thousand decisions on top of it.

Live observability means three things:

1. Sub-second visibility. What the agent did one second ago is visible right now, not after a batch job runs.

2. Inline enrichment. Events arrive with the policy decisions, the user identity, the model used, the cost, the latency, and the structured reason — not as raw text.

3. Queryable in human time. “Every agent run that touched customer data in the last hour” returns in milliseconds, not minutes.

If your observability is not live in all three senses, you can see history but not the present. You can do forensics but not operations. You can prove what happened, but not steer what is happening.

What Live Observability Actually Lets You Do

The benefits compound across roles. Five worth naming explicitly:

For platform engineers: catch breakage early

An agent’s failure mode is rarely a crash. It is degradation — slower responses, more retries, fallback paths firing, higher token cost per request. Live observability turns those signals into alerts before users notice.

Examples that come up in customer environments:

• A model provider regresses. Latency jumps from 800ms to 4s. Without live observability, your agents tank for a week before someone notices. With it, you see the spike inside an hour and reroute.

• A prompt template changes. Token cost per request doubles. Without live observability, your monthly bill is a surprise. With it, you see the regression in the deploy that caused it.

• A tool dependency throttles you. Retries cascade. Without live observability, you find out from a stakeholder. With it, your error-rate dashboard pages on-call.

For security teams: catch misuse early

An agent’s worst-case behavior is not a crash either. It is use of capabilities it was given but should not have used in this context. A summarization agent calling out to an unexpected endpoint. A data-exploration agent emitting un-redacted PII. A coding agent disabling a validation check to satisfy a failing test.

Live observability lets security teams:

• See policy decisions in real time. Every block, every allow, every augment.
• Drill into anomalies the moment they fire — not next week.
• Run “what if I tighten this policy” simulations against the live event stream.
• Trace a suspected misuse from one event to the agent run that produced it to the user who triggered it.

For compliance teams: replace forensics with queries

The EU AI Act’s Article 12 — automatically generated logs for high-risk systems — does not specify “post-hoc batch logs.” It says automatically generated. The teams that will pass an Article 12 audit on day one are the teams whose audit log is their observability stream, in real time, with cryptographic chaining for tamper evidence.

For compliance specifically, live observability turns six questions that take six weeks into six queries that take six seconds:

• “Every agent that called a model in the EU region with customer data in the prompt in the last 90 days.”
• “Every policy override that was applied, and who approved it.”
• “Every anomaly the system flagged in Q1, broken down by team.”
• “Every model deprecation event and what we migrated to.”
• “Every framework control mapping and what evidence supports it.”
• “Every retention boundary event for legal holds.”

For finance and ops: see cost in motion

AI inference is volatile. Anthropic prices changed three times in 2025. Bedrock cross-region pricing shifted in early 2026. A team running a fleet of agents without live cost observability is running a fleet of agents on optimism.

Live observability surfaces:

• Cost per agent, per team, per project, in motion.
• Cost per prompt template — so you see the regression on the deploy that introduced it.
• Cost trend deltas against rolling averages, with alerts when something breaks the curve.
• Provider-side anomalies (a provider’s billing spikes for reasons unrelated to your usage) so you can challenge them.

For product and engineering: learn from real behavior

Agents are not deterministic. The interesting thing about running an agent in production is not whether it crashes — it is the distribution of what users ask, what the agent does, where it succeeds, and where it fails to produce a useful answer.

Live observability gives product teams:

• A representative sample of real prompts, automatically PII-redacted.
• The model’s structured outputs alongside the policy decisions.
• Latency and cost per intent category.
• Drift detection: “the distribution of questions this agent handles has shifted week-over-week — is the underlying user need changing?”

This is product analytics for AI products. It is rarer than it should be.

What Live Observability Needs to Provide

Putting the requirements together, a live agent observability layer has eight properties. Treat this as a checklist:

1. Inline policy enforcement is the source of truth. Every event is generated as a side effect of a policy decision. There is no separate “logging” path that can diverge from the enforcement path.

2. Structured events, not strings. Every event has a typed schema: who, what, when, which model, which tool, which policy, which decision, which cost, which latency, which scope.

3. Tamper-evident audit chain. Events are cryptographically linked. Retroactive tampering is detectable. This is what makes the audit log defensible in front of an auditor or a regulator.

4. Sub-second propagation to UI. The CISO opens a dashboard. Events from one second ago are visible. The dashboard is a real view of present reality.

5. Multi-provider normalization. Anthropic, OpenAI, Bedrock, Vertex, Azure, Mistral, Cohere, Ollama, plus a dozen more — all normalized to the same event shape. The dashboard does not care which provider; the engineer asking a question does not care; only the analytics that explicitly slice by provider do.

6. Real-time anomaly detection. Statistical anomaly detection — 2σ-based on rolling baselines — on every meaningful dimension: latency, cost, error rate, content scanner hit rate, policy denial rate. Anomalies surface as alerts, not as data points buried in a dashboard.

7. Policy simulator that reads the live stream. “If I enforced this new policy starting today, how many events in the last 30 days would have changed?” The simulator runs against the live event store; it is not a separate copy.

8. Identity scoping all the way down. Every event is tagged with the org / team / project / app / user scope that produced it. Queries that scope by team return only that team’s events; admin queries return everything.

If any one of these eight is missing, you have observability that is good for forensics but not for operations.

Where Zentinelle Fits

Zentinelle , with the zentinelle-sdk , is the governance and observability pillar of Calliope’s private-AI stack — and it is built on the principle that the policy decision is the event. There is no separate logging path. Every event the dashboard shows is an event that was produced as a side effect of a real, enforced policy decision.

The features mapped to the checklist above:

• 24 policy evaluators, each emitting structured events on decisions.
• Tamper-evident audit chain with cryptographic verification.
• Real-time event stream in the portal — live polling, filters, drill-down.
• Usage analytics — tokens, cost, latency by agent, model, provider, team, project.
• Anomaly detection — 2σ-based statistical detection on rolling baselines.
• Alerts management — acknowledge, resolve, dismiss workflow.
• Policy simulator — dry-run new policies against the live 30-day history.
• Per-tenant identity model — Org → Team → Deployment → Endpoint → User, with policies, audit, and analytics all scoped to the same hierarchy.

For agents that sit behind a gateway, Zentinelle’s Django proxy or Go gateway mediates and observes everything. For agents that need to participate from inside their code, the zentinelle-sdk (Python, TypeScript, plus framework plugins for LangChain, CrewAI, Vercel AI SDK, LlamaIndex, MS Agent Framework, and n8n) provides the same evaluate / emit / register / log primitives — so an agent your team owns can participate in the observability stream without proxying its traffic.

Both modes produce the same event shape. Both feed the same dashboards. Both contribute to the same tamper-evident chain.

The Operational Difference

Two teams ran a similar AI agent in production through the first quarter of 2026. Both ran on private infrastructure. Both used Claude and GPT models. Both had reasonable security postures on paper.

Team A had post-hoc observability. Logs in a warehouse. Reports run weekly. Audit log generated quarterly.

Team B had live observability. Every event indexed in real time, with anomaly detection on five dimensions and an alerts queue staffed during business hours.

In Q1:

• Team A discovered, in the Q1 review, that a tool call had been silently failing for six weeks. They had not noticed because the agent’s downstream behavior was indistinguishable from “the user did not need this tool today.” Cost to fix: a week of investigation, a week of remediation, a quarterly post-mortem.

• Team B discovered the same failure mode within four hours of it starting. Their error-rate-per-tool dashboard fired an alert. The on-call engineer rolled back the tool’s most recent deploy. Cost to fix: a half-day.

Multiply that delta across every quarter, every team, every agent. The compounded cost of post-hoc observability is what makes “live” not a nice-to-have but the operational baseline for serious AI in 2026.

Where to Start

If you are setting up live agent observability for the first time, the order that has worked:

1. Pick the platform first, not the metrics. Choose a layer that makes policy decisions the source of truth for events. Otherwise you will spend a year arguing about which “log” is correct.

2. Get one team’s agents fully instrumented before you scale. A team with five agents and live observability beats a department with fifty agents and post-hoc logs.

3. Tune anomaly detection by hand for the first month. Baselines need real production traffic. Expect false positives. Expect to tighten thresholds.

4. Wire the dashboards into your existing on-call rotation. Observability you do not look at is not observability.

5. Then add the policy simulator and the audit chain. Once the live stream is real, simulations and audit become straightforward queries on top of it.

In May 2026, with three months until the EU AI Act’s high-risk obligations land, with foundation model providers changing pricing and latency monthly, with agents moving from prototype to production faster than most platform teams can keep up — knowing what your agents are doing, live, in your own cloud, is the single most leveraged investment you can make in your AI stack.

Not because it makes the agents better. Because it makes you able to keep running them.

This is the third post in our private-AI series. Next: how preview environments stitch the runtime and governance pillars together, so policy violations surface in CI — not in a 2 a.m. Slack page from compliance.