The Audit Trail: Every Agent Action, Every Connector Call, Every Decision
How we structure the trace that answers 'what exactly did the agent do and why?' — from input ingestion through model calls to connector outcomes and tamper-evident storage.
What 'audit trail' actually needs to mean
Most logging systems tell you that something happened. An audit trail for a governed AI agent needs to tell you what the agent received, what it decided, what external calls it made, what those calls returned, and what the final output was — with enough fidelity that a compliance officer, security investigator, or regulator can reconstruct the agent's reasoning from the record alone.
That's a different requirement from application logging. It means structured records, not log lines. It means capturing inputs and outputs at the model layer, not just at the API boundary. It means recording connector call arguments and return codes, not just 'connector was called'. And it means writing those records in a way that can't be modified after the fact.
The trace structure
Every agent invocation in Orchestrik produces a single trace document. The trace has a top-level identity block (trace ID, tenant ID, agent ID), an invocation block (trigger source, start and end timestamps, terminal status), and a steps array containing every action the agent took in sequence.
Each step has a type — connector_call or model_call — and the full context for that type. Connector calls record the connector name, the operation, the arguments passed (sanitised — no credential values), and the result code. Model calls record the model identifier, input and output token counts, and a structured summary of the output. Approval steps record the approver identity, the decision, and the timestamp.
The trace closes with an outcome block (resolved, escalated, timed out, failed) and a chain hash. The hash is computed over the serialised trace body plus the hash of the previous trace written by the same agent — a simple hash chain that makes any retroactive modification detectable.
{
"trace_id": "trc_01HV3KQXZ7B4MNJPW9CSRD6F",
"tenant_id": "ten_acme_prod",
"agent_id": "agt_customer_support_v2",
"invocation": {
"trigger": "webhook",
"trigger_ref": "zendesk:ticket:12847",
"started_at": "2025-02-14T09:22:04.381Z",
"ended_at": "2025-02-14T09:22:11.047Z",
"status": "resolved"
},
"input": {
"summary": "Password reset not working for SSO users",
"channel": "email",
"customer": { "id": "cust_8831", "tier": "enterprise" }
},
"steps": [
{
"seq": 1,
"type": "connector_call",
"connector": "zendesk",
"operation": "get_ticket",
"args": { "ticket_id": 12847 },
"result": "ok",
"duration_ms": 312,
"credential": "cred_zendesk_prod [resolved, not logged]"
},
{
"seq": 2,
"type": "model_call",
"model": "orchestrik/classify-v3",
"input_tokens": 841,
"output_tokens": 24,
"duration_ms": 1203,
"classification": { "category": "auth", "sub": "sso_password" }
},
{
"seq": 3,
"type": "connector_call",
"connector": "knowledge_base",
"operation": "search",
"args": { "query": "SSO password reset flow", "top_k": 3 },
"result": "ok",
"duration_ms": 218
},
{
"seq": 4,
"type": "model_call",
"model": "orchestrik/respond-v2",
"input_tokens": 2104,
"output_tokens": 187,
"duration_ms": 2841,
"output_summary": "Draft response generated"
},
{
"seq": 5,
"type": "connector_call",
"connector": "zendesk",
"operation": "reply_ticket",
"args": { "ticket_id": 12847, "public": true },
"result": "ok",
"duration_ms": 291
}
],
"outcome": {
"type": "resolved",
"detail": "ticket_replied",
"escalated": false
},
"hash": "sha256:3f8a2b...",
"prev_hash": "sha256:9c14d7..."
}Credentials in the audit record
Connector calls include the credential reference used — the identifier of the credential in the vault — but never the credential value itself. The audit record shows that cred_zendesk_prod was used to make the call. It does not log the API key. This is enforced at the connector execution layer, not at the logging layer: the connector resolves the credential at call time, makes the call, and records the result. The value is never passed to the agent runtime or the audit writer.
This means the audit trail is safe to export to external SIEM systems, safe to share with auditors, and safe to query through the control plane dashboard without any redaction step. The sensitive values never enter the record.
Write path and tamper evidence
The audit writer is a separate process from the agent runtime. Traces are written to the audit store over an internal API that only supports append operations — there's no update or delete path. The agent runtime calls the audit writer to open a trace, appends steps as they execute, and closes the trace on completion or failure.
The audit store backend is abstracted — it can be a Postgres table with triggers that prevent UPDATE and DELETE, an object storage bucket with object lock enabled, or an append-only log file on a write-protected volume. The write path is the same regardless of backend. The append-only constraint is enforced at the API layer, not just at the storage layer.
Traces are hashed on close. The hash covers the full trace body (every step, all arguments, all results, the outcome) plus the hash of the immediately prior trace for the same agent. An adversary who wants to modify a trace must re-hash every subsequent trace in the chain and update the audit store for each — which is detectable if you retain an out-of-band checkpoint of recent hashes, which the control plane does by default.
Retention, query, and export
Trace retention is configured per tenant. SaaS plans retain audit traces for the duration of the active subscription. Enterprise on-premise retention is fully configurable — some regulated deployments retain indefinitely, some purge after 7 years per their own compliance schedule.
The control plane dashboard provides a structured query interface: filter by agent, by date range, by outcome type, by connector, by customer or ticket ID. Traces can be exported as JSON (the native format), as CSV summaries, or as NDJSON for ingestion into Splunk, Elastic, or Datadog.
For on-premise deployments, the audit store is in the customer's environment and the customer controls export. The control plane provides the query and export interface; the data never leaves the customer's network unless they choose to export it.
What the trace doesn't capture
The full text of model prompts and responses is not stored in the trace by default. Token counts and structured output summaries are logged. Full prompt logging is available as an opt-in setting per agent — it's off by default because (a) it's expensive at volume, (b) prompts often contain customer PII that shouldn't be in the audit store, and (c) the structured classification and output summary are sufficient for most compliance use cases.
If you're doing prompt auditing for research or safety review, you can enable full prompt logging on a specific agent in the control plane dashboard. It logs to a separate store with separate access controls from the operational audit trail.
In summary
- Every invocation produces a single structured trace with a steps array covering all connector and model calls
- Credentials are referenced by ID only — values never appear in the trace
- Traces are written to an append-only store by a process separate from the agent runtime
- A hash chain over sequential traces makes retroactive modification detectable
- Retention, query, and export are configurable per tenant and available to self-host
Related reading
The audit trail design was shaped significantly by the decision to build on-premise first. For the broader architectural context, see the companion post.
Why We Built On-Premise First →