audit-log-llm
Structured audit logging for LLM calls. GDPR-friendly. Queryable. One weekend to add.
§ 01 · Overview
A TypeScript library that wraps every LLM call and writes a structured audit record: timestamp, model, prompt version, input, output, confidence, cost, latency, session, user. Includes a query API supporting relative windows and where-clauses over confidence, session, user, model, and system. GDPR primitives are first-class — configurable retention, right-to-erasure by session or user, both returning removal counts for DSAR audit evidence.
§ 02 · Estimated impact
§ 03 · Architecture
- 01
Wrap an existing LLM call site with `audit.wrap(fn, { system, promptVersion })`. Logging becomes automatic; the wrapped function accepts a call envelope with session and user identifiers, then the original arguments.
- 02
Records persist to the configured storage adapter. v0.1 ships MemoryStore and a write-serialised JSON FileStore; SQLite and Postgres adapters are in the roadmap. Filter by confidence, session, user, model, system, and time window through the query API.
- 03
GDPR primitives are exposed as explicit methods: `forgetSession(id)`, `forgetUser(id)`, and `pruneExpired()`. Each returns the number of records removed, giving you evidence to attach to a DSAR response.
- 04
Analyst CLI (`audit-log-llm query|show|forget-session|forget-user|prune`) enables ad-hoc investigation without opening a database client.
Stack
§ 04 · Positioning
Commercial LLM observability platforms typically charge per trace and are US-hosted, which is not viable for UK regulated industries whose customer data cannot leave their control. Self-hosted general-purpose observability tools require additional engineering to add the LLM-specific query patterns and GDPR primitives that this library provides directly.
| Alternative | Trade-off |
|---|---|
| Commercial LLM observability SaaS | US-hosted with per-trace pricing (£50-£500/mo per project). Customer data leaves the controller's control, which is not compatible with UK regulated industries (legal, healthcare, finance). No self-host option in most cases. |
| Custom in-house logging | Consistently under-invests in the GDPR primitives (retention windows, right-to-erasure by identifier). Typically takes a week to build and requires ongoing maintenance. The library provides the same behaviour as an `npm install` and a one-line function wrap. |
| General-purpose observability platforms self-hosted | Solid general logging, but LLM-specific query patterns (confidence bands, prompt version, cost per call) and GDPR primitives still need to be built on top. This library is opinionated on both from day one. |
§ 05 · Frequently asked
How does audit-log-llm handle GDPR right-to-erasure?
Two first-class methods: forgetSession(id) and forgetUser(id). Both return the number of records removed, giving you literal audit evidence to attach to a DSAR response. Retention pruning is handled separately via pruneExpired(), which is safe to cron. Deletion is by exact identifier match — no cascade surprises.
Which LLM providers does audit-log-llm support?
All of them. The library wraps any async function that calls an LLM — the underlying provider is invisible to it. Use the extract callback to pull model name, confidence, and cost from your specific provider's response shape. Anthropic, OpenAI, Groq, and locally-hosted models are all supported out of the box.
What is the storage overhead of adding audit-log-llm to a production system?
Per-record cost is approximately 2-4 KB depending on input and output size. Ten thousand LLM calls per month is under 50 MB — negligible on any modern storage. Retention pruning keeps long-running deployments bounded.