An untraceable AI is a legal and operational risk
The European AI Act, applicable from August 2026, imposes the traceability of high-risk AI systems. But beyond compliance, logging is an indispensable operational tool: it lets you diagnose problems (why did the AI give this answer?), audit decisions (which document was used?), and defend the company in case of litigation. The key components: structured logs for each query, immutable audit trail, GDPR-compliant retention policy, and a search interface for auditors. An AI logging project takes 3 to 5 weeks and integrates into your existing LLMOps stack.
The problem
An employee contests a decision made on the basis of an answer from the AI assistant. The DPO receives a GDPR access request about the data processed by the AI. An external auditor wants to verify that the AI system respects the internal procedures. In each of these scenarios, you must be able to retrieve exactly what the AI answered, on what basis, and when.
Without structured logging, you are unable to respond:
- Regulatory non-compliance — The AI Act (article 12) requires automatic logging capabilities for high-risk AI systems. The absence of logs exposes the company to sanctions that can reach 3% of worldwide revenue. Even outside the high-risk category, internal AI governance recommends complete traceability.
- Diagnosis impossible — A user reports a bad answer. Without a log of the query, the retrieved context and the prompt sent to the LLM, you cannot identify whether the problem comes from the retrieval, the prompt or the model.
- Blurry responsibility — In case of litigation (erroneous legal advice, incorrect financial information), the company must prove that the system was operating in accordance with its design. Without an audit trail, the burden of proof is reversed.
- Blind improvement — Without a history of queries and answers, you cannot identify the failure patterns nor the uncovered topics. Continuous improvement is impossible.
The AI solution
A complete AI logging system rests on three pillars: exhaustive structured logs, an immutable audit trail, and a consultation interface for the different profiles.
Structured logs per query
Each interaction generates a JSON log: timestamp, pseudonymized user_id, question, retrieved chunks (IDs + scores), complete prompt, generated answer, model used, tokens consumed, latency, and feedback. The logs are enriched with contextual metadata (feature, session). Storage on Elasticsearch, Loki or S3 + Athena.
Immutable audit trail
Critical logs (decisions, advice, automated actions) are written to an append-only journal with a cryptographic signature. Each entry contains a chained hash that guarantees the integrity of the sequence. Implementation: Amazon QLDB, Azure Immutable Blob Storage, or PostgreSQL with integrity triggers.
Multi-profile audit interface
A web interface enables consultation by profile: the tech team searches by trace_id for diagnosis, the DPO filters by user_id for GDPR requests, the external auditor filters by date and decision type. RBAC access control to limit the visibility of sensitive data.
Implementation
Setting up an AI logging system takes 3 to 5 weeks and integrates into your existing infrastructure.
Defining the log schema (week 1)
Define the JSON schema in collaboration with the tech, legal and compliance teams. Identify the mandatory fields (query, answer, sources, model, timestamp) and optional ones (feedback, post-answer actions). Define the pseudonymization policy and the retention policy: 6 months for standard logs, 3 years for critical decisions, with automatic purge.
Pipeline instrumentation (weeks 2-3)
Instrument your RAG pipeline to emit a structured log on each query. Use OpenTelemetry for distributed tracing and Langfuse for storage. Implement pseudonymization (SHA256 hash of user_ids) and PII detection (Microsoft Presidio) to remove personal data. Test the completeness of the logs on 100 reference queries.
Audit interface and procedures (weeks 4-5)
Develop the interface with filters (date, user_id, trace_id, type) and CSV/JSON export. Implement RBAC: admin (full), tech (traces without user_id), DPO (access by user_id), auditor (read-only). Write the internal audit procedure and train the teams. Test a complete end-to-end audit scenario.
Results
Results measured after setting up AI logging, complementing AI governance.
FAQ
What information should be logged for an AI assistant?
At a minimum: each user query (timestamped, anonymized), the documents retrieved by the retriever (IDs, scores), the complete prompt sent to the LLM, the generated answer, the model and its parameters (version, temperature), the latency per component, and any user feedback. For high-risk systems (AI Act), add: the user's identity, the decision context and the actions taken following the answer.
Does the AI Act impose a logging obligation?
Yes. Article 12 of the European AI Act requires high-risk AI systems to have automatic logging capabilities allowing the system's operation to be traced throughout its lifecycle. The logs must be kept for an appropriate duration (minimum 6 months recommended, more for certain sectors).
How do you reconcile logging and personal data protection?
Three principles: pseudonymization of user identifiers in the logs (irreversible hash), automatic removal of personal data identified in queries (PII detection via Presidio or regex), and a retention duration aligned with the GDPR (no indefinite retention). Store the logs on European infrastructure.
What is the storage cost of an AI assistant's logs?
An AI assistant processing 1000 queries per day generates around 5 to 10 GB of structured logs per month (queries, contexts, answers, metadata). On object storage (S3, GCS), the cost is around 0.50 to 1 euro per month. On Elasticsearch or a SIEM, expect 20 to 50 euros per month.
For technical profiles
AI logging architecture
The reference architecture relies on OpenTelemetry for collection, a transformation pipeline (pseudonymization, PII detection), and a dual storage backend: real-time (Elasticsearch/Loki) for diagnosis and long-term (S3 + Athena/BigQuery) for audit.
Reference log schema: trace_id, timestamp, user_hash, session_id, query (sanitized), retrieved_chunks (array of {chunk_id, score, doc_id}), prompt_template_version, model, model_params, response, citations, tokens_input, tokens_output, latency_ms, feedback, pii_detected.
Transformation pipeline:
- Pseudonymization — SHA256(user_id + salt). The reverse mapping is kept separately with restricted DPO access.
- PII Detection — Microsoft Presidio scans queries and answers. PII are replaced by tokens ([NAME], [EMAIL]).
- Integrity — Each daily batch is signed with HMAC-SHA256. Any subsequent modification is detectable.
Comparison of logging backends
| Criterion | Langfuse + S3 | Elasticsearch | Datadog | Loki + Grafana |
|---|---|---|---|---|
| Native LLM tracing | Excellent | Via plugin | Good | Basic |
| Full-text search | Basic | Excellent | Good | LogQL |
| Long-term retention | S3 unlimited | Costly | Very costly | S3 backend |
| Immutability | S3 Object Lock | Via snapshot | Managed SaaS | Via S3 |
| Cost (1000 req/day) | ~5 euros/month | ~30 euros/month | ~100 euros/month | ~10 euros/month |