Net NinjasBook a call
All talks and takeaways

Template

The AI decision log

A copy-ready log for AI features that touch money or records: what the model proposed, what it saw, and who or what approved it.

The finance pattern from the talk is short: the model proposes, the ledger decides. An AI feature never writes to the system of record directly. It produces a suggestion, and tested code or a named person approves it. This log is the record of that hand-over.

I'm starting to see the same question from banks, auditors, and enterprise buyers: why did the system decide this? A team with this log answers in one query. A team without it rebuilds the answer from chat history and application logs. The table goes into PostgreSQL as it is, and the field list works for any other store.

From the talk: Your AI feature is an architecture problem

The fields

FieldWhat it holdsWhy it's there
request_idOne ID per proposal, carried through every service.It joins the log to the traces and to the system of record.
tenant_id, actor_idThe customer, and the signed-in user or service that asked.It shows on whose authority the request ran.
use_caseA fixed name, such as onboarding-document-check.Evaluation scores and retention differ per use case.
input_ref, input_sha256A pointer to the input and a hash of it.It proves what the model saw without a copy of personal data in the log.
context_refsThe IDs and versions of the retrieved documents.It shows where each statement came from.
model, prompt_versionThe exact model and the version of the prompt.A change in behaviour traces back to a change in one of the two.
outputThe proposal, as structured data.It's what the approver saw.
checksEach deterministic rule and its result.It separates what code verified from what the model claimed.
decision, decided_by, decided_atApproved, rejected, edited, or expired, and by which person or rule.It names the human oversight.
action_refThe ID of the resulting call in the system of record.It links the proposal to what actually happened.
correctsThe request_id of an earlier row, when this row corrects it.The log stays append-only.
input_tokens, output_tokens, latency_msUsage per request.Cost per decision comes from the same row.

The table

PostgreSQL. The approval service writes one row per decision. The last three statements make the log append-only for the application.

ai_decision_log.sql
-- ai_decision_log.sql (PostgreSQL)
-- From the talk "Your AI feature is an architecture problem" (netninjas.eu/events).
-- One row per AI proposal and the decision on it. The approval service writes
-- the row; the model never does.

CREATE TABLE ai_decision_log (
  request_id     uuid        PRIMARY KEY,
  created_at     timestamptz NOT NULL DEFAULT now(),
  tenant_id      text        NOT NULL,
  actor_id       text        NOT NULL,              -- the signed-in user or service that asked
  use_case       text        NOT NULL,              -- a fixed name, such as 'onboarding-document-check'
  input_ref      text        NOT NULL,              -- a pointer to the input in the system of record
  input_sha256   text        NOT NULL,              -- proves which version of the input the model saw
  context_refs   jsonb       NOT NULL DEFAULT '[]', -- IDs and versions of the retrieved documents
  model          text        NOT NULL,
  prompt_version text        NOT NULL,
  output         jsonb       NOT NULL,              -- the proposal, as structured data
  checks         jsonb       NOT NULL DEFAULT '[]', -- each deterministic rule and its result
  decision       text        NOT NULL CHECK (decision IN ('approved', 'rejected', 'edited', 'expired')),
  decided_by     text        NOT NULL,              -- a person, or the ID of the rule that decided
  decided_at     timestamptz NOT NULL,
  action_ref     text,                              -- the resulting call in the system of record, if any
  corrects       uuid        REFERENCES ai_decision_log (request_id), -- a correction points at the old row
  input_tokens   integer,
  output_tokens  integer,
  latency_ms     integer
);

CREATE INDEX ai_decision_log_tenant_time ON ai_decision_log (tenant_id, created_at);

-- Append-only: the application role may insert and read, never change or delete.
CREATE ROLE ai_log_writer NOLOGIN;
REVOKE ALL ON ai_decision_log FROM PUBLIC;
GRANT INSERT, SELECT ON ai_decision_log TO ai_log_writer;

Four rules that make the log worth having

  1. The model gets read-only access. Any action that moves money or changes an account goes through an existing, tested API after approval.
  2. The log is append-only. The application role inserts and reads. A correction is a new row that points at the old one.
  3. Personal data stays in the system of record. The log holds pointers and hashes. A deletion request then touches one system, not two.
  4. Retention has a number. For a high-risk system, the AI Act sets at least six months for these logs (art. 19(1) and art. 26(6) Regulation (EU) 2024/1689). A customer contract often sets a longer one.

Where the law asks for it

High-risk AI systems have to record events automatically over their lifetime (art. 12(1) Regulation (EU) 2024/1689). People have to be able to oversee them while they're in use (art. 14(1)). Credit scoring and insurance pricing are on the Act's Annex III list, and those rules apply from 2 December 2027 (art. 113). A feature outside that list owes none of this. I'd keep the log anyway, because a bank's security team asks the same question an auditor would.

What I leave out on purpose

I leave out an agent that writes to the ledger by itself, a separate audit platform, and full prompts stored with customer data. The first removes the control. The other two add cost and a second copy of personal data to protect.

Next step

I highly recommend bringing one AI decision from your product to a thirty-minute call. We'll trace it through the fields and see which ones your system can't fill today. If your first bank customer sent its questionnaire tomorrow, which field would be empty?

Book a call

These takeaways are an engineer's reading of the rules and the tools. I wrote them to help teams decide what to build. They aren't legal advice. I recommend checking the current text with your own counsel before planning around a date or a threshold.

Let's talk.

A 30-minute call, no pitch. We look at your current setup, your stage, and the thing that's bothering you about it. I give a straight opinion on what to change, what to leave alone, and whether the problem is the one it looks like. What's the one thing in your stack you'd fix tomorrow?