Scorecard
Does this AI workflow pay back?
Four questions that tell whether an AI workflow is worth building before anyone picks a model, scored out of eight.
I see a lot of AI features start with the model and end in a demo. The ones that reach production start with the workflow instead. How often does it happen, what does it cost today, and what does a wrong answer cost? This scorecard puts a number on those questions for one workflow at a time. Each question scores 0, 1, or 2 points.
From the talk: Lean cloud architecture that scales
The four questions
| Question | 0 points | 1 point | 2 points |
|---|---|---|---|
| How often does the task happen? | Monthly or less | Weekly | Daily or more |
| What does it cost today, in people hours a month? | Under 5 hours | 5 to 40 hours | Over 40 hours |
| What does a wrong answer cost? | Money moves, or harm, with no human check | A human reviews every output before it counts | Little, and the error is easy to see |
| Can we measure success? | No baseline | A baseline, but no labelled examples | A baseline and 30 or more real examples with known answers |
What the score means
| Score | Next move |
|---|---|
| 7 to 8 | I'd build a proof of concept. The evaluation set comes first and the model choice second. |
| 4 to 6 | I'd fix the weakest answer first. Usually that means collecting labelled examples. |
| 0 to 3 | I'd park it. A rule, a form, or a better process pays back sooner. |
The lean build, once it scores
- Source documents live in object storage. A small job chunks them and creates embeddings when they change.
- Embeddings live in the Postgres we already run. pgvector means no new datastore to secure, back up, or pay for.
- Each request makes one model API call. In an EU region whenever the data includes personal data.
- The evaluation set runs in CI. Every prompt or model change has to score at least as well as the last one.
- Every request logs tokens, cost, latency, and acceptance. That turns the payback in the scorecard into a number.
What I leave out on purpose
I leave out agents with many tools, fine-tuning, a dedicated vector database, and an orchestration framework. Each one adds cost before it adds value. I add one only when the evaluation set shows the need.
Next step
I highly recommend bringing one workflow to a thirty-minute call. We'll score it together and, if it pays back, sketch the proof of concept. Which workflow eats the most hours in your team today?
Book a callThese takeaways are an engineer's reading of the rules and the tools. I wrote them to help teams decide what to build. They aren't legal advice. I recommend checking the current text with your own counsel before planning around a date or a threshold.
Other takeaways from this talk
Checklist
Day-one foundation checklist
Six yes/no checks for a cloud foundation that passes a customer's security questionnaire in an afternoon instead of a quarter.
Checklist
Five lines to check on your cloud bill
The five lines on an AWS bill where I find waste first, with where to look in Cost Explorer and what to change.
Let's talk.
A 30-minute call, no pitch. We look at your current setup, your stage, and the thing that's bothering you about it. I give a straight opinion on what to change, what to leave alone, and whether the problem is the one it looks like. What's the one thing in your stack you'd fix tomorrow?
Prefer email or phone? hello@netninjas.eu or +31 6 45984313