Field Note: Where You Keep the Final Say

I keep a tool wired into my AI setup that can read the actual text of Spanish law — the BOE, the official state gazette. In layperson terms it’s a connector (an “MCP server”) that lets the AI check a real and validated source, instead of answering from memory or confabulating / hallucinating. I built it and put it on GitHub, for exactly the kind of moment I hit this week: a question about a Spanish regulation where the answer matters and getting it wrong has a cost.

So, while I was working through my regulation question, the AI could have asked the BOE tool straight away and settled it in one go, but it didn’t. Instead, it reached for what it already “knew” and gave me a confident answer from memory. The official source was sitting right there, one query away, and it strolled past it down Las Ramblas. I’d built the thing for this exact moment and then watched it get ignored.

So when I double-checked the real source, the confident answer it originally gave turned out to be off the mark somewhat. No real harm done in the end, but it stuck with me, because it’s the opposite of the failure everyone warns you about.

The worry you usually hear about is that an agent will do too much: go off and act on something it shouldn’t, which is also a reality. Case in point: one builder put his own AI assistant online and invited the public to try to talk it into misbehaving. Security peeps call it prompt injection. The model can’t reliably tell your instructions from text it picked up along the way, so a stranger’s note can read like an order from you.

That’s a tool failing in two opposite directions. It won’t open the official rulebook I handed it, but it’ll potentially take a stranger’s note as an authoritative command. It doesn’t know which source to trust.

2026 is certainly the year of ‘agentic AI’, but the common discourse talks about “connecting AI to your tools” as if the connecting is the hard bit. Sure, it’s technical, but it’s fundamentally no different than installing any other bit of software. The hard bit is the judgment: knowing when to reach for the authoritative source, and when to be suspicious of the thing in front of you. The agent doesn’t have that yet. So it has to come from somewhere.

It comes from you.

The same week, the creator of htmx wrote up almost this exact thing from the coding side: he had Claude chase down a nasty parser bug — sharp at finding the cause, sharp at writing the tests — and then it handed him three fixes, each wrong in its own way. The clean one only happened because he knew the code well enough to overrule it. That’s the same shape: the tool does the legwork, the judgment stays human.

That’s why none of my setups send, post, or file anything on their own. They draft; I check; and the send rests with me. People hear that and assume I don’t trust the tech, but it’s the other way round. I trust it plenty, but for the things it’s good at. I just know it can’t yet tell the official source from the confident guess, or the real instruction from the planted one. That judgment is the part I keep. The tools are here to augment what I do, not stand in for me.

Which makes the real question less “how much can it do?” and more “where do I keep the final say?” Get that one right and the rest gets a lot safer.

So, before the rest of the week: where in your own setup does the AI get to act without you checking first?

Also this week

Hand your agent the calls you’ve already made. A small open-source tool, rac-core, feeds a coding agent the decisions your team has already settled so it stops re-litigating them — product knowledge treated like code. Same idea as the BOE server, pointed the other way: give it the authoritative record instead of letting it guess.

— Jim

The system I keep the final say over: the member hub and most of my task setup live in Notion. If you’re a startup or solo founder, you can get 3 months of Notion Business free — unlimited AI, no credit card through my link. Start your 3 free months →​

Notion affiliate link — I get a small credit if you start a trial