All insights
Principle AI Product Building AI Agents Architecture

Policy enforcement must run independently of model cooperation — hooks, not prompt instructions

Hermes runs lifecycle hooks that block, rewrite, or audit operations at fixed events, so policy and side-effects never depend on the model choosing to comply

@aparnadhinak (Aparna Dhinakaran) — Hermes Harness Architecture · · 10 connections

If your safety and auditing live in the system prompt, they hold only as long as the model cooperates — which is exactly when you can’t count on it. Hermes places enforcement in lifecycle hooks that fire at fixed points (pre/post tool call, gateway dispatch, approval) and can block, rewrite, or pass through any operation, plus filesystem-installed scripts for host side-effects — both designed so policy, auditing, and side-effects execute independently of model cooperation. This generalizes the tool-API case below out to the whole loop, and it’s a concrete expression of how Intelligence location — code vs prompts — determines system fragility and flexibility: deterministic code carries the guarantees while the model carries the judgment, the same division that lets Production agents route routine cases through decision trees, reserving humans for complexity.

The tool-API case comes from reverse-engineering three production Excel AI agents (@nicbstme — “Lessons from Reverse Engineering Excel AI Agents”). Claude embeds overwrite protection at the API level — “the blocking is in the tool, the consent is in the prompt” — while Microsoft Copilot has no overwrite protection at all (it just overwrites) and Shortcut AI relies on system-prompt instructions to prevent destructive operations. Behavioral compliance is inherently unreliable at millions of sessions; tool-enforced safety is deterministic. The tool contract is where to encode safety invariants the non-deterministic model cannot violate, per Tools are a new kind of software — contracts between deterministic systems and non-deterministic agents, and it pairs with Separate tool registration from tool exposure — install broadly, reveal narrowly — the fewer tools a run reveals, the smaller the surface any guardrail has to defend.