The failure nobody demos
The agent demo always works. The customer asks for a refund, the agent calls the right tool, and the money moves. Then it ships, and a week later it refunds an order that was never delivered. Or it issues a refund larger than the purchase. Or it approves an expense for a "manager" who has no reports.
None of these are reasoning failures. The model reasoned fine. It just acted on something that isn't true in your business, because nothing in the system told it what is true.
Prompts ask. Ontologies define.
The usual fix is a longer system prompt: never refund more than the order total; only refund delivered orders. That helps, until the prompt grows to 4,000 tokens and the rule you care about competes with forty others for the model's attention. A prompt is advice. The model follows it most of the time.
An ontology is a different kind of object. It is a formal, machine-readable statement of what can exist in your domain and how things relate: an Order has exactly one Customer, a Refund points to exactly one Order, a Manager has at least one direct report. It doesn't ask the model to behave. It defines the world the model is allowed to act in, and it can be checked mechanically, every time, in milliseconds.
The pattern is simple: the LLM proposes, the ontology disposes. The agent stays flexible in how it understands a request. It loses the freedom to commit an action that contradicts the domain.
A guardrail you can read
The W3C standard for this is SHACL (Shapes Constraint Language). Here is the refund rule as a shape:
ex:RefundShape a sh:NodeShape ;
sh:targetClass ex:Refund ;
sh:property [
sh:path ex:forOrder ;
sh:class ex:DeliveredOrder ;
sh:minCount 1 ; sh:maxCount 1 ;
sh:message "A refund must reference exactly one delivered order." ] ;
sh:property [
sh:path ex:amount ;
sh:datatype xsd:decimal ;
sh:minExclusive 0 ] .
The "amount can't exceed the order total" rule is a few more lines, written as a SPARQL constraint inside the same shape. Notice who can review this. A compliance officer who will never read your Python can read "a refund must reference exactly one delivered order" and tell you whether it's right.
Closing the loop
The guardrail sits between the agent's decision and the side effect:
- The agent proposes an action as structured data (JSON-LD maps cleanly to the graph).
- A validator such as pySHACL checks it against the shapes. No model call, no tokens.
- If it conforms, the action executes.
- If not, the validation report goes back to the agent as a tool result: which shape failed, on which field, with the human-written message.
That last step matters. "Action rejected" teaches the agent nothing. "A refund must reference exactly one delivered order; order 8812 has status Shipped" gives it something to reason about. In practice it either corrects itself or escalates to a human, which is exactly the behavior you wanted the prompt to produce.
What it buys you past the demo
One artifact does three jobs that teams usually build separately:
- Context. The same ontology that validates actions can generate the agent's tool descriptions and entity definitions. The model sees the domain as it actually is, in a few hundred tokens instead of a few thousand.
- Guardrails. Rules are declarative, deterministic, and versioned in git. A rule change is a reviewed diff, not a prompt tweak someone made on a Friday.
- Evals. Every rejection is a labeled failure: shape, field, timestamp. Count violations by shape across a test suite and you know exactly where the agent misunderstands your domain, instead of reading an LLM-judge score of 6 out of 10.
The trade-off is real. Someone has to model the domain, and that is slow, careful work. But the knowledge already exists in your database schema, your policy documents, and your senior people's heads. The ontology is where it stops being tribal.
Where to start
Don't model your whole enterprise. Pick the one agent action that would hurt most if it went wrong — the refund, the record deletion, the outbound email. Write three shapes for it. Put the validator in front of that one tool call and log every rejection for a week.
You'll learn two things quickly: how often your agent was about to do something your domain forbids, and how many of those cases your prompt had already "handled."
Frank Coyle, PhD, teaches generative AI and LLMs at UC Berkeley's School of Information and taught computer science at SMU for 32 years. He trained in neuroanatomy and neurophysiology at Emory before moving to computer science. He spoke on "Ontologies as Guardrails" at AIE Code Summit.
Top comments (7)
Those refund and expense examples are exactly where prompt-only rules break down: a natural-language guideline competes with every other instruction for attention, while a schema or ontology lives outside the model and can be enforced deterministically before an action fires. This mirrors how we treat trusted boundaries elsewhere — the model proposes, but cardinality, referential integrity, and business invariants should be checked in code or a rule layer, not trusted to the model to remember. The practical split is to let the LLM handle intent and selection, and make the schema the last gate that rejects an inconsistent or invalid action before it reaches a tool.
A prompt asks the model to follow a rule; a validator on the tool call refuses the call when the rule breaks. The refund limit belongs in the validator, because the model can lose a sentence from the prompt, while the check still runs each time the call is made.
The refund examples landed for me. A prompt that says never refund more than the order total competes with forty other rules once it grows to 4,000 tokens, so it is easy to see how an undelivered order still gets refunded. I liked the loop where a failed SHACL shape goes back to the agent as a tool result with the exact field, instead of a bare rejection. Starting with the one action that would hurt most if it went wrong feels like the right way to try this without modelling a whole domain first.
the part that stuck with me is the validation report going back to the agent as a tool result with the real order status in the message. a bare action rejected teaches it nothing, agreed. we did something close with typed errors on a refund tool and retries got way cleaner once the error named the field and the current value. my gotcha tbh is shape drift, the shapes slowly disagree with the database after a few migrations and nobody notices until the validator rejects good actions. how do you keep the ontology in sync with the schema, generate it or review it by hand?
I would version the shapes in the same change as the migration, and fail the build if a stored example no longer validates. That way a schema change cannot leave the checker describing a world the database has already left.
"Prompts ask. Ontologies define" is the line that makes this click, and "the LLM proposes, the ontology disposes" is going straight into my notes.
One thing I'd push on: the validation report is written for a compliance officer to read, but it's delivered to the model as a tool result. Those are different readers with different needs. A shape with five failed constraints could easily produce a wall of text a human audits fine and a model drowns in. Have you found that one message per shape works better than one message per violation in practice?
The "how many of those cases your prompt had already handled" line is the one I'd put in bold — that week of logging is the whole argument.
Interesting read! I recently built a small AI agent from scratch in Python as a learning exercise, and separating the LLM from tool validation was one of the most useful things I learned.
One thing I'm curious about is how you would approach this in smaller projects. Would you introduce something like SHACL from the start, or begin with simple Python validation and move to an ontology only when the domain becomes more complex?