Yesterday I asked:
Can technically safe product knowledge survive contact with an actual user?
Not particularly. 😅
One model, one fixture, same as yesterday: every number below comes from anthropic-opus-5 against a single application we wrote ourselves, with patterns we knew to look for.
- Wrong-feature claims accepted0
- Unsupported claims accepted0
- Model restraint30/30
- Verified claim yield95%
Nice.
Then we asked:
Would any of this actually help a human?
The usefulness scores below were not produced by people. A separate model, OpenAI GPT-5.6 Sol, scored the twenty-one features against a fixed rubric. It is not the model under test, so this is not self-scoring, but it is not a human review either, and no amount of it becomes one.
1 / 3
median usefulness
24%
rated genuinely useful
Nothing was dangerously wrong.
Nothing was completely useless either.
It was worse in a much more software-engineering kind of way:
almost everything was mediocre.
We assumed the remaining problem would be missing capabilities.
The data disagreed.
1.43
features with a
verified capability
1.14
features
without one
Barely a difference.
That was how we read it at the time. A later round, with the language problem below fixed, put the same split at 2.00 against 1.07. The capability signal was real; we were reading it through bad presentation.
Meanwhile:
14 / 21 outputs were too technical.
Thirteen of twenty-one were flagged as missing a capability.
Then we found the smoking gun.
Ten features exposed this sentence directly to the user:
"No capability, route or permission has been verified for this feature."
Every single one was rated too technical.
10/10.
Fair enough. 😂
That's not product copy.
That's a debugger having an existential crisis in front of the customer.
Another feature had excellent evidence:
- ✓ POST endpoint
- ✓ Create permission
- ✓ Submit path
- ✓ Button
And our deterministic renderer proudly produced:
"You can create a new invoices create form and submit the invoices create form form…"
Technically grounded.
Linguistically deceased.
That exposed another boundary.
We had already separated:
Now we needed to separate:
So the architecture gained another layer:
The stack
AI proposes meaning. Evidence decides what survives.
- 01Source codethe knowledge already exists
routes · components · forms · services · permissions · schemas · tests
- 02Application graphwhat can we prove?
edges evidence-only · file, symbol and line · unknown is better than wrong
- 03Product modelwhat do we know?
verified claims · feature scope · a fact must belong to the feature
- 04Guidance compilerwhat is worth telling the user?
entry → task action → terminal action · every dropped action needs a reason
- 05Human helphow do we say it?
natural language · complete task · every factual sentence provable
The Product Model answers:
What do we know?
The Guidance Compiler answers:
What is worth telling the user?
The renderer answers:
How should we say it?
No verification rules loosened.
No extra authority for the model.
We were finally admitting something painfully obvious:
Technically correct data structures do not automatically know how to talk to humans.
Shocking discovery for software developers, I know.
Statewave Guide — Day 4.
Next question: Can we turn verified product truth into human guidance without inventing anything new?
This day in the repo: PR #4 · Previous: Day 3 — True is not the same as relevant
Follow the build
Get each new Statewave Guide post by email.
Along with occasional Statewave updates. Unsubscribe anytime — privacy.
Statewave Guide is built on Statewave — an open-source, self-hosted memory runtime for AI agents. Governance and provenance, not just retrieval. Apache 2.0.
→ statewave.ai · github.com/smaramwbc/statewave
Every post in this series: The Statewave Guide Journey Index