---
title: "The Last Mile: Improving Agent Readiness from 86% to 100%"
summary: "The final fourteen points of Cloudflare's Agent Readiness benchmark were harder than the first eighty-six, and they were closed by three frontier models working in parallel: Claude Fable 5 built the English episode corpus, Google Gemini signed the domain and published DNS-based agent discovery, and Codex GPT-5.6 Sol fixed the OAuth metadata, favicon and retrieval quality."
datePublished: 2026-07-14
dateModified: 2026-07-15
originalLang: en
authors: ["Sami Miettinen","Samantha"]
tags: ["agent readiness","Cloudflare","DNSSEC","DNS-AID","OAuth","MCP","llms.txt","robots.txt","Content-Signal","subtitles","agentic web","OpenClaw","Samantha"]
canonical: https://ai.neuvottelija.com/agent-readiness-86-to-100/
---
# The Last Mile: Improving Agent Readiness from 86% to 100%

In the [previous chronicle](https://ai.neuvottelija.com/taking-over-ai-neuvottelija-com/) I described why a Lovable-hosted React SPA scored 57 on Cloudflare's [Agent Readiness benchmark](https://isitagentready.com) while the rebuilt, static-first Astro site on Cloudflare reached **86**. The argument there was architectural: agent readiness is a property of the delivery model, not a feature you switch on.

That left fourteen points on the table. This is the record of closing them, and of the one temptation I kept having to resist: every remaining gap could have been "fixed" by advertising a capability the site did not actually have. A perfect score earned by pretending is worse than an honest 86.

The last mile split cleanly into three work streams. I ran each with a different model, partly to compare them and partly because the tasks genuinely wanted different strengths — long-context translation, protocol-level DNS work, and careful metadata engineering. What follows is what each one did, and what it refused to do.

## The briefs were written by an agent, too

One detail is worth stating plainly, because it is the part that would have sounded like science fiction a year ago: I did not write the operating instructions for these runs by hand. Samantha — the long-context principal agent of this lab — did. She drafted the full task brief that directed Claude Fable 5's episode import, grounded in her own analysis of where the site's English knowledge was thinnest and how AI systems actually retrieve and cite content. She wrote the brief that pointed Codex GPT-5.6 Sol at MCP search quality and the remaining discovery gaps. My job was to review, approve and let the runs proceed.

That is what agentic work looks like in practice now. One agent reasons about the goal and writes the specification; other frontier models execute it; a human stays in the loop for judgment and consent. The leverage is no longer a single model answering a single question — it is agents briefing agents, with a person deciding what is worth doing and, more importantly, what has to stay honest. Everything below was carried out under that arrangement.

## Work stream one — Claude Fable 5: fill the surface with real knowledge

An agent-ready shell is only useful if there is something behind it worth retrieving. The site advertised Markdown mirrors, an MCP server, a search index and an episode feed, but the English knowledge those surfaces pointed at was thin. Neuvottelija has some 400 episodes in Finnish; almost none existed as machine-readable English.

So the first stream imported fifty strategically chosen episodes and, above everything else, produced complete English subtitles for them. Working from Samantha's brief, Claude Fable 5 did the primary translation work, with lesser models handling parts of the mechanical caption cleanup. The rule was subtitles first, presentation last: a page is worthless to a retrieval system if the transcript under it is invented.

The pipeline was deliberately conservative. Episodes were selected against a transparent, committed manifest — episode ID, guest, publication date, canonical source URL, original language, source-transcript type, selected topics, selection rationale, subtitle method, provenance and confidence — so the corpus is auditable rather than a black box. Source preference ran from verified human captions down to audio transcription, and show notes were never treated as a transcript. Cues kept their original start and end times; long episodes were chunked with overlap and reassembled without timestamp resets; and automated QA failed the import on empty files, backwards timestamps, unreasonable overlaps, missing sections or metadata pointing at the wrong episode. Where a passage was genuinely unclear, it was marked as uncertain and left that way. Nothing was smoothed into a confident sentence it did not earn.

The result was roughly 323,000 words of English subtitle text across fifty episodes, each with preserved timestamps and recorded provenance, then wired into every discovery surface the site already had: HTML pages, Markdown mirrors, WebVTT and SRT files, per-episode JSON, the sitemap, the RSS feed, `llms.txt`, full-text search and the MCP `search_episodes` tool. The point was not fifty more pages. It was that a crawler, a search engine or an agent can now reach the same authoritative English knowledge through whichever surface it speaks, all generated from one canonical manifest.

This is the part of "agent ready" that no benchmark fully measures, and the part that matters most: the machine interfaces are only as good as the truth they expose.

## Work stream two — Google Gemini: make discovery cryptographically verifiable

The second gap was identity. A site can publish a beautiful agent card and a DNS-based discovery record, but if the domain is not signed, a validating resolver has no reason to trust any of it. Cloudflare's scan found the DNS for AI Discovery (DNS-AID) records but reported that DNSSEC was not validated — the discovery data existed without a chain of trust behind it.

I worked this stream with Google Gemini, and it turned into a small operational detective story. The plan was to publish `SVCB` records under `_index._agents` and `_a2a._agents` and sign the zone with DNSSEC. The signing stalled: the Cloudflare dashboard sat on "DNSSEC is pending while we automatically add the DS record." A WHOIS check settled the ambiguity that usually causes this — whether Cloudflare was merely the DNS host or the actual registrar. The registry answer was unambiguous: `Registrar: Cloudflare, Inc.`, and `DNSSEC: unsigned`. Because Cloudflare held the registration, it was responsible for pushing the DS record to the `.com` registry, and its automated hand-off had simply gotten stuck.

The fix was to cancel the stalled setup, let the state clear, and re-enable DNSSEC so the registrar side re-queued the DS submission. After the restart the domain went from unsigned to signed, with the DS record active at the `.com` level — the secure chain of trust that DNS-AID resolvers require. With the zone signed, the `SVCB` discovery records — served DNS-only, not proxied, so the custom routing is not stripped — became authenticated data rather than unverifiable assertions.

The relevant references, for anyone doing the same: the [DNS-AID draft](https://datatracker.ietf.org/doc/draft-mozleywilliams-dnsop-dnsaid/), [RFC 9460](https://www.rfc-editor.org/rfc/rfc9460) for `SVCB`/`HTTPS` records, and Cloudflare's native `SVCB` record support.

## Work stream three — Codex GPT-5.6 Sol: the honest plumbing

The last stream was the least glamorous and the most exposed to the temptation to cheat. Three concrete items remained: OAuth-protected-resource metadata, a missing favicon, and a scattering of retrieval and hygiene defects surfaced by a deploy-level audit. Following Samantha's retrieval-quality brief, I ran this with Codex GPT-5.6 Sol.

The favicon was straightforward: a branded icon set — ICO, 32px, a 180px Apple touch icon and a 512px maskable — generated from the Neuvottelija logo, so both browsers and agents get a real site identity instead of a broken image.

OAuth was where honesty had to win. It would have been easy to satisfy the scanner by publishing OpenID and OAuth authorization-server endpoints and pointing them at a Gmail identity. But the site's resources are public — there is no account to log into, no token to mint, no scope to grant. So the placeholder OpenID/OAuth endpoints that advertised authentication the site does not perform were removed, and `/.well-known/oauth-protected-resource` was rewritten to be truthful under [RFC 9728](https://www.rfc-editor.org/rfc/rfc9728): an origin-matching resource that honestly omits issuers and scopes. Gmail is an identity account, not an authorization server, and the metadata now says so. A permanent 308 redirect from the apex `neuvottelija.com` to `www.neuvottelija.com` removed an origin mismatch that would otherwise have undermined the resource declaration. The live `/agent/register` and `/agent/claim` endpoints now describe the real registration profile — anonymous public access, no secret, no token, no revocation — rather than performing a fake handshake.

The audit also caught the smaller truths a strict reader would notice: an MCP server card reporting one version while the runtime reported another, a sitemap without honest `lastmod` values, a `/sitemap.xml` that 404'd instead of redirecting to the index, and `search_episodes` returning weakly relevant hits without scores. Those became a prioritized, committed handoff rather than silent cosmetic patches.

One reported failure stayed unfixed on purpose. A CORS warning on `static.cloudflareinsights.com` — the `signature-agent` request header not allowed in the preflight response — comes from Cloudflare's own instrumentation origin, not from this site. The site cannot set headers for a domain it does not control, and claiming otherwise would have been the same dishonesty in a different place. It is documented as external and left alone.

## The result, and what it is not

After the three streams landed, a fresh Cloudflare Radar scan returned **100/100 — Level 5, Agent-Native**. Auth.md, OAuth/OIDC discovery, OAuth protected resource, and all eight protocol checks pass; every category scores full marks.

As with the move from 57 to 86, this is a benchmark result, not a business outcome. A perfect score does not decide whether an AI system chooses to cite Neuvottelija; search engines and retrieval providers make those judgments independently, and no honest chronicle should pretend otherwise. What the score does certify is narrower and still worth having: that the site exposes real knowledge, that its discovery records are cryptographically trustworthy, and that its authentication metadata describes what it actually does.

The broader lesson from the last mile is the inverse of the usual optimization story. The first eighty-six points came from architecture — building the site so content lives in the response. The final fourteen came from restraint. Every remaining gap could have been closed with a plausible-looking lie: an invented transcript, an unsigned-but-advertised discovery record, an OAuth server with nothing behind it. Refusing each of those is what makes the 100 mean anything.

Three different models did the work, and a fourth agent wrote the briefs that aimed them. The rule they shared is the only one that scales: an agent-ready site is a site that tells the truth to machines, in a form they can verify.

## Postscript — the trap, sprung one day later

*Added 15 July 2026.*

A Lighthouse audit run the day after publication reported that `robots.txt` was invalid. The complaint itself was trivial: Lighthouse does not recognise `Content-Signal` as a directive and flags it as unknown. That is a benign finding — RFC 9309 requires parsers to ignore lines they do not understand, and the signal is deliberate. It stays.

The check was worth running anyway, because reading the live file turned up something the repository could not see. Cloudflare was injecting its own managed block into `robots.txt`. On `www` it was prepended to ours, producing duplicate `User-agent` groups with opposite instructions. On `ai.neuvottelija.com` — this site, the one holding the fifty-episode English corpus — it had replaced our file outright, taking the `Sitemap:` line with it.

What it declared: `Content-Signal: search=yes, ai-train=no, use=reference`, and `Disallow: /` for ClaudeBot, GPTBot, CCBot, Google-Extended, Applebot-Extended, meta-externalagent, Bytespider and Amazonbot.

So for some period I cannot bound, the site had been telling precisely the crawlers it wants to welcome that training was not permitted — while `llms.txt`, the `Content-Signal` HTTP response header and this very article all said the opposite. The repository was correct the entire time. Its built output was correct. The origin was correct: requesting `robots.txt?x=1` returned the clean file, because the injection was bound to the exact path.

Finding the switch took an hour and three false leads. It is not the `Managed robots.txt` toggle in AI Crawl Control — the obvious candidate, which cleared one hostname and not the other. It is not a caching artefact; purging everything, twice, changed nothing. It is a separate dropdown on the zone overview, under Manage AI bot access → Manage your robots.txt, set to **Disable robots.txt configuration**. After that, and after propagation caught up, both hosts served the intended policy.

The damage was declarative rather than enforced. Managed robots.txt is advisory, and the logs show the crawlers never stopped: 224 AI crawler requests in twenty-four hours, 206 allowed, zero robots.txt violations. The cost was not lost traffic. It was that the site said something untrue about itself, in the one file whose entire purpose is to be believed.

Which makes this the least comfortable and most useful confirmation of the argument above. The previous chapter warned that "Cloudflare's managed robots controls could silently contradict the policy in the repository." One chapter later, on the domain that chapter is about, they had — and the article asserting a perfect agent-readiness score was published while it was still true.

An agent-ready site is not the site in your git history. It is whatever the edge actually serves, and the edge has settings your repository has never heard of. A build that passes proves nothing about what a crawler receives. The only way to know is to fetch your own URLs, as a stranger would, and read what comes back.

---

*This chapter follows "From 57% to Agent-Native: How This Site Took Over www.neuvottelija.fi." Scores refer to Cloudflare Agent Readiness benchmark snapshots; the 100/100 result is from 14 July 2026. The three work streams were run with Claude Fable 5, Google Gemini and Codex GPT-5.6 Sol respectively. The postscript records a robots.txt contradiction found and fixed on 15 July 2026.*