Why it matters more for answer engines
Search engines have spent two decades learning to tolerate messy sites. Googlebot renders JavaScript, follows long redirect chains, and guesses at canonical URLs. Answer engines mostly do not:- Most AI crawlers do not run JavaScript. A page whose content arrives only after a script runs is empty to OAI-SearchBot, PerplexityBot and Claude-User. Nothing about the copy can compensate.
- robots.txt is a hard wall. A brand that disallows
GPTBotorPerplexityBotis absent from those engines, however good its pages are. Many sites did this in 2023 and never revisited it. - Answer engines quote passages, not pages. They need clean, extractable text, one clear answer per section, and metadata that says what the page is. Noise in the markup becomes noise in the answer, or no answer.
- Fewer crawls, less patience. An AI crawler visits a page less often than Googlebot does. A redirect chain, a slow response or a buried page costs more when there are fewer chances to get it right.
What Narrative does
1
Crawl your whole site
Narrative’s own crawler, NarrativeBot,
walks your site the way an engine does: it reads robots.txt and your
sitemaps, follows internal links, honours crawl delays, and records every
page, link, redirect and response. It does not sample. From the result it
builds your internal link graph.
2
Audit every page against the library
Each page is judged against the audit library:
per-page checks that read the page itself, and whole-site checks that
read the graph, because a page cannot know what links to it or that
another page is a copy of it. Field performance comes from
Chrome’s real-user data.
3
Show you the state of every page
Site Health (under Context in the sidebar) lists every page with what
is wrong with it, what it served, how deep it sits, how much internal
authority reaches it, and how fast it was. Group by page, by issue, or
walk the link graph. Site-wide vitals sit on the Analyze page.
4
Turn findings into Repairs and Fixes
From Site Health you open a Repair:
one family of issues, every page affected, plain-language steps. Each
page inside it is a Fix that tracks its own state. Repairs live under
Projects next to your campaigns.
5
Re-crawl and close what is fixed
Run the crawl again whenever you want. A Fix the audit can no longer see
closes itself. One that still shows is recorded as still observed, with
the date. You never have to tick a box to say a problem is gone.