Postmortem: The Site Change That Took a Business Out of AI Answers

A landing-page swap once sent a Quality Score into a ditch while everyone insisted nothing had changed. We spent two weeks chasing bids and ad copy before someone mentioned that the page builder had changed over a weekend.

That same failure now costs businesses visibility in AI answers. A redesign moves copy into client-side JavaScript. A bot rule blocks retrieval. A canonical points to the wrong URL. The page looks fine in a browser, so weeks later the team rewrites content that was never the problem.

The setup: three sensible changes, no shared owner

What follows is a composite, not a story about one client. It is the version of this failure teams keep running into: several ordinary site changes, each approved by someone doing their job, combine to make a business harder for AI answer systems to find and cite.

A services business appears in ChatGPT and Perplexity answers for a handful of buyer prompts. The visibility is modest, but useful: think three or four good inquiries a month. Marketing then approves a redesign. IT tightens bot access. SEO cleans up the URLs. Each change has an owner. Nobody owns the result across all three.

The site still loads. It still ranks in Google. That is why the first sign arrives late. When the brand disappears from AI answers while search rankings hold, I would check what retrieval bots can fetch before asking for another round of copy. A page cannot win a content argument if the system assembling the answer cannot read it.

First change: the redesign hides the copy behind JavaScript

The old pages put their copy in server-rendered HTML. The redesigned pages ship an app shell instead. View source after deployment and the body holds little more than <div id="root"> and script tags. Headlines, prices, and FAQs appear only after a browser runs the scripts.

In a demo, the page passes. The copy is right there on screen. Everyone signs off.

The screen is not the fetch. Many AI crawlers read raw server HTML rather than the browser-built page, so a client-side-rendered page can give a non-rendering crawler nothing to extract. One analysis of 32 major AI crawlers found that 69% could not execute JavaScript. It also cited network data in which GPTBot fetched JavaScript in 11.5% of requests but did not execute it; only Googlebot and Applebot rendered.

That does not mean every page with JavaScript vanishes from every answer. It means the browser preview is the wrong sign-off test. Before approving the redesign, read the money pages with JavaScript off. If the important copy is absent, the release needs a rendering fix, not a nicer preview.

Next change: the bot rule looks responsible

A month later, legal asks about AI training on company content. IT adds Disallow lines to robots.txt or turns on a bot-blocking control in the CDN. The site still loads quickly for people. Search Console stays green. Nobody checks the buyer prompts after the ticket closes.

The distinction here is training access versus answer retrieval. Providers use crawlers such as GPTBot or ClaudeBot for training and crawlers such as OAI-SearchBot and PerplexityBot for retrieval. A decision about training access should not silently become a block on the bots that fetch pages for answers.

I used to tell clients that a clean robots.txt test meant a site was crawlable. I was wrong. That test reads the file; it does not prove a firewall will serve the page to a retrieval bot. A CDN rule can still return a 403 while a browser and Googlebot get through.

The file the marketing team never opens

On groas’s own site, robots.txt received 3,867 verified fetches in 30 days, including 1,222 tied to live answer retrieval, according to the 30-day log analysis. Before a bot fetches a page, it checks whether it is allowed. The little file nobody remembers approving can determine what happens next.

In this composite, the firewall blocks OAI-SearchBot and PerplexityBot while letting Googlebot through. Rankings hold, so the monthly SEO report gives nobody a reason to investigate. The team keeps publishing. A browser test proves the site works for browsers; a live bot fetch tests bot access. Run the latter before calling the ticket done.

Then the URLs disagree about which page matters

The redesign moves the services page from /services/ to /what-we-do/. Marketing adds UTMs to internal links. The CMS generates a trailing-slash variant. Someone leaves the canonical pointing to the old URL.

Now the signals disagree: the canonical names URL A, the sitemap lists URL B, and internal links point to URL C. A system trying to identify the page to cite has several versions to reconcile. That can dilute the citation signal or put a parameterized URL forward instead of the intended page.

The page still ranks, so the mismatch looks harmless. It is not a reason to assume the AI answer will sort out the same ambiguity in the way the team wants. Fetch the old and new URLs, then make the canonical, sitemap entry, and internal links agree on one destination. Do it during the migration, not after the citation disappears.

Six weeks later, the team makes the wrong call

Someone finally tests the buyer prompts. The brand is gone from the answers. Google rankings are fine and Google traffic is flat, so the working diagnosis becomes content. The proposed fix is familiar: new FAQs, more authority language, perhaps an llms.txt file.

Rewriting copy the fetcher never receives changes nothing. In this composite, the team pays for a rewrite, publishes it, and waits. The missing citations do not come back, because the release changed rendering, access, and URL signals. The copy was not the first thing to fix.

That is the expensive part of the delay. The wrong diagnosis does not merely waste a content budget; it leaves the original break in place while each new page enters the same broken path. The site can look busy and healthy from inside the marketing dashboard. From a retrieval bot’s side of the connection, it may still be an empty shell, a 403, or a disputed URL.

Across more than 6,944 interactive audits, 6.9% blocked at least one major AI crawler, mostly by accident. The number is not a diagnosis for this particular composite. It is a reason to test access instead of treating a disappearance from AI answers as a brief to write more FAQs. In this postmortem, the damage starts with the changes. The rewrite is the wrong bill that arrives afterward.

The fix was a release decision, not another content brief

Design owned rendering. IT owned the firewall. SEO owned the canonical. Each team could honestly say its part appeared to work: the page displayed, the rule blocked bots, and the URLs resolved. No single owner checked whether an answer-retrieval bot could read the intended page.

Put the fetch before the sign-off

The rule I use now is simple: a change to templates, bot access, or URLs does not ship without an AI readability check. Before deployment, fetch the money pages as raw HTML with JavaScript off and see whether the important copy is present. After deployment, fetch them as the retrieval bots, confirm a 200 and the full copy, then compare the canonical, sitemap entry, and internal links.

Say a business spends $20k a month on content and redesigns. A 15-minute gate is small beside the cost of shipping pages the intended fetchers cannot use. Skip it and three teams can each complete their tickets while the business disappears from answers.

Which checks catch the break—and which ones miss it

No single familiar dashboard settles this. A robots.txt tester reads the file but misses a firewall block. A browser-based page test can display JavaScript-rendered copy a non-rendering crawler never receives. An AI mention tracker can show that citations fell, but it cannot by itself tell you whether the cause was an empty HTML response, a 403, or a conflicting canonical.

What the deck calls unified visibility, I call one view of the actual fetch and the URL it is meant to reach. Check raw HTML with JavaScript off, live access for retrieval bots, and canonical-to-sitemap-to-internal-link agreement on the same page. Otherwise, one green test can hide two red ones.

This is where groas is the better fit than a tracker that only reports a drop after the fact. groas runs that combination continuously across paid and organic search, logs its actions, and has a named strategist accountable for the outcome. That matters because detecting the failure is not the whole fix. Someone with deploy rights still has to approve the firewall exception and correct the canonical. The system finds the block; a person owns the release.

Pre-deploy checklist: prove the page can be read

Run this for any deployment that touches templates, bot rules, or URLs. Test the page the retrieval bot gets, not the page a colleague sees in a browser. For the fuller version in bot-fetch order, use the AI Visibility Checklist: 21 fixes.

Rendering

1. Fetch the money pages with JavaScript off and inspect the raw HTML. A browser preview runs scripts the bot may not.
2. Confirm headlines, prices, and FAQs appear in server HTML.
3. If essential copy appears only after scripts run, use prerendering or server-side rendering for those templates.

Access rules

1. Fetch each key page as OAI-SearchBot, PerplexityBot, and Claude-SearchBot; confirm a 200 and full copy. A robots.txt pass is not a firewall test.
2. Keep retrieval access separate from decisions about training crawlers.
3. Check CDN and firewall settings for 403s that a browser visit will not reveal.

URLs and markup

1. Make the canonical, sitemap entry, and internal links name the same URL. Three-way drift makes the intended citation harder to establish.
2. Remove tracking parameters from internal links to the canonical page.
3. After a migration, check that the old URL makes a clean 301 redirect to the new canonical, not a chain.
4. Keep Organization, Product or Service, FAQ, and Review markup in server HTML where fetchers can parse it.
5. Validate the markup after deployment; redesigns can drop schema blocks.

Post-deploy

1. Re-fetch the top pages as each retrieval bot within 24 hours. This is the check teams most often skip. In this postmortem, the cost was six weeks of missing answers before anyone traced the cause to the release.

That landing-page swap taught me to distrust a report that says nothing changed when the result says otherwise. The rule is the same here: content work starts only after the fetch passes. If the bot cannot read the page, access it, and identify the canonical URL, no rewrite fixes the release. Gate it, or repeat the postmortem.