Why Crawlers Saw an Empty React Site (and How We Prerender the HTML They Actually Read)
The articles were 2,000 words. Curl returned a title, one sentence, and a chat widget. AdSense called that low value. Here is the exact failure on simeoncreatives.com and the build that ships the real body in the first HTML.
AdSense said the site had low value content. The blog had 36 articles. Several were longer than 3,000 words. Curl told the truth the reviewer saw.
A flagship SEO guide returned about 12 kilobytes of HTML. The body was a heading, a meta description, and a third-party chat widget. The rest of the article existed only after React replaced the root node. Users were fine. Reviewers were not.
The host was a React single-page app on Cloudflare. Users got the designed page. Reviewers got the stub. That is what we shipped, and what we had to undo.
What curl actually returned
Every prerendered route used the same pattern: an empty #root, then a “crawler landmark” stuffed with the title and description, then CSS that clipped it to a 1×1 pixel box. React wiped #root on boot, so visitors never kept the duplicate heading. Bots that skip JavaScript never left the stub.
<div id="root">
<!-- Crawler-only copy, off-screen, then React wiped #root -->
<main data-seo-shell="true">
<h1>What Is SEO? A Beginner\</h1>
<p>A plain-English intro to SEO for founders…</p>
</main>
</div>The live CSS was the classic hidden-text kit: one-pixel box, overflow hidden, clip rectangle. Two problems sit in that snippet. Hidden copy. Thin copy (one sentence where a 16-minute article should be). AdSense’s rejection links pointed at both: webmaster quality guidelines, and thin content.
The other bugs the same shell carried
Titles truncated on apostrophes
The generator did not import the post module. It scanned the file with a single-quoted string regex. Four titles containing an escaped apostrophe died mid-word, including two of the strongest guides. Open Graph, Twitter, the H1, and BlogPosting.headline all inherited the broken string.
Every URL was HTTP 200
Cloudflare was set to single-page-application not-found handling. /this-page-does-not-exist and /blog/fake-post both returned 200 with the thin shell. Google logs those as soft 404s. A reviewer sees an infinite supply of empty pages.
Canonicals fought the trailing slash
The sitemap listed slashless URLs. The host 307-redirected them to a trailing slash. The trailing-slash document’s canonical pointed back at the slashless URL. That is a loop. Coverage treats it as a page with a redirect and often skips indexing.
Placeholders on live hubs
“In the library soon” resource cards and empty curriculum sections read as an unfinished site. They were honest internally. They were a gift to a human reviewer looking for thin.
The rule we locked
The first HTML must contain the same words a visitor can read after the bundle runs. Not a summary. Not a hidden landmark. The article.
React can still hydrate and replace #root. That is fine. The difference is what exists before JavaScript.
flowchart LR
Build["vite build"] --> Shell["generate-seo-html.mjs"]
Shell --> Dist["dist/{path}/index.html
full visible body"]
Dist --> Bot["Crawler, no JS"]
Dist --> App["React replaces #root"]
Bot --> Read["Reads the article"]
App --> User["Sees the designed page"]How the generator works now
- Load data with Vite, not regex: ssrLoadModule on blogPosts, hubs, checklists, and case studies. Titles with apostrophes stay intact because they are real strings.
- Serialize blocks to HTML: src/utils/prerenderHtml.js mirrors the Markdown serializer. Headings, lists, tables, figures, FAQs, code. New block types must land in both files.
- Write a file per route: dist/blog/{slug}/index.html, hubs, work studies, resources, policy pages. Policy copy that lives in JSX is renderToStaticMarkup of the page component.
- Leave the body visible: normal flow, readable type, no clip. A nav list so a non-JS crawler can walk the site.
Media still sits on Cloudflare R2. The HTML is on the Worker. That split is unchanged from R2 for websites.
Cloudflare settings that must stay
| Setting | Value | If you flip it |
|---|---|---|
| not_found_handling | 404-page | Unknown URLs become 200 thin pages (soft 404s). |
| html_handling | drop-trailing-slash | Sitemap URLs 307 to a twin whose canonical points back. |
404-page only works because every App.jsx route has a generated index.html. Add a Route without a matching shell and production 404s that URL. That is the trade. It is the correct trade.
How to verify before you call it shipped
curl -sL https://simeoncreatives.com/blog/what-is-seo-a-beginners-guide \
| python3 -c "
import sys,re,html
h=sys.stdin.read()
s=h[h.index('<div id=\"root\">'):h.index('<!-- ElevenLabs')]
print('words', len(html.unescape(re.sub(r'<[^>]+>',' ',s)).split()))
print('hidden', bool(re.search(r'clip\\s*:\\s*rect', s, re.I)))
"On a long guide you want thousands of words, not two sentences, and zero clip styles in #root. Then open the same URL with JavaScript disabled. You should still be able to read the article. After deploy, inspect the live URL in Google Search Console: raw HTML and rendered HTML should both contain the body.
What this does not fix
- Thin writing: a 600-word outline is still thin after prerender. The shell only reveals what you wrote.
- Duplicate intent: three beginner SEO posts still compete. Information gain is a content problem. See the information-gain framework.
- AdSense policy besides content: navigation, privacy, ownership, and traffic quality are separate. This article is the empty-HTML class of rejection.
If you are building a marketing site on a React edge stack and want the same crawl path, the websites work is where we start: indexable HTML, then design. Contact is /contact.
Frequently asked questions
Does Google execute JavaScript?
Googlebot can render JavaScript, with delay and limits. AdSense review, many social unfurlers, Bing URL Inspection without render, and a large share of AI fetchers do not. If the first HTML is empty, a reviewer can mark the site thin even when users see a full article after the bundle boots.
Is offscreen text for crawlers allowed?
No. Hidden text (clip, 1×1 boxes, display none, crawler-only landmarks) is a spam-policy issue. The body a bot reads must be the body a person can read without running your app.
Why did titles end with a backslash?
A build script parsed JavaScript source with a naive quote regex. A Beginner\'s Guide stopped at the escaped quote, so the live title was “A Beginner\”. Load the data module. Do not regex the file.
Should every unknown URL return 200 with the app shell?
No. That is a soft 404: infinite thin pages with HTTP 200. After every real route has a prerendered file, unknown paths should return 404 with a real document.
Do Markdown copies of articles get indexed?
Not on this site. HTML is canonical. /blog/{slug}.md is an export for people and models, sent with X-Robots-Tag: noindex, follow. We do not Disallow those files in robots.txt.