21 pages stuck in “Crawled – currently not indexed”. Here’s what actually fixed it

If Search Console shows a column called “Crawled – currently not indexed” with a double-digit number next to it, you have probably already read the standard advice: add content, add internal links, wait.

I did that twice. The number did not move.

The problem was not content quality. It was in the first HTML payload.

The short version

A 126-page content site had 21 URLs stuck in that report. Diagnosis found three distinct causes:

1. Three core pages were client-side rendered — the first HTML payload was a heading and the word Loading…, 46 characters in total
2. Two pages had 613 characters of body copy, far too thin to rank
3. Six pages duplicated each other — the same topic written three to five times under different URLs

All 21 left the report after the fixes below. Everything here is reproducible.

---

Step 1: Find out what Google actually receives

This is the step most people skip.

Fetch the raw HTML (no JavaScript execution) and count the body text:

curl -s https://your-site.com/page \
  | python3 -c "
import sys, re, html
raw = sys.stdin.read()
m = re.search(r'<main[^>]*>(.*?)</main>', raw, flags=re.S|re.I)
seg = m.group(1) if m else raw
t = re.sub(r'<(script|style)[^>]*>.*?</\1>', ' ', seg, flags=re.S|re.I)
t = re.sub(r'<[^>]+>', ' ', t)
t = re.sub(r'\s+', ' ', html.unescape(t)).strip()
print('body chars:', len(t))
print('first 200:', t[:200])
"

Output on my site:

body chars: 46
first 200: Free Oracle Reading — Ask Anonymously Loading…

That is the answer. Googlebot’s first request returned an H1 and a loading state. There was nothing to index.

Why “just wait for rendering” does not work

Google can execute JavaScript, but:

• Rendering is queued, not immediate. A new page can wait days or weeks.
• Rendering has a budget. The number of pages Google will render for a given site is limited — especially for sites without much authority.
• If rendering fails, the page is treated as empty.

So “the content lives in JavaScript” is not a speed problem. It is a “may never be indexed correctly” problem.

---

Step 2: Move content to the server without rewriting the page

The affected pages were interactive (forms, results display). The business logic had to stay client-side.

The fix was to add a server-rendered content block in the layout, not to rewrite the page:

// app/oracle/layout.tsx — server component (no "use client")
export default function Layout({ children }: { children: React.ReactNode }) {
  return (
    <>
      <h1>Free Oracle Reading — Ask Anonymously</h1>
      {children}          {/* client interaction, untouched */}

      {/* server-rendered: lands in the first HTML payload */}
      <SeoContentBlock
        title="How an Online Oracle Reading Works"
        blocks={[/* what it is, how it works, limits */]}
        faqs={[/* 8 questions */]}
      />
    </>
  );
}

Why this works:

• layout is a server component, and it is outside the child page’s Suspense boundary — its content is guaranteed to be in the first payload
• Zero changes to the client page, so zero business-logic risk
• FAQ content goes inside <details>, so it stays quiet visually but is fully present in the HTML

Same page after the change:

body chars: 3677

46 → 3,677. That is not an optimisation. It is going from “no content” to “content”.

Add structured data while you are there

The same block emits FAQPage JSON-LD:

const jsonLd = faqs?.length ? {
  "@context": "https://schema.org",
  "@type": "FAQPage",
  mainEntity: faqs.map(f => ({
    "@type": "Question",
    name: f.q,
    acceptedAnswer: { "@type": "Answer", text: f.a },
  })),
} : null;

One rule: the markup must match content that is actually visible on the page. Marking up FAQs that do not exist is a structured-data violation and gets manual actions.

---

Step 3: Kill the duplicate topic clusters

This one is usually underestimated. It was the single biggest contributor in my case.

The site had:

Topic	URLs	Example paths
Full moon ritual	5	/full-moon-release-ritual, /rituals/full-moon, /blog/full-moon-ritual-guide
Moon water	2	/blog/moon-water-ritual, /blog/moon-water-how-to-make-and-use
Career reading	2	/career-reading, /career-oracle-reading

Google’s view is simple: you wrote the same thing five times, it will index one and leave the rest in the not-indexed bucket.

Two different treatments — do not confuse them:

A. Pure duplicates → 308 permanent redirect

// next.config.ts
async redirects() {
  return [
    { source: "/blog/moon-water-ritual",
      destination: "/blog/moon-water-how-to-make-and-use",
      permanent: true },
    { source: "/career-reading",
      destination: "/career-oracle-reading",
      permanent: true },
  ];
}

Then remove the redirected URLs from the sitemap — otherwise you are still actively sending Google to pages that only redirect.

B. Different function → keep both, but differentiate the copy

/new-moon-ritual (explainer) and /rituals/new-moon (the actual tool) are not duplicates, but their copy was nearly identical.

Do not merge. Make each answer a different question:

• Explainer: what it is, who it is for, what you get
• Tool page: how to do it tonight, when, what to prepare, common mistakes

After that change, the tool page went from 613 to 4,100+ characters with no overlap.

---

Step 4: Stop wasting index budget on pages you do not want indexed

Three quick wins:

1. noindex on private pages. A personal data page has no value to searchers but consumes an index slot. robots: { index: false, follow: true }.
2. Legal pages out of the sitemap. /disclaimer, /refund-policy do not need to be actively submitted.
3. Sitemap diet. 126 → 121 entries, only what you actually want crawled.

---

Result

Metric	Before	After
Core page body text	46 chars	3,677 chars
Thin page body text	613 chars	4,100+ chars
Duplicate clusters	3 (9 URLs)	3 URLs
Sitemap entries	126	121
“Crawled – not indexed”	21	0

Timeline: 2–4 weeks for Google to recrawl. I also used the “Request indexing” button in URL Inspection on the most important URLs (limited daily quota).

---

Checklist you can run today

[ ] curl the raw HTML, count <main> text
    → under 1,000 characters is a problem
[ ] Check whether the page is "use client" with all content in JS
[ ] Pull the exact URL list from the GSC report
[ ] Group those URLs by topic and look for duplicates
[ ] Pure duplicates → 308 merge, remove from sitemap
[ ] Different function → differentiate copy to 2,000+ chars each
[ ] Private/legal pages → noindex or out of sitemap
[ ] Add FAQPage structured data (matching visible content)

---

One caveat

This fixes “Google cannot read your page”. It does not fix “nobody wants your page”.

My own site is the evidence: those 21 pages all got indexed after the fixes, and organic traffic did not meaningfully increase. Fixing indexing patches a leaking bucket. Whether the bucket has water in it is a separate problem.

Patch the bucket first. Then talk about water.