What does a noindex checker look for?

It fetches the live URL and reads HTML meta robots / googlebot tags plus the X-Robots-Tag response header to see whether indexing is discouraged for search engines.

Can a page be noindex but still crawlable?

Yes. noindex asks engines not to show the URL in results; robots.txt may still allow the fetch. Use both the Noindex Checker and the robots.txt checker when diagnosing why a page is missing from Google.

Is “Crawled – currently not indexed” the same as noindex?

No. That Search Console status means Google fetched the URL and chose not to index it. A noindex is an explicit publisher signal. If this checker shows the page is allowed to be indexed, read the crawled-currently-not-indexed guide instead of hunting a missing tag.

Why check X-Robots-Tag as well as meta robots?

Some CDNs, reverse proxies, and PDF/non-HTML responses set indexing rules only in HTTP headers. Checking both avoids false “indexable” conclusions from HTML alone.

Is this free? Do I need an account?

Yes — free, no registration on TheSeoSoul. We only fetch the URL you submit; we do not submit URLs to Google.

Free tools/On-page

Free Noindex & X-Robots-Tag Checker

Check whether a live URL tells search engines not to index it. We read meta robots, googlebot meta, and the X-Robots-Tag header — free, no signup.

How it works

01
Paste any public URL
We request the page (following sensible redirects) and capture HTML plus response headers.
02
Extract robots directives
Meta robots, googlebot-specific meta, and X-Robots-Tag are parsed for noindex / none and related tokens.
03
See a clear indexability summary
Use the result to confirm staging, thank-you, or thin templates stay out of the index — or that production pages are not accidentally blocked.

What this noindex checker inspects

HTML <meta name="robots" content="noindex"> declarations
Engine-specific directives such as <meta name="googlebot" content="noindex">
HTTP response headers for X-Robots-Tag: noindex, nofollow, or none
The critical robots.txt trap: ensure noindex pages are NOT blocked in robots.txt so Google can fetch the directive

Scenarios

Real-World Noindex Scenarios & Debugging

Accidental noindex tags cause sudden, catastrophic de-indexing in Google Search. Here are the most frequent causes we observe:

Staging Leaks

Deploying Staging Metadata to Production

The Problem: CI/CD pipelines deploying environment variables or header configs intended for dev.example.com into production, immediately de-indexing core pages.

The Fix: Audit production headers for X-Robots-Tag and enforce environment-aware robots metadata in your build step.

CMS Setting

WordPress 'Discourage Search Engines' Toggled

The Problem: The 'Discourage search engines from indexing this site' option remains checked after launching a new WordPress site, outputting noindex sitewide.

The Fix: Uncheck the setting in Settings > Reading, and inspect with this tool to confirm all meta robots tags are clear.

Static / PDF Assets

Hidden X-Robots-Tag on CDN Assets

The Problem: Cloudflare Transform Rules or Nginx configs injecting X-Robots-Tag on entire MIME types or subdirectories inadvertently.

The Fix: Check raw HTTP response headers on PDF documentation, public landing pages, and APIs to isolate header-level blocks.

How to Implement or Remove Noindex Correctly

To prevent a page from appearing in search results while still allowing search engines to follow its links, use the standard HTML meta robots tag:

Standard HTML Meta Robots (Noindex, Follow)html

<!-- Tells Google not to index this page, but still follow outbound links -->
<head>
  <meta name="robots" content="noindex, follow" />
</head>

Next.js App Router (metadata.robots)typescript

In Next.js, declare indexing rules per page or layout:

import type { Metadata } from 'next';

export const metadata: Metadata = {
  robots: {
    index: false,
    follow: true,
  },
};

Nginx HTTP Header (X-Robots-Tag)nginx

For non-HTML files (e.g., internal PDF reports, staging subpaths):

location /staging/ {
    add_header X-Robots-Tag "noindex, nofollow" always;
}

Frequently asked questions

What does a noindex checker look for?: It fetches the live URL and reads HTML meta robots / googlebot tags plus the X-Robots-Tag response header to see whether indexing is discouraged for search engines.
Can a page be noindex but still crawlable?: Yes. noindex asks engines not to show the URL in results; robots.txt may still allow the fetch. Use both the Noindex Checker and the robots.txt checker when diagnosing why a page is missing from Google.
Is “Crawled – currently not indexed” the same as noindex?: No. That Search Console status means Google fetched the URL and chose not to index it. A noindex is an explicit publisher signal. If this checker shows the page is allowed to be indexed, read the crawled-currently-not-indexed guide instead of hunting a missing tag.
Why check X-Robots-Tag as well as meta robots?: Some CDNs, reverse proxies, and PDF/non-HTML responses set indexing rules only in HTTP headers. Checking both avoids false “indexable” conclusions from HTML alone.
Is this free? Do I need an account?: Yes — free, no registration on TheSeoSoul. We only fetch the URL you submit; we do not submit URLs to Google.

Keep going

What to check next

If the page never gets fetched, or signals disagree, check these next.

Need the whole picture? Run a full audit · browse every tool.