Indexability Checker

How to Use This Tool

1. 1

   Enter the URL

   Use the exact address you care about, including http or https if you are testing a redirect. Both the page and the site's robots.txt are fetched server-side.

2. 2

   Read the verdict

   Indexable, blocked, or conflicted. Conflicted is the one worth slowing down for — it means the signals disagree and the outcome is not what any of them intended alone.

3. 3

   Work down the signals

   Each one shows what was actually found and what it means. The status code comes first because nothing below it matters if the page does not return 200.

4. 4

   Follow the numbered actions

   They are ordered so that fixing them in sequence works. Resolving a conflict usually has to happen before the individual fixes will do anything.

5. 5

   Confirm in Search Console

   This tool reports what the page tells crawlers. Only the URL Inspection tool reports what Google did with it.

Crawlable Is Not the Same as Indexable

This is the distinction the whole subject turns on, and getting it backwards causes more wasted work than any other technical SEO misunderstanding.

Crawling is a search engine fetching the page. Indexing is a search engine storing it and considering it for results. They are separate steps, controlled by separate mechanisms, and one does not imply the other.

robots.txt controls crawling. It says which URLs a crawler may request, and nothing else. It does not say whether a URL may be indexed.

The robots meta tag and the X-Robots-Tag header control indexing. They live in the page and in the response, which means a crawler has to fetch the page to read them.

Once those two sentences sit next to each other, the most common mistake in the field becomes obvious.

The noindex Nobody Can Read

Someone wants a page out of Google. They add a noindex meta tag. Nothing happens after a few weeks, so they also block the URL in robots.txt to be thorough. The page stays in the index, sometimes for years.

Blocking crawling stops Googlebot fetching the page. If it cannot fetch the page, it cannot read the noindex. The instruction to remove the page is sitting behind a door the crawler is not allowed to open.

Worse, blocking does not remove anything on its own. A URL that other pages link to can be indexed on the strength of those links alone, with no content behind it — which is why results occasionally show a bare URL and a line explaining that no description is available.

The fix is counterintuitive and it is the right one: allow the URL in robots.txt, keep the noindex, and wait for the page to be recrawled and dropped. Only once it has dropped out is blocking it again a reasonable thing to do, and by then it is rarely worth doing.

This validator names that conflict specifically rather than reporting two separate findings, because two separate findings is exactly how it stays unnoticed. Our Robots.txt Tester is the tool for the robots.txt rules on their own.

The Signals, and What Each One Decides

• HTTP status — a page has to return 200 to be indexed. A 404 or 410 removes it, a 5xx suspends it, and a 3xx moves the question to whatever it redirects to.
• robots.txt — controls fetching only. Blocked means Googlebot never sees the page, including anything the page says about itself.
• The robots meta tag — noindex here is the standard way to keep a page out of the index. It only works if the page can be crawled.
• X-Robots-Tag — the same directives delivered in the response header. Invisible in the page source, which makes an accidental noindex here genuinely difficult to find, and the only way to apply one to a PDF or an image.
• Canonical — not a block, but a nomination. A canonical pointing at another URL asks search engines to index that one instead of this one.
• Redirects — the destination is what gets indexed. A 302 tells search engines the move is temporary and to keep the old URL, which is often not what was meant.

Canonicals Are the Quiet One

A canonical is a hint rather than a directive, and Google is free to disagree with it. In practice it is usually respected, which makes a wrong one an effective way to remove a page from search without ever intending to.

The failure looks like this: a template applies one canonical to every page in a section, all of them pointing at the section's index page. Every individual page is crawlable, returns 200, has no noindex anywhere, and is politely asking not to be indexed. Nothing in a crawl report necessarily flags it.

Two other canonical faults worth knowing. More than one canonical on a page, usually from a theme and a plugin both adding one, is generally resolved by search engines ignoring all of them. And a canonical declared in the Link header can disagree with the one in the HTML, which is hard to spot because only one of the two is visible in the page source — this checker reads both.

A self-referencing canonical is the right answer for any page you want indexed in its own right. Our SEO Analyzer checks the canonical alongside the rest of the on-page setup.

Indexable Does Not Mean Indexed

This tool answers a technical question: does anything on this page prevent it being indexed? A clean result means nothing is stopping Google. It does not mean Google has indexed the page, and it does not mean it will.

Google indexes a subset of what it crawls. Pages get left out for reasons that have nothing to do with directives — thin content, near-duplicates of something already indexed, a page nothing links to, or simply a site where crawl demand is low. Search Console reports these as Discovered or Crawled but not indexed, and neither is a technical fault you can fix with a header.

So a clean verdict here narrows the problem rather than solving it. If a page is technically indexable and still absent, the answer is in the content or the internal linking, not in the directives. Our Internal Link Analyzer is a reasonable next stop, because a page nothing links to is a page Google has little reason to keep.

The only authoritative source for what Google actually did with a specific URL is the URL Inspection tool in Search Console. No third-party tool can report that, including this one, and any that claims to is inferring.

When a Page Should Not Be Indexed

Plenty of pages should be excluded, and knowing which mechanism to use for each is most of the skill.

• Thin, duplicated or internal pages you want out of the index — noindex, and let it be crawled so the directive is read.
• Faceted navigation and filtered URLs that generate endless combinations — robots.txt, because the goal is to stop the crawling itself rather than to remove anything.
• Duplicate versions of a page that should consolidate into one — canonical, not noindex. A noindex throws the page away; a canonical merges its signals into the version you kept.
• Staging and development sites — HTTP authentication. Directives are the wrong tool here, and a staging site relying on a noindex is one deployment mistake away from being public.
• Something that must come out of results now — the Removals tool in Search Console for the temporary suppression, plus a noindex or a 404 for the permanent fix. The removal alone expires.

Where This Fits

Indexability is the first question, not the last one. A page that cannot be indexed makes every other optimisation moot, which is why this is worth checking before spending time on anything else.

Once it comes back clean, the rest of the technical picture is worth a look. Our HTTP Header Checker shows the full response including caching and security headers, the XML Sitemap Validator confirms the URLs you are submitting actually resolve, and the Redirect Chain Checker traces a URL through every hop when the chain is the thing you are chasing.

Frequently Asked Questions

Related Tools