How the Meta Robots Tag Works
The meta robots tag sits in a page's <head> and tells search engines what to do with that specific page — whether to index it, whether to follow its links, and how it may be shown in results. It's a per-page control, unlike robots.txt which works at the path level and can't remove an already-indexed page.
Pick your directives with the checkboxes and copy the exact tag to paste into your template or CMS.
How to Use This Tool
Choose indexing and link following
index or noindex decides whether the page can appear in results; follow or nofollow decides whether links on it are followed.
Add any extra directives
noarchive, nosnippet, noimageindex, notranslate, and the preview limits max-snippet, max-image-preview and max-video-preview.
Copy the tag
Use the general robots tag, or the googlebot version if the rule should apply to Google only.
Paste it into the page's <head>
Then run the Indexability Checker on the live URL to confirm the tag is there and not contradicted elsewhere.
index/noindex and follow/nofollow
index and follow are the defaults — a page with no meta robots tag at all behaves as if it said index, follow. You only need the tag when you want to change that default.
noindex removes a page from search results at the next crawl. nofollow tells crawlers not to pass ranking signals through the links on that page, though it does not stop them from being crawled if they're linked elsewhere.
The Less Common Directives
- noarchive — stops Google from showing a cached version of the page.
- nosnippet — no text or video snippet is shown in the search result; also removes the cached link.
- noimageindex — images on this page won't be indexed by Google Images.
- notranslate — Google won't offer to translate the page in results.
- max-snippet:[number] — caps the snippet length shown in characters. Use 0 for no snippet, -1 for no limit.
- max-image-preview:[none|standard|large] — controls the largest image preview size shown.
- max-video-preview:[number] — caps the video preview length in seconds. -1 means no limit.
meta robots vs the X-Robots-Tag Header
The X-Robots-Tag HTTP header does the exact same job as this meta tag but is set at the server level, which means it works on non-HTML files like PDFs where there's no <head> to put a meta tag in. If a page has both, the directives are combined — the most restrictive wins for things like noindex.
Check what a page is actually sending with the HTTP Header Checker before assuming the meta tag is the only thing controlling indexing.
Common Mistakes
- Setting noindex on a page you actually want ranked — easy to leave in place after a staging-to-production copy.
- Using noindex and a canonical tag pointing elsewhere on the same page — check both point the same direction with the Canonical Tag Checker.
- Blocking a page in robots.txt while also adding noindex to it — if robots.txt blocks crawling, Google may never see the noindex tag at all, so the page can still appear in results with no snippet.
What Happens When There Is No Tag
With no robots meta tag and no X-Robots-Tag header, search engines treat the page as index, follow. That means you only need the tag when you want something other than the default — most pages on most sites need no tag at all.
Writing <meta name="robots" content="index, follow"> is harmless but does nothing. It is only worth including if a template elsewhere might otherwise add a noindex.
Targeting a Specific Search Engine
name="robots" applies to every crawler that honours the tag. Replace robots with a crawler's name — googlebot, bingbot — to give that crawler different instructions. If both a general and a specific tag are present, search engines combine them and apply the more restrictive rule.
Crawler-specific tags are rarely needed. The most common legitimate use is googlebot-news, to keep a page out of Google News while leaving it in normal search.
Adding the Tag in Common Platforms
- WordPress — SEO plugins such as Yoast and Rank Math add a per-page noindex option in the editor. The Search engine visibility setting under Settings → Reading adds noindex to the whole site, which is a common cause of sites vanishing after launch.
- Shopify — edit theme.liquid to add a conditional tag, or use an SEO app with a per-page setting.
- Next.js — set the robots field in a page's metadata export.
- Plain HTML — paste the tag inside <head> on each page that needs it.
- Files that are not HTML, such as PDFs — use the X-Robots-Tag HTTP header instead, since there is no <head> to put a tag in.
Frequently Asked Questions
Do I need this tag on every page?
No. Only add it where you want to override the default (index, follow) — typically on thank-you pages, internal search results, staging content, or duplicate parameter URLs you want kept out of the index.
Does noindex remove a page immediately?
No — it takes effect the next time the page is crawled, which can range from hours to weeks depending on how often Google crawls that URL. For urgent removals, use Search Console's Removals tool alongside the tag.
Can I combine noindex with nofollow?
Yes — content="noindex, nofollow" is common on thin or duplicate pages you want out of the index and don't want to pass link signals from either.
Why does Google still show my noindex page in results?
The most common cause is robots.txt blocking the page from being crawled at all, so Google never sees the noindex directive. Check with the Robots.txt Tester and the Indexability Checker.
What is the default if a page has no robots meta tag?
index, follow. Search engines may index the page and follow its links unless something tells them otherwise, so you only need the tag to change that behaviour.
Does this generator support unavailable_after?
Not yet. unavailable_after tells Google to stop showing a page after a date; you can add it by hand as unavailable_after: followed by a date in a widely used format such as ISO 8601.
Is noindex the same as blocking a page in robots.txt?
No. robots.txt stops crawling; noindex stops indexing. A page blocked in robots.txt can still appear in results from links elsewhere, and Google cannot see a noindex tag on a page it is not allowed to crawl.
Related Tools
Indexability Checker
See the combined verdict from robots.txt, meta robots, and X-Robots-Tag together.
Robots.txt Generator
Set path-level crawl rules alongside your page-level meta robots tags.
Canonical Tag Generator
Generate the matching canonical tag for this page.
HTTP Header Checker
Check whether a page also sends an X-Robots-Tag header.