Some pages on your site should never show up in Google. Think thank-you pages, login screens or a staging site you forgot to lock down. Leave them indexed and they clutter your results, split your authority and sometimes leak private URLs to the public. A noindex tag fixes this. It tells search engines to keep a specific page out of their results while still letting people reach it directly. This guide covers what noindex does, when to use it, how to add it across every major platform, and the mistakes that quietly break it.
What Is a Noindex Tag?
A noindex tag is a rule that tells search engines not to include a page in their search results. You add it either as a meta robots tag in the page’s <head> or as an X-Robots-Tag in the HTTP response header. The page can still be crawled and visited directly, but it will not appear when someone searches. Noindex works page by page, so you control exactly which URLs stay hidden.
The most common form looks like this:
<meta name="robots" content="noindex">
Place that inside the <head> section and any search engine that supports the rule will drop the page. Google confirms this in its block indexing with noindex documentation.
How Does a Noindex Tag Work?
The order of events matters. Googlebot has to crawl the page first, read the noindex rule, and only then drop the URL from the index. So the page must stay open to crawlers for the rule to work. If a crawler cannot reach the page, it never sees the tag.
Once Googlebot extracts the rule, Google removes the page from search results even if other sites link to it. Noindex is a directive Google honours, not a hint it can overrule for ranking.
Does Google Always Respect Noindex?
Google respects a noindex rule it can actually see. The catch is in that last part. If the page is blocked in your robots.txt file, or the crawler cannot access it for some other reason, Google never reads the rule, and the page can still appear in results, for example if another site links to it. Google spells this out in its noindex implementation guide.
Other search engines usually honour noindex too, but not always in the same way. Google notes that some engines may interpret the rule differently, so a page you have hidden from Google might still surface elsewhere.
Why Blocked Pages Still Appear in Search
This trips up a lot of people. Blocking a URL in robots.txt stops crawling, not indexing. If that URL is linked from somewhere, Google can still list it, usually as a bare link with no description. Because Google never crawled the page, it never saw any noindex rule you added. So the page you tried to hide twice ends up visible anyway. If you want a page out of results, let it be crawled and use noindex, not a robots.txt block. Our guide to the robots.txt file covers where that line sits.
Noindex vs. Nofollow vs. Disallow vs. Canonical
These four directives get mixed up constantly, yet each solves a different problem. Here is how they compare at a glance.
Directive | What it controls | Where it lives | Page still crawled? | Stays in index? |
|---|---|---|---|---|
noindex | Whether the page appears in search results | Meta robots tag or X-Robots-Tag | Yes | No |
nofollow | Whether crawlers follow links / pass equity | Meta tag or link rel attribute | Yes | Yes |
disallow | Whether crawlers may access the URL | robots.txt file | No | Maybe (URL-only) |
canonical | Which duplicate is the indexable version | link rel in the head | Yes | Only the chosen URL |
Noindex vs. Nofollow
Noindex controls whether the page itself gets indexed. Nofollow controls whether search engines follow the links on a page and pass equity through them. They are separate switches. You can noindex a page and still let its links be followed, which is why noindex, follow exists.
Noindex vs. Disallow (robots.txt)
Disallow blocks crawling. Noindex blocks indexing. Disallow alone does not reliably keep a page out of results, as covered above. Use disallow to save crawl budget on sections you never want fetched, and noindex when the goal is specifically to keep a page out of search. Our crawl budget guide explains when crawl control is worth the effort.
Noindex vs. Canonical
A canonical tag handles duplicates. It tells Google which version of near-identical pages should be the indexed one, and both stay crawlable. Noindex removes a page from search entirely. So reach for canonical when you have duplicate or near-duplicate content and want one version to rank, and reach for noindex when a page should not appear at all.
When to Use Each
A quick way to choose:
- Want a page gone from search but reachable by link? Use noindex.
- Want to stop passing link equity through a link? Use nofollow.
- Want to stop crawling of a whole section? Use disallow.
- Have duplicate pages and want one to rank? Use canonical.
The Noindex Meta Tag vs. the X-Robots-Tag HTTP Header
There are two ways to deliver a noindex rule, and Google treats them as equivalent. Pick whichever suits the content type.
The meta robots tag lives in the HTML <head>, applies to one page, and is usually added by whoever manages the site content. The X-Robots-Tag is sent as an HTTP response header, works on any resource including non-HTML files, and is usually set at the server level by a developer. Google documents both in its robots meta tag specifications.
When to Use the X-Robots-Tag Instead
Reach for the header when a meta tag will not fit. That covers non-HTML files such as PDFs, images and videos, which have no <head> to edit. It also helps when you want to apply a rule in bulk by file type, or when editing the HTML head is awkward. A basic PDF example for an Apache server:
<Files ~ "\.pdf$">
Header set X-Robots-Tag "noindex, nofollow"
</Files>
The same rule on NGINX:
location ~* \.pdf$ {
add_header X-Robots-Tag "noindex, nofollow";
}
Meta Robots Directive Values You Can Combine With Noindex
Noindex rarely travels alone. You can pair it with other robots rules, separated by commas, to control snippets, caching, images and more. Most competitor guides stop at follow and nofollow. Here is the fuller set Google supports, one by one.
Value | What it does | Supported by |
|---|---|---|
noindex, follow | Exclude the page, but follow its links | All major engines |
noindex, nofollow | Exclude the page and ignore its links | All major engines |
none | Shorthand for noindex, nofollow | All major engines |
noarchive | No cached copy shown | All major engines |
nosnippet | No text or video snippet (also acts as noarchive) | All major engines |
noimageindex | Do not index images on the page | Google only |
unavailable_after | Drop the page after a set date | Google only |
indexifembedded | Index only when embedded elsewhere | Google only |
Noindex, Follow
This excludes the page but lets crawlers follow its links, which is common on paginated and archive pages. One caveat: Google has said that over the long term it treats the links on a persistently noindexed page as nofollow anyway, so do not rely on it forever.
Noindex, Nofollow
This excludes the page and tells crawlers to ignore its links. You can write the same instruction as none.
None
A shorthand that combines noindex and nofollow in one value. It is less common because the shorthand is easy to misread, so many teams prefer spelling out both rules.
Noarchive
This stops search engines showing a cached version of the page. You can combine it with noindex when you want neither a listing nor a cached copy.
Nosnippet
This prevents any text or video snippet in results, and it also acts as noarchive. Worth knowing: Google has confirmed that pages carrying nosnippet will not appear as sources in AI Overviews, so it is a lever for staying in search but out of AI answer boxes, per this robots and meta reference.
Noimageindex
This asks Google not to index the images on a page. Only Google supports it, so treat it as a Google-specific control rather than a universal rule.
Unavailable_after
This is a timed noindex. Google drops the page from results after a date you set, which suits event pages, limited promotions or job posts with a known expiry. Specify the date in a widely used format such as RFC 850. Google introduced the wider set of these controls in its robots specifications.
Indexifembedded
This one is niche but useful for publishers. Paired with noindex, it lets Google index your content when it is embedded in another page through an iframe or similar tag, while keeping the source page itself out of results. Google explained the rule when it introduced indexifembedded in 2022, and only Google supports it.
When Should You Use Noindex?
Noindex earns its keep on pages that have a job to do for users but no business appearing in search. Here are the usual candidates.
Thank-You and Confirmation Pages
These only make sense to someone who has just completed a form or purchase. In search results they offer nothing, so noindex them.
Internal Search Results Pages
Pages generated by your on-site search tend to be thin and near-endless. Keeping them out of Google avoids low-value clutter and wasted crawling.
Login, Account and Admin Pages
These are for signed-in users, not searchers. Noindex keeps private or functional screens out of public results.
Staging, Test and Development Environments
A staging site indexed by mistake can compete with your live pages. Noindex the whole environment while you build, but remember to remove it at launch.
Thin, Duplicate or Low-Value Pages
Printable versions, parameter-based URLs and session pages rarely deserve their own listing. Noindex the ones that add nothing, though a canonical is often the better fix for true duplicates.
Tag, Archive and Date-Based Pages
Blogs and news sites spin up large numbers of tag and archive pages. Unless a given archive adds real value, noindex it to avoid duplication.
Paid Landing Pages and A/B Test Variants
Pages built for a specific ad campaign or a split test are not meant for organic search. Noindex keeps them from competing with your core pages.
Gated or Downloadable Resources
PDFs and other files behind a form can be excluded with the X-Robots-Tag header, since they have no HTML head to hold a meta tag.
When You Should Not Use Noindex
Noindex is a blunt instrument, so a few situations call for a different tool. Do not noindex pages that earn organic traffic or authority, since you would be handing away visibility. Do not use it to solve duplicate content when a canonical tag would keep the value and pick a winner. And never apply it site-wide by accident, which is easier to do than it sounds during a redesign or migration.
How to Add a Noindex Tag
You have two delivery methods and one whole-site option. Here is each.
Add Noindex via the HTML Meta Tag
Drop this into the <head> of the page, then save and deploy:
<meta name="robots" content="noindex">
To target only Google’s crawler, name it directly:
<meta name="googlebot" content="noindex">
Add Noindex via the X-Robots-Tag
Send the rule in the HTTP response header instead. A single header looks like this:
X-Robots-Tag: noindex
This is the route for PDFs, images and other non-HTML files, using the server rules shown earlier.
How to Noindex an Entire Site
To hide a whole site, apply an X-Robots-Tag across every response at the server, or use your platform’s built-in switch. On WordPress that switch sits under Settings > Reading, labelled “Discourage search engines from indexing this site.” It is the right tool for a site still in development, and the wrong thing to leave on after launch, which is how live sites vanish from Google overnight.
How to Add Noindex in Your CMS
Most platforms give you a toggle so you never touch code. The exact path differs, so here is each major one, verified against the platform’s own documentation.
WordPress
WordPress has a site-wide switch under Settings > Reading (“Discourage search engines”), but for a single page you want a plugin so you can control one URL at a time. The three common plugins all expose a per-post robots setting.
Rank Math
Edit the post, open the Rank Math box, click the Advanced tab, and under Robots Meta tick No Index. To set a whole post type or taxonomy by default, use Titles & Meta settings.
Yoast SEO
In the post editor, scroll to the Yoast box, open the Advanced tab, and set “Allow search engines to show this post in search results?” to No. Site-wide control lives under Search Appearance > Content Types.
All in One SEO
Open the post, find the AIOSEO settings, go to the Advanced tab and switch on No Index under Robots Meta.
Shopify
Shopify offers a built-in metafield. Under Settings > Metafields and metaobjects, add a definition with the namespace and key set exactly to seo.hidden, then set that field to 1 on any product, page or blog post. Shopify then adds a noindex, nofollow tag, drops the item from your sitemap and hides it from internal search, per its hide a page guide. For collections, edit theme.liquid with a conditional instead. Our Shopify SEO guide covers the wider setup.
Wix
In the Wix Editor, open the Pages panel, click the menu next to the page, choose the SEO tab, and turn off “Let search engines index this page.” Wix then inserts the noindex meta tag for you, as described in its page indexing article.
Squarespace
Open the Pages panel, click the settings icon for the page, go to the SEO tab, and switch on “Hide page from search results.” Squarespace explains the toggle and its limits in its hiding from search guide.
Webflow
Open Page Settings, scroll to the SEO section, and turn off the Sitemap indexing toggle. Webflow then adds a noindex meta tag and removes the page from your sitemap automatically, as its disable indexing guide explains.
Magento (Adobe Commerce)
On a catalogue or CMS page, set the Meta Robots field to NOINDEX, NOFOLLOW (or NOINDEX, FOLLOW) in the page’s SEO settings. Store-wide, the same directive can be applied through the design configuration for non-production modes.
How to Check Whether a Page Has a Noindex Tag
Before you trust that a rule is working, confirm it. There are four reliable ways.
View the Page Source
Open the page, press Ctrl+U to view source, and search for “noindex.” If the meta robots tag is there, you will see it near the top inside the head.
Check the HTTP Headers
A noindex delivered by X-Robots-Tag will not show in the HTML. Inspect the response headers using your browser’s network tab or a header-checking tool to catch it.
Use the URL Inspection Tool in Google Search Console
Paste the URL into the URL Inspection tool in Google Search Console. It tells you whether Google saw a noindex when it last crawled the page, which is the version that actually matters.
Use a Free Noindex Checker
Several free tools report a page’s index status in one click. They are handy for a quick check, though the source and header methods above give you the full picture.
How to Fix the “Excluded by Noindex Tag” Error in Search Console
This status in Google Search Console means Google crawled the page, found a noindex rule and left the page out of results. Sometimes that is exactly what you wanted. Sometimes it is a page you meant to rank. Here is how to sort it out.
Confirm Whether the Exclusion Is Intentional
First decide if the page should be hidden. Thank-you pages and staging URLs belong here. A key service page or blog post does not, so treat those as errors to fix.
Find and Remove the Rogue Noindex
Track down where the rule comes from. Common culprits are a CMS or plugin setting, a template, or an X-Robots-Tag added at the server or CDN. Remove the noindex at its source, then confirm the page source and headers are clean. Our SEO audit service can surface these across a whole site.
Request a Re-Crawl in Search Console
Once the rule is gone, use URL Inspection to request indexing, then use the Validate Fix option on the issue. Deindexing and reindexing both wait on a re-crawl, so give Google time to revisit.
The JavaScript Rendering Caveat
If you run a single-page app built with React, Vue or Angular, watch where your noindex tag lives. Googlebot reads the raw HTML first and makes its initial indexing call from that markup, before it renders JavaScript. A noindex injected only after the page hydrates can be missed or acted on late, so the page may get indexed when you did not want it to. Put the noindex rule in the server-rendered HTML, not in client-side script, so the first pass sees it.
Common Noindex Mistakes to Avoid
Most noindex problems come down to a handful of repeat offenders. Watch for these.
Blocking a Noindexed Page in Robots.txt
If you disallow the URL, Google cannot crawl it to see the noindex rule, so the page can still be indexed. Let it be crawled, or the rule never fires.
Leaving a Staging Noindex Live After Launch
A site-wide noindex added during development will keep your live site out of Google if you forget to remove it. Make its removal part of your launch checklist.
Removing a Page From the Sitemap Too Early
Keep a noindexed page in your sitemap until Google has actually dropped it, otherwise you slow down deindexing. Our XML sitemap guide covers what belongs there.
Using Noindex for Duplicate Content
For true duplicates, a canonical keeps the value and picks a winner. Noindex just removes the page, which is usually not what you want.
Applying Noindex Site-Wide by Mistake
Redesigns and migrations are the classic moment for an accidental global noindex. Check the whole site after any big change.
Expecting Instant Removal
Deindexing is not immediate. Google has to re-crawl the page to see the rule and act on it, so give it time rather than assuming the tag failed.
Frequently Asked Questions
Noindex is a rule that tells search engines not to include a page in their results. You add it as a meta robots tag in the page head or as an X-Robots-Tag header. The page stays reachable by direct link and can still be crawled, but it will not show up in search.
Yes, as long as Google can crawl the page and read the rule. If the page is blocked in robots.txt, Google never sees the noindex and the URL can still appear in results. Other search engines usually honour it too, but may interpret it slightly differently.
View the page source (Ctrl+U) and search for “noindex,” check the HTTP response headers for an X-Robots-Tag, or use the URL Inspection tool in Google Search Console to see whether Google detected a noindex on its last crawl.
Apply an X-Robots-Tag noindex across every server response, or use your platform’s built-in switch, such as WordPress’s “Discourage search engines” option under Settings > Reading. This suits sites in development, but remember to turn it off before launch.
Used well, noindex keeps your search results clean and your authority focused on the pages that earn it. The trick is matching the tool to the job: noindex to hide a page, canonical for duplicates, disallow for crawl control, and always let a page be crawled if you want its noindex rule to count. If you would like a second pair of eyes on which pages should stay out of Google, our SEO agency in Singapore can help you get the setup right. Found this useful? Share it with someone wrangling their index.
The Bottom Line
SEO takes three to six months to show real traction and six to 12 to drive meaningful growth, with new and competitive sites at the longer end. The clock starts when work is implemented, so the sooner you begin, the sooner that timeline runs.
The most useful thing you can do is set realistic expectations and start with a solid foundation. That means a technical audit, a clear roadmap and tracking the early signals that prove momentum before rankings arrive.
If you found this helpful and you are weighing up SEO for your own business, our SEO agency in Singapore can map a realistic timeline around where your site stands today.
