Practical guide
Noindex check: should this page appear in Google?
Noindex tells a search engine not to include a page in its index after the directive is processed. First decide whether that exclusion is intentional. Private reports and test pages should often remain excluded; a public product page accidentally marked noindex needs investigation.
Qlinnt · Reviewed
Two places to look for noindex

- Inside the HTML head
<meta name="robots" content="noindex">Inspect the HTML response and rendered head. In Next.js, also check inherited metadata from layouts and page-level robots settings.
- In the HTTP response headers
X-Robots-Tag: noindexInspect the response headers. The directive may come from the server, CDN or hosting configuration and can also apply to non-HTML files.
Check both locations on the same deployed URL. Removing one noindex directive is not enough if another remains. A missing directive does not guarantee indexing.
Inspect headers and the HTML response
Read the final response after redirects. X-Robots-Tag can exclude a page even when the HTML has no meta robots tag. Inspect the rendered DOM too if scripts modify metadata.
Use browser developer tools to view the response and page source. A public checker is not the same as Google’s URL Inspection and cannot prove what Google has indexed.
curl -sS -L -D headers.txt https://example.com/page -o page.htmlIdentify the exclusion mechanism
Inspect the relevant headers and metadata. If multiple directives conflict, adding index elsewhere does not reliably cancel an existing noindex.
| What you observe | What to check | Next step |
|---|---|---|
| Meta robots: noindex | Look for <meta name="robots" content="noindex"> and crawler-specific tags. | Remove an accidental exclusion from the public page’s metadata source. |
| X-Robots-Tag: noindex | Inspect the origin, CDN or route header configuration. | Change the matching rule only for pages that should be searchable. |
| Blocked by robots.txt | The crawler may not be able to read a noindex directive. | Allow access to the directive when exclusion from search is intended. Robots rules alone are not a reliable removal method. |
| No exclusion found | Check URL Inspection, selected canonical and indexing reasons. | Investigate the specific reason. Absence of noindex does not guarantee indexing. |
Fix an accidental exclusion in Next.js
Check page metadata, inherited layouts and next.config headers. A public page can set index and follow, but a remaining restrictive HTTP header must also be corrected. Deploy the change and inspect the response again.
export const metadata = {
robots: { index: true, follow: true },
};Keep deliberate exclusions
A shared scan report may be reachable by anyone holding its link while still excluded from search. Noindex is not a password and cannot stop a person or a crawler that ignores the directive.
Keep deliberate exclusions on reports, recovery links and test pages. For sensitive material, use authorization controls as well. Do not remove every noindex just to clear an audit warning.
Verify the same page after deployment
Check the same canonical public URL and its final headers, then use Search Console to request inspection where appropriate. A fresh public scan can verify the technical signal it measured; it cannot promise when Google processes a change.
- The intended public URL has no accidental noindex in HTML or headers.
- Redirects lead to the correct canonical page.
- The crawler can access the response it needs to read.
- Private and test pages retain their intentional exclusions.
Sources
Scan your public website
Start with a free scan. Verify your email to read every finding explained. Full evidence and tool-adapted prompts require full access; new purchases are currently paused.
Scan for free