Free Indexability Matrix Builder

Build a complete indexability matrix for any page. This tool checks all five critical indexing signals simultaneously — robots.txt, meta robots, X-Robots-Tag, canonical tag, and HTTP status code — and gives you a clear verdict on whether search engines can and will index the page.

Why You Need an Indexability Matrix

A page's indexability is controlled by multiple signals spread across different locations: the server configuration (robots.txt, HTTP headers), the HTML source (meta tags, canonical), and the response itself (status codes). A single misconfigured signal can prevent your page from appearing in search results. Checking these signals individually is time-consuming and error-prone — you might fix a noindex tag only to discover the page is also blocked by robots.txt.

An indexability matrix gives you the complete picture in one view. Instead of jumping between tools, you see every signal, its status, and the raw value — making it easy to identify conflicts and fix issues quickly.

How to Use This Tool

1. Enter a URL — paste the full URL of the page you want to analyze, including the path (e.g., https://example.com/blog/my-post)
2. Click "Build Matrix" — the tool fetches the page, its headers, and the site's robots.txt simultaneously
3. Review the matrix — each signal shows a clear Pass, Warning, or Fail status with the raw value and an explanation
4. Check the verdict — the overall indexability verdict tells you whether the page can be indexed and highlights any blockers

Understanding the Results

• Pass (green): The signal allows indexing. No action needed.
• Warning (yellow): The signal does not directly block indexing but may affect how the page appears in search results. Examples include nofollow directives, non-self-referencing canonicals, or redirects.
• Fail (red): The signal actively prevents indexing. This must be fixed for the page to appear in search results.

The overall verdict is "NOT INDEXABLE" if any signal has a Fail status that blocks indexing. Even one failed signal (noindex in meta, robots.txt disallow, or a 4xx/5xx status) is enough to prevent the page from appearing in search results.

Common Indexability Problems

• Robots.txt blocking + noindex tag: If robots.txt blocks crawling, search engines cannot see a noindex tag. The page may still appear in results based on external links. Remove the robots.txt block if you want the noindex directive to work.
• Canonical pointing elsewhere: A canonical to a different URL signals that the current page is a duplicate. Search engines will typically index the canonical URL instead.
• Conflicting meta robots and X-Robots-Tag: When both are present, search engines follow the most restrictive directive. If meta says "index" but the X-Robots-Tag says "noindex", the page will not be indexed.
• Staging noindex leaked to production: CMS platforms often add noindex during development. If this setting persists after launch, important pages silently disappear from search results.
• Soft 404s: A page returning a 200 status but displaying a "not found" message may be treated as a soft 404 by Google, even though the status code appears fine in the matrix.

Frequently Asked Questions

What is an indexability matrix?

An indexability matrix is a comprehensive view of all signals that determine whether a search engine can and will index a specific page. It includes robots.txt rules, meta robots tags, X-Robots-Tag HTTP headers, canonical tags, and HTTP status codes — all checked simultaneously to give you a complete picture of a page's indexing status.

What is the difference between robots.txt and meta robots?

Robots.txt controls crawling — it tells search engine bots whether they are allowed to access a URL at all. Meta robots tags control indexing — they tell search engines whether to include the page in search results after crawling it. A page can be crawlable (allowed in robots.txt) but not indexable (has a noindex meta tag), or it can be blocked from crawling but still appear in search results based on external links.

What is the X-Robots-Tag header?

The X-Robots-Tag is an HTTP response header that provides the same directives as the meta robots tag (noindex, nofollow, noarchive, etc.) but is sent at the server level. It is especially useful for non-HTML resources like PDFs, images, and video files that cannot contain HTML meta tags. Both Google and Bing respect X-Robots-Tag directives.

How does the canonical tag affect indexability?

A canonical tag (<link rel="canonical">) tells search engines which URL is the preferred version of a page. If the canonical points to a different URL, search engines will typically index the canonical URL instead of the current page. A self-referencing canonical confirms the current page as the preferred version. Missing canonical tags can lead to duplicate content issues when the same content is accessible at multiple URLs.

Can a page blocked by robots.txt still appear in search results?

Yes. If a page is blocked by robots.txt, search engines cannot crawl it to see any noindex directive. However, the page can still appear in search results if other pages link to it — Google may show a minimal listing with just the URL and no description. To truly prevent indexing, use a noindex meta tag and ensure the page is not blocked by robots.txt so that crawlers can see the directive.