What Is Indexing?

Indexing is when a search engine stores a page it has found, so the page can be shown in search results later. For a health practice, a page that is not indexed is invisible to patients searching Google, and Google says it also cannot appear in its AI Overviews or AI Mode. Many indexing problems start with a setting on your own site, which makes them fixable.

How indexing works

Google describes search in three stages. First it finds pages, a step called crawling. Then it tries to understand what each page is about and stores it, which is indexing. Finally, when someone searches, it pulls matching pages from that stored collection and shows them, which is serving. Think of the index as a library shelf. A page that does not reach the shelf cannot be picked up by any reader.

Each stage can fail. Google states that it does not promise to crawl, index or serve your page, and that not every page it processes will be indexed. So indexing is a requirement, and it is not a reward you automatically get.

For AI features, the requirement is stated plainly. Google's documentation says a page must be indexed and eligible to be shown in Google Search with a snippet to appear in AI Overviews or AI Mode. It says there are no additional requirements, and no special markup or AI text file is needed. That makes indexing the first thing to confirm before anything else. See AI Overviews for how those summaries are built.

The usual reasons an important page is missing from the index are these.

• A noindex setting. A noindex instruction, placed in the page's code or sent by the server, tells Google to drop the page entirely from search results. It is meant for pages you want private. It becomes a problem when it lands on a service page.
• A robots.txt block. Google says that if a page is blocked in robots.txt, it cannot see a noindex instruction on that page, and the page can still show up if other pages link to it. Robots.txt controls crawling and it is a different tool from noindex.
• Login walls and pop-ups. If a page needs a login, or content only appears after a script runs, a crawler may see little.
• Duplicates and redirect chains after a move. If several versions of a page exist, Google picks one, and it may not pick the one you want.

The redesign case deserves its own mention. Developers often build a new site on a private staging copy and mark it noindex so the unfinished version stays out of search. If that setting is copied to the live site at launch, every page can tell Google to leave. Nothing looks broken to a visitor, so the problem can sit unnoticed for weeks.

The files that block crawlers are covered under AI bot blocking.

Pages that load their words late are covered under extractability.

Other search engines keep their own indexes, so the same logic applies to Bing. See Bing Webmaster Tools for that side.

Example: indexing at a health practice

This example is illustrative, not a transcript. A functional medicine clinic relaunches its website in the spring. The new design is cleaner and the pages load faster. Six weeks later the owner notices fewer calls from the website and assumes the new design is the problem.

The owner asks the developer to look. The developer inspects the homepage, a thyroid service page, the pricing page and the contact page in Google Search Console. The URL Inspection tool says the pricing page and the thyroid page are excluded by a noindex tag. The tag was part of the staging copy and moved to the live site when the pages were published. The homepage was indexed, because it had been fixed by hand during launch week. The other pages were missed.

The fix takes one afternoon. The developer removes the noindex setting from the affected templates. Then the developer uses the Request indexing button for each corrected page and confirms the sitemap lists them. Google says a request does not assure that a page will appear in the index, so the clinic rechecks the pages after a week and again after a month.

Two lessons come out of it. First, the redesign did not cause the drop, a hidden setting did. Second, the clinic's normal checks, such as looking at the site in a browser, could not have found it. The quickest way to see it was to ask Google about each page.

How to check yours in 60 seconds

1. Write down your ten most important pages, such as your services, pricing, contact and top condition pages.
2. Open Google Search Console and paste the first address into the search bar at the top. That opens the URL Inspection tool.
3. Read the result. It reports whether the URL is on Google. If it says the page is excluded by a noindex tag or blocked by robots.txt, mark it.
4. If you do not have Search Console, type site: followed by your address into Google. Google says that search does not list every indexed URL, so if a page is missing, treat it as a reason to inspect and not as proof.
5. Repeat for the other nine pages.

Any page that should earn patients and is not indexed goes to the top of your fix list.

What to do about it

1. Set up Google Search Console for your site if you have not already. It is free.
2. Inspect your ten most important pages and record the result and the date.
3. Ask your developer to remove any stray noindex setting from live pages, in the page code and in the server response.
4. Check that robots.txt does not block folders that hold service pages.
5. Make sure each important page is in your sitemap and linked from a hub page such as your services overview.
6. Use Request indexing for each corrected page. Google says this does not assure inclusion.
7. Put the same ten-page check in the launch checklist for every redesign, migration and plugin change.
8. Recheck each quarter, since plugins and host changes can rewrite settings without notice.
9. Name one person who owns this check, so it does not depend on memory.