black friday 2026 label for the hellobar

Get ready for Black Friday! See offers →

Marketing9 min. read

What is Google indexing and how do you get your site indexed?

Last updated: 16 Nov 2026
Google crawler indexing website pages and displaying them in search results

When Google indexes a page, it has crawled the page, analysed what's on it and stored it in the Google index. That index is the collection of pages Google searches when someone looks something up. A page that isn't indexed can't appear in Google Search, however good it is. Google indexing usually happens without your help. You can still check which of your pages are indexed and ask Google to crawl the ones that are missing.

How Google crawls, indexes and ranks your pages

The Google indexing process runs from crawling, when Google finds and reads a page, to indexing, when it stores what it found. The last stage is serving, when Google ranks the pages that match a search and shows them as results. Not every page makes it all the way through.

Crawling: how Googlebot finds your pages

Before Google can crawl a page, it has to know the page exists. Google's crawlers follow the links on pages they already know, and they also read the sitemaps that website owners submit. The web keeps changing, so crawling never stops. The crawlers keep track of how often a page tends to change and return when it's due.

The program that visits your pages for Google is called Googlebot, and crawlers like it are also known as robots or spiders. Google's crawler strategy depends on the website. How much Googlebot crawls varies with your site's size, how often it's updated, the quality of its pages and their relevance. Googlebot also tries to crawl without overwhelming your server. You can't pay Google to crawl your website more often, either.

Before Googlebot fetches a page, it checks your website's robots.txt file and skips any page the file blocks. A page that only opens after a login stays out of reach too. Google also renders each page it finds, much as your browser does. That matters if your content management system (CMS) or theme loads text with JavaScript, because Google has to run that code before it can see the content.

Indexing: what Google stores about a page

Once a page is crawled, Google analyses its content and meaning. It notes signals such as the words on the page and how fresh the website is. It also checks whether the page is a duplicate of another one. When pages are duplicates or very similar, Google picks one of them as the canonical, the version it may show in search results.

What Google learns is stored in the Google index. Google doesn't promise that every page makes it in, though. A page with a noindex rule, or one Google sees as a duplicate, is left out.

Serving: when your page appears in results

When someone searches, Google looks through its index for the pages that match and ranks them to show the most relevant, useful results first. So being in the index lets your page take part, and the ranking decides where it appears.

How to get your website indexed by Google

Google Search Console, Google's tool for website owners, is where you check your pages and ask Google to crawl them. You add and verify your website there first, as steps 1 and 2 below show.

You'll need a sitemap for steps 3 to 5. It's a file that tells Google which pages your website has. If you use a website builder or another hosted platform, it probably manages your sitemap for you, so check its help pages first. Some content management systems can create one for you automatically, and so does our Aida AI Website Builder. In a content management system, search for how your CMS creates its sitemap and note the sitemap's address for step 3.

To check your site in Google Search Console and ask Google to crawl a page, go through these steps in order:

  1. Add your website to Search Console as a property, the tool's name for a website you manage there.
  2. Verify the site with one of the methods Search Console lists for it, so Google knows you manage it. Its verification page shows which methods it recommends for your site.
  3. Check that your sitemap is live on your site, usually at an address such as harbourlanebakes.co.uk/sitemap.xml.
  4. Test that Google can open the sitemap. It must not sit behind a login.
  5. Add the sitemap's address in the Sitemaps report and click Submit.
  6. Inspect the address of a page you want indexed with the URL Inspection tool.
  7. Click Request indexing on the result page.

For step 4, run a live test of the sitemap's address in the URL Inspection tool. If Page fetch shows "Successful", Googlebot can open the sitemap and no login is in the way.

How long Google takes to index a page

Google can take from a few days to a few weeks to crawl a new page. A request doesn't make indexing instant, and it doesn't guarantee that the page gets into the index. You can only send a limited number of requests a day, so save them for the pages that matter most. The URL Inspection tool shows when Google last crawled the page, so you can see whether it has been back.

Checking whether Google has indexed your site

You can check your website in three ways, from a quick look to the full picture:

  • For a quick look, type site:harbourlanebakes.co.uk into Google, with your own domain in place of the example, to see indexed pages from your website. The list isn't always complete, especially on a bigger website, so a missing page here proves nothing on its own.
  • For a reliable answer on one page, enter its full address in the URL Inspection tool in Search Console. "URL is on Google" means the page can appear in results, although that isn't guaranteed.
  • For the full picture, open the Page indexing report in Search Console. It shows how many of your pages are indexed and which ones aren't, with the reason Google gives for each.

Two reasons you may see in that report start with "Discovered" and "Crawled", both followed by "currently not indexed". Discovered means Google knows the page but hasn't crawled it yet, typically because it put the crawl off to avoid overloading your site. Crawled means Google has read the page but hasn't stored it for now, so make the page more useful and link to it from your other pages.

Why Google is not indexing your page

If a page of yours isn't on Google, the URL Inspection tool usually shows why, for example a robots.txt block or a noindex rule. Check the page against these common causes.

  • Crawling is blocked. A rule in your robots.txt file can stop Googlebot from crawling the page. Change or remove the rule if you want the page in Google.
  • The page has a noindex rule. A noindex rule tells Google to leave the page out. Your CMS may have a search engine settings page that switches it on, so check there first and remove the rule if you want the page indexed.
  • Google can't reach the page. A page behind a login, or a server that returns errors, keeps Googlebot out. Check that the page opens in a private browser window without logging in.
  • Google chose another page. When pages are very similar, Google indexes the one it picks as the canonical and leaves the others out. Give each page you want people to find its own content.
  • The content is thin. To appear in Google, a page has to follow the Google Search Essentials, Google's own guidelines for its results. Add the information a visitor came for.
  • Nothing links to the page. A page with no links to it that isn't in your sitemap is hard for Google to find. Link to it from related pages, and search your sitemap for the page's address.
  • The site is new. It can take Google a few weeks to notice a brand-new website. Request indexing for your homepage and be patient.

A noindex rule sits either in a meta tag in the page's code or in the page's HTTP header. To find the meta tag yourself, search the page's source code for "noindex". If you want to keep a page out of Google on purpose, add the rule instead, and don't also block the page in robots.txt. Googlebot has to crawl the page to see the rule.

Ways to help Google crawl and index your site

A few habits can make your pages easier for Google to find and to understand.

  • Link your pages to each other. Googlebot follows links, so internal links from pages it already knows lead it to new ones, as long as they're real links with an address rather than buttons that only work through a script.
  • Keep your sitemap up to date by adding new pages as you publish them. Add the sitemap's address to your robots.txt file too, so crawlers can find it.
  • Put the important information in words on the page. Google's own checklist asks for text, not only images or video.
  • Write page titles and clear headlines that say what each page covers, for visitors and for Google.
  • Use descriptive page addresses (URLs). They help people understand whether a result is useful before they click it.
  • Google wants pages that answer what people came for, so write useful content for your visitors rather than for search engines.
  • A link from another website can lead Google to yours. Don't pay for links, though, as that can count as a breach of Google's spam policies.

You may read about crawl budget, the limit on how much Google crawls a single website. Google's crawl budget guide is meant mainly for large websites, such as those with a million pages or more that change about once a week, or 10,000 pages or more that change every day. For a website without lots of pages that change quickly, an up-to-date sitemap and a regular look at the Page indexing report are enough.

Starting a new website? Our Aida AI Website Builder creates a sitemap for you, so search engines can find your pages from the start.

Build my website

Indexed is not the same as ranked

Getting indexed means your page can appear in Google. It doesn't decide where. For each search, Google's ranking systems weigh many factors. They include the words of the search, how relevant and usable each page is and the searcher's location and settings. An indexed page won't appear for every search. Results also differ from person to person, so its position can vary.

Moving up the results is the work of search engine optimisation (SEO). To see what to improve first, you can add marketgoo, an SEO tool that analyses your site and gives you a list of recommendations to work through. It's a paid monthly subscription, and the price depends on whether you choose marketgoo Start or Evolution.

Once Google can find and index your pages, that SEO work has something to build on. If you're starting from scratch, you can build your website with us and add it to Search Console from day one, so you see from the start which pages Google has indexed.


Recommended reading

  • Illustration of a Black Friday online shop with sale products, a megaphone, sales growth chart, checklist, and marketing idea lightbulb.
    Marketing19 Oct 2026

    15 ideas for Black Friday marketing

    Make the most of Black Friday with 15 marketing ideas to attract customers and boost sales. Discover ways to promote your offers, reach more shoppers and make your campaign stand out.
  • AI SEO dashboard showing organic traffic growth, keyword rankings, backlinks, CTR, and AI-powered optimisation recommendations.
    Marketing30 Dec 2025

    AI SEO: how to use AI for search engine optimisation

    A basic guide to AI in SEO – let's look at how artificial intelligence can help you speed up your SEO tasks.
  • AI search results highlighting a freelance web designer for small businesses with AI visibility insights.
    Marketing30 Dec 2025

    AI search optimisation: how to get your website into AI answers

    A basic guide to AI in SEO – let's look at how artificial intelligence can help you speed up your SEO tasks.