So… why would you need to get pages into the index quicker? Well, you may have a time-based offer that you want the engines to display within a week or two. Your next natural crawl might not be due for another six weeks. So that’s six weeks of sales that you might be missing out on. Essentially most web owners will want to get the changes they have done to their websites into major search engines indexes to get the maximum benefit from their search engine optimisation strategies.

How to get into the Google index
There are a few ways to get into the Google index. Some of them take time; others can create an environment that gets you into the Google index a little quicker.

1. Get inbound links to your website
This is by far one of the best methods to make sure your pages get into the Google index. Not only will it get you into the index quicker than if your pages had no inbound links, it will also be more beneficial in the long run for your SEO strategy. Remember though, make sure your links have the correct anchor text, otherwise you’re missing a great opportunity to help your website rank better.

2. Submit your domain name
Submitting your domain name is probably the most common method that general people know about to get your website into the Google index. Please be aware that by submitting your website, this does not guarantee submission or rankings of any kind. Also, it could take weeks, perhaps months for your site to show up in the Google index.

3. Create a .html sitemap on your website or blog and submit this to Google.
This is a step up from the option listed above as it gives Google links to all the pages on your website on one page, rather than making Google’s GoogleBot go to your homepage and crawl through the site (remember, if you have information architecture issues i.e. Google can’t crawl your site for some reason or Google doesn’t crawl your entire site… believe me, it happens).

4. Create an XML sitemap and upload this into Google Webmaster Tools
If you want a thorough and dependable crawl of your site, this is the option I would go for . It’s easier than it seems and I will be giving you a full rundown of how to do this further in part two of this article.

How to find out how many pages you have in the Google index
We all know that getting optimised pages into the Google index is an important factor in generating rankings for driving more targeted traffic to your website, but how do you find out how many pages you have in the Google index? Simple! Use an advanced operator. Most major search engines have what’s called advanced operators. These are usually a command followed by a colon or :

To find out how many pages you have in the Google index, use the site: operator. Put site: into the search box followed by your website address.

If I wanted to find out how many pages of this website were in the Google index, I would type site:www.seohub.com.au into Google’s search box.

This is a really handy way to find out lots of other useful information about your website and your competitors’ websites.

Creating an XML Google sitemap
To create an XML sitemap to upload into the Google Webmaster Tools, you will need to use a sitemap generator . There are many available for free on the net, both online and desktop applications. (An online application is a piece of software that can be used through a web browser, whereas a desktop application is something that you download and install on your computer.)

I use www.sitemapbuilder.net . These guys have a great online and downloadable application that generates Google-friendly sitemaps. (Note - the downloadable application has a little more functionality.)

If you’re using sitemapbuilder.net it’s a simple case of putting your domain name into the URL field and hitting ‘Extract Links’.

I’ve added my domain but no no URLs are showing – what’s up with that?
If you’ve added your domain name into sitemapbuilder.net and it only comes back with a couple of pages (or just your domain name), then you may have some indexing issues. Basically sitemapbuilder.net works similar to a search engine’s crawler or bot. If sitemapbuilder.net can’t crawl all the pages on your site, then there’s a good chance the major search engines won’t be able to crawl your site either. I won’t go into specifics of why some menu structures and the ways in which sites are built make it difficult for engines and bots to crawl your site. I’ll just give you a solution to get around it.

Ways to make your website crawlable
If you don’t have the budget or skills to redevelop your site in a search engine friendly manner there are a few things you can do to ensure that search engines have the best chance to crawl your website:-

1. Deep link your pages
Deep linking is a fancy name for an easy concept. All deep linking means is linking the text on your pages to other pages on your website. For example, if on your homepage you had the text, ‘use the navigation items above to go to our photo gallery’, you would simply replace that text with something like ‘click on our photo gallery page to see some great photos’ and the term ‘photo gallery’ would be linked to the photo gallery page. Pretty simple hey?

2. Create a html sitemap on your site and link to this from the homepage
When I say a html sitemap, this is simply a page that’s set up with text links to every page on your website. Then, when you link to this from your homepage, a bot or crawler should follow this link and crawl through your sitemap.

These are just two ways that I commonly use to get around crawling issues if proper coding isn’t an option.

I’ve made my sitemap XML file… where do I put it?
Now that you’ve created your sitemap file you’ll need to put this in the root folder of your server. The root folder is the top folder or the first layer of your website’s folders. If you’re not sure about this, ask your web designer, programmer or IT department to do this for you.

So now that you have everything ready to upload to Google, you will need to log into your Google Webmaster Tools area. (Note: if you don’t have a Google account you will need to create one .)

Once you log into Google Webmaster Tools you will get to a dashboard area. This is where you can manage multiple websites or only one if you choose. This is also where you will tell Google about your newly uploaded XML sitemap. Start at the top of the site where it says ‘click here to add a site’. Put in your domain name and click ‘add site’.

The next step is to verify your site. This is really important if you wish to get loads of helpful stats from Google so I highly recommend doing this if you can. The verification process simply allows Google to determine that you indeed own this website before they give out any stats about the site. Verification is done in two ways:-

1. You can upload a verification file (do this by opening up notepad, putting in the verification code as the filename and uploading this to the root folder of your website) or

2. Use a meta tag verification method (you can add a meta tag to your site to verify).

Either method will end up with the same result… fascinating and very useful statistics from Google.

Now that you’ve verified your site, you will need to tell Google where your XML sitemap file is stored. On the left menu of the Google Webmaster Tools area there is a menu item called ‘Sitemaps’. This is where you will add the address of your sitemap file to the Google Webmaster Area. Under this there is another menu item called ‘add sitemap’ – click on this, then click the drop down box that says ‘choose type’. Most of you will be adding a general website sitemap (’cos let’s face it, if you’re adding a code or mobile sitemap you wouldn’t be needing to read this post!). You will then be asked to add the name of your sitemap file.

Remember, you don’t need to add your entire domain name into the box that shows up, just the name of your sitemap file. Click on the submit button and you’re away!

If anyone has any problems with their XML sitemap file, email me at This e-mail address is being protected from spambots. You need JavaScript enabled to view it and I’ll get back to you with some options.