The Science of Meta tagging
Did you know that search engines don’t usually send their creepy crawlers to index your site without permission? This is one of the biggest mistakes website owners make.
I can see many of you reading this have your eyes wide open, asking yourself if you have done this part! It is good question to ask because I would say 7 out of 10 people desperate for my SEO help do not have this done and anything else they do to help doesn’t make a difference.
There are two ways you can allow spiders, bots and crawlers to come index your site. You can include a file called robot.txt with permissions (we all talk about this later on this blog.) or you use the Meta Robots tags to do this which is the most efficient way. If you haven’t done this, well you might see your landing page on a website, but you will never see a good ranking and none of your other pages will get indexed.
The Meta Robots Tag
This gives you the ability to specify whether search engine bots, spiders and crawlers should index page or follow the links appearing on that page.
The following is the Meta Robots tag which should be on your landing page. (You will learn where to put Meta tags in the next post.) I add it to every page of the website:
<Meta name=”robots” content=’index, follow’>
There are other syntax’s you can use instead of “index, follow” which are:
- index: This allows the spider to index that page.
- noindex: This instructs the spiders not to index the page.
- follow: This instruct the spiders to follow the links from that page and index them.
- nofollow: This instructs the spider not to follow links from that page for indexing.
- all: This is identical to “index, follow”.
- none: This is identical to “noindex, nofollow” telling the search engines to ignore the web page.
The other acceptable tags:
<meta name=”robots” content=”noindex, follow”>
<meta name=”robots” content=”index, nofollow”>
<meta name=”robots” content=”noindex, nofollow”>
<meta name=”robots” content=”all”>
<meta name=”robots” content=”none”>
Note: Keep in mind you only use one of the tags. They are not used in combination with each other.