The new X-Robots-Tag allows webmasters to add information about search engine indexing to the HTTP header. Google introduced the new X-Robots-Tag directive in 2007 to allow webmasters to control access to non-web page documents, such as Adobe PDF files, video and audio files.

At this time, the X-Robots-Tag supports the following commands:

  1. X-Robots-Tag: NOINDEX (Use this tag if you don’t want to show the URL in Google’s search results)
  2. X-Robots-Tag: NOARCHIVE (Use this tag if you don’t want to see a Cache link in the search result pages for the document)
  3. X-Robots-Tag: NOSNIPPET (Use this tag if you don’t want to display a summary in the search result pages.)
  4. X-Robots-Tag: NOFOLLOW (Use this tag if you don’t want Googe and Yahoo to index the links in the page.)

The X-Robots-Tag is currently supported by Google and Yahoo. Other search engines don’t support the tag yet. If your web server uses Apache, you can use the htaccess file to modify your HTTP headers.

Instead of using the X-Robots-Tag, you can also add meta tags with the same functionality to your web pages:

<meta name=”robots” content=”noarchive”>
<meta name=”robots” content=”nosnippet”>
<meta name=”robots” content=”nofollow”>
<meta name=”robots” content=”noindex”>

You can edit these tags directly in the head part of your web pages. They have the same effect as the corresponding X-Robots-Tags explained above.

This article come from Axandra.com


Related Posts:
6 Major Reason Search Engine Ranking Dropped
Google Top 10 Ranking Criteria on SERP
8 Steps Wordpress SEO Beginner Guide
Bali Villas Keywords Booming in Google
Is It Link Exchange Still Worthed?

Posted on Wednesday, 12th December 2007 in SEO | Comments (0)


Leave a Reply