You are here: Home » User-agent List » Googlebot 2.1 » User-agent string detail
Googlebot 2.1 User-Agent String: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
| User-Agent Name | Googlebot 2.1 [1 items] |
| User-Agent URL | http://www.google.com/bot.html |
| Category | Robot, Spider, Crawler |
| Organization | Google Inc.  |
| User-agent String | Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) |
| Sponsor | |
| Visit Frequency | 2879209 |
| First Visit | Oct 23, 2007 04:43 PDT |
| First IP | 66.249.70.241 - crawl-66-249-70-241.googlebot.com |
| Last Visit | Dec 31, 2011 23:59 PST |
| Last IP | 66.249.68.233 - crawl-66-249-68-233.googlebot.com |
Last updated: Nov 26, 2008 06:16 PST
The description about Googlebot 2.1
A Googlebot is a search bot used by Google. It collects documents from the web to build a searchable index for the Google search engine.
If a webmaster wishes to restrict the information on their site available to a Googlebot, or another well-behaved spider, they can do so with the appropriate directives in a robots.txt file,[1] or by adding the meta tag to the webpage. [2] Googlebot requests to Web servers are identifiable by a user-agent string containing "Googlebot" and a host address containing "googlebot.com".
Verifying Googlebot
You can verify that a bot accessing your server really is Googlebot by using a reverse DNS lookup, verifying that the name is in the googlebot.com domain, and then doing a forward DNS lookup using that googlebot name. This is useful if you're concerned that spammers or other troublemakers are accessing your site while claiming to be Googlebot.
For example:
> host 66.249.66.1 1.66.249.66.in-addr.arpa domain name pointer crawl-66-249-66-1.googlebot.com. > host crawl-66-249-66-1.googlebot.com crawl-66-249-66-1.googlebot.com has address 66.249.66.1
Google doesn't post a public list of IP addresses for webmasters to whitelist. This is because these IP address ranges can change, causing problems for any webmasters who have hard coded them. The best way to identify accesses by Googlebot is to use the user-agent (Googlebot)