|Category||Robot, Spider, Crawler|
|Last Visit||Feb 01, 2013 22:24 PST|
|#||User-Agent String||Visit Frequency||Last Visit||View|
|1||Mediapartners-Google||522673||Feb 01, 2013 22:24 PST||View Detail|
Mediapartners-Google is a Google's web crawling bot, it's for adsense only. Google AdSense delivers advertisements to a broad network of affiliated sites. It indexes a page so the Adsense Publishers Program knows what a page is about, so they can provide related advertisement.
In order for Google to properly target ads to the website, Google crawler needs to be able to access the webpages. The useragent Google uses to crawl the site will contain the string "Mediapartners-Google". If you are a webmaster, please set up your web server to grant access to your content any useragent that contains "Mediapartners-Google".
Mediapartners-Google obeys the Robot Exclusion standard. You need a robots.txt file only if your site includes content that you don't want Google search engines to index.
This example indicates that Mediapartners-Google should stop visit this site further:
User-agent: Mediapartners-Google Disallow: /
The IP addresses used by Googlebot change from time to time. The best way to identify accesses by Googlebot is to use the user-agent (Googlebot). You can verify that a bot accessing your server really is Googlebot by using a reverse DNS lookup.