RSS feed of httpuseragent.org RSS   Contact  

Contact Andrew

My E-mail:

Subscribe HttpUserAgent.org

RSS Feed of HttpUserAgent.org

Submit Your User-Agent

You are here: Home » User-agent List » Speedy Spider » User-agent string detail

Speedy Spider User-Agent String: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) Speedy Spider (http://www.entireweb.com/about/search_tech/speedy_spider/)

Speedy Spider

User-Agent NameSpeedy Spider [2 items]
User-Agent URLhttp://www.entireweb.com/about/search_tech/speedy_spider/
CategoryRobot, Spider, Crawler
OrganizationEntireweb.com  The city and country location of web http://www.entireweb.com/
User-agent StringMozilla/5.0 (Windows; U; Windows NT 5.1; en-US) Speedy Spider (http://www.entireweb.com/about/search_tech/speedy_spider/)
Sponsor
Visit Frequency115134
First VisitMar 12, 2010 03:37 PST
First IP88.131.106.30 - static-88.131.106.30.addr.tdc.se
Last VisitMay 05, 2012 16:37 PDT
Last IP88.131.106.22 - static-88.131.106.22.addr.tdc.se

Last updated: Jan 04, 2012 18:13 PST    

The description about Speedy Spider

Speedy Spider is an automated web crawler used to build the search engine index at Entireweb. The robot runs constantly, and visits sites in a random order.

About Entireweb

Entireweb is a trademark of Entireweb Sweden AB, a privately held, profitable corporation based in Halmstad, Sweden. Entireweb Sweden AB, incorporated in 1999, is dedicated to providing state-of-the-art search technology for both Internet and Intranet environments.

Entireweb currently handles millions of searches every month, and is a trusted partner of internationally acclaimed search engines such as Exactseek and IxQuick, who rely on Entireweb to provide world-class search results.

How to block Speedy Spider

Speedy obeys the Robot Exclusion standard. To prevent a particular file or directory of your site (or the entire site) from being crawled by Speedy, please follow the steps below:

  1. Place a file named robots.txt in the root directory.
  2. Example: http://www.httpuseragent.org/robots.txt.

  3. In this file you may specify disallowed resources in the following format:
  4. Indicates that Speedy should stop visit whole site further:

    User-agent: Speedy
    Disallow: /
    

    Indicates that Speedy should stop visit secret directory further:

    User-agent: Speedy
    Disallow: /secret

Speedy also supports HTTP META tag.

  • To prevent Speedy from indexing a certain page, add the following line to the header section:
  • meta name="speedy" content="noindex"
  • Speedy discovers new pages by following the links it finds on the web. If you do not wish Speedy to process the links it finds on your pages, add the line:
  • meta name="speedy" content="nofollow"
  • Both these restrictions can be written as the single line:
  • meta name="speedy" content="noindex,nofollow"
  • To prevent all bots (i.e. not just Speedy) from indexing/following links, simply change "speedy" to "all".

From which IP addresses/hostnames does Speedy operate?

Entireweb's crawlers primarily operate from the IP range 88.131.106.0/26. These IP numbers map to hostnames of the form cNN.entireweb.com, where NN is a hexadecimal number.