RSS feed of httpuseragent.org RSS   Contact  

Contact Andrew

My E-mail:

Subscribe HttpUserAgent.org

RSS Feed of HttpUserAgent.org

Submit Your User-Agent

You are here: Home » User-agent List » Speedy Spider » User-agent string detail

Speedy Spider User-Agent String: Speedy Spider (http://www.entireweb.com/about/search_tech/speedy_spider/)

Speedy Spider

User-Agent NameSpeedy Spider [2 items]
User-Agent URLhttp://www.entireweb.com/about/search_tech/speedy_spider/
CategoryRobot, Spider, Crawler
OrganizationEntireweb.com  The city and country location of web http://www.entireweb.com/
User-agent StringSpeedy Spider (http://www.entireweb.com/about/search_tech/speedy_spider/)
Sponsor
Visit Frequency45239
First VisitOct 23, 2007 18:57 PDT
First IP88.131.153.91 - speedyspider.entireweb.com
Last VisitMay 06, 2010 08:47 PDT
Last IP88.131.106.2 - static-88.131.106.2.addr.tdc.se

Last updated: Oct 30, 2010 06:44 PDT    

The description about Speedy Spider

Speedy Spider is an automated web crawler used to build the search engine index at Entireweb. The robot runs constantly, and visits sites in a random order.

About Entireweb

Entireweb is a trademark of Entireweb Sweden AB, a privately held, profitable corporation based in Halmstad, Sweden. Entireweb Sweden AB, incorporated in 1999, is dedicated to providing state-of-the-art search technology for both Internet and Intranet environments.

Entireweb currently handles millions of searches every month, and is a trusted partner of internationally acclaimed search engines such as Exactseek and IxQuick, who rely on Entireweb to provide world-class search results.

How to block Speedy Spider

Speedy obeys the Robot Exclusion standard. To prevent a particular file or directory of your site (or the entire site) from being crawled by Speedy, please follow the steps below:

  1. Place a file named robots.txt in the root directory.
  2. Example: http://www.httpuseragent.org/robots.txt.

  3. In this file you may specify disallowed resources in the following format:
  4. Indicates that Speedy should stop visit whole site further:

    User-agent: Speedy
    Disallow: /
    

    Indicates that Speedy should stop visit secret directory further:

    User-agent: Speedy
    Disallow: /secret

Speedy also supports HTTP META tag.

  • To prevent Speedy from indexing a certain page, add the following line to the header section:
  • meta name="speedy" content="noindex"
  • Speedy discovers new pages by following the links it finds on the web. If you do not wish Speedy to process the links it finds on your pages, add the line:
  • meta name="speedy" content="nofollow"
  • Both these restrictions can be written as the single line:
  • meta name="speedy" content="noindex,nofollow"
  • To prevent all bots (i.e. not just Speedy) from indexing/following links, simply change "speedy" to "all".

From which IP addresses/hostnames does Speedy operate?

Entireweb's crawlers primarily operate from the IP range 88.131.106.0/26. These IP numbers map to hostnames of the form cNN.entireweb.com, where NN is a hexadecimal number.