RSS feed of httpuseragent.org RSS   Contact  

Contact Andrew

My E-mail:

Subscribe HttpUserAgent.org

RSS Feed of HttpUserAgent.org

Submit Your User-Agent

You are here: Home » User-agent List » Speedy Spider

Speedy Spider User-Agent String

Speedy Spider

User-Agent NameSpeedy Spider
User-Agent URLhttp://www.entireweb.com
CategoryRobot, Spider, Crawler
OrganizationEntireweb.com
Total Strings2
Last VisitMay 05, 2012 16:37 PDT

All user-agent string from Speedy Spider

#User-Agent StringVisit FrequencyLast VisitView
1Speedy Spider (http://www.entireweb.com/about/search_tech/speedy_spider/)45239May 06, 2010 08:47 PDTView Detail
2Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) Speedy Spider (http://www.entireweb.com/about/search_tech/speedy_spider/)115134May 05, 2012 16:37 PDTView Detail

The description about Speedy Spider

Speedy Spider is an automated web crawler used to build the search engine index at Entireweb. The robot runs constantly, and visits sites in a random order.

About Entireweb

Entireweb is a trademark of Entireweb Sweden AB, a privately held, profitable corporation based in Halmstad, Sweden. Entireweb Sweden AB, incorporated in 1999, is dedicated to providing state-of-the-art search technology for both Internet and Intranet environments.

Entireweb currently handles millions of searches every month, and is a trusted partner of internationally acclaimed search engines such as Exactseek and IxQuick, who rely on Entireweb to provide world-class search results.

How to block Speedy Spider

Speedy obeys the Robot Exclusion standard. To prevent a particular file or directory of your site (or the entire site) from being crawled by Speedy, please follow the steps below:

  1. Place a file named robots.txt in the root directory.
  2. Example: http://www.httpuseragent.org/robots.txt.

  3. In this file you may specify disallowed resources in the following format:
  4. Indicates that Speedy should stop visit whole site further:

    User-agent: Speedy
    Disallow: /
    

    Indicates that Speedy should stop visit secret directory further:

    User-agent: Speedy
    Disallow: /secret

Speedy also supports HTTP META tag.

  • To prevent Speedy from indexing a certain page, add the following line to the header section:
  • meta name="speedy" content="noindex"
  • Speedy discovers new pages by following the links it finds on the web. If you do not wish Speedy to process the links it finds on your pages, add the line:
  • meta name="speedy" content="nofollow"
  • Both these restrictions can be written as the single line:
  • meta name="speedy" content="noindex,nofollow"
  • To prevent all bots (i.e. not just Speedy) from indexing/following links, simply change "speedy" to "all".

From which IP addresses/hostnames does Speedy operate?

Entireweb's crawlers primarily operate from the IP range 88.131.106.0/26. These IP numbers map to hostnames of the form cNN.entireweb.com, where NN is a hexadecimal number.