Home      Catalog      Shopping      Entertainment


In Norwegian

In English

Boitho distributed crawler (BETA).

This is a distributed crawler, where everybody can donate there superfluous computer resources and spare bandwidth, to help us create a bigger and better search engine.

Our goal is to make a general internet search engine with a thumbnail picture of all the pages. The problem is that the bandwidth and computer resources needed to make a thumbnail of a internet page is several times the resources needed just to download a HTML page. This means that Boitho has to spend more resources on crawling than other search engines.

To make the most of our available resources, and to allow volunteers to donate their superfluous bandwidth and idle CPU time, we have developed a distributed crawler for Boitho, like seti@home and Grub. That way people can install a program on their computers and help us with the crawling.


If you have any questions you can contact Runar Buvik, e-mail: runarb (at) boitho (dot) com, phone: +47.93030627




How to install


  • If you are running an old version of the crawler, you will have to close it, and then wait until the taskbar icon disappears.
  • Download and run the file BoithoCrawler.exe
  • Follow the instructions on the screen
  • Start the crawler by clicking on the start-menu -> All programs -> BoithoCrawler -> Start Boitho Crawler

If you want to you can create an account. That way you get bether statistics over how many URLs an robots.txt you have crawled. Go to Tools -> Options -> User Info -> Create User





Minimum system requirements

Operating system: Windows XP, Windows 2000 or Windows Millennium.
Hard disk: 1 MB for the program, and 30 Mb for cachefiles.


Blocking us from accessing you site

You may block access by create/modify you robots.txt file. Please refer to the crawlers homepage for blocking instructions.

Known bugs

  • The bandwidth meter does not always work properly if you have two or more network cards.

© Copyright Boitho 2011