Joe J. - Blog

GWebCrawler goes Open Source

May 27th, 2006

GWebCrawler, is a project I started to work on approx. 1 year or so ago. Currently I am busy with other projects, so I have decided to let others work on this Google Sitemap Creator and you can download the free source code of this application right here. GWebCrawler’s web indexing engine is running under the MS Windows environment and it’s written purely in VB.NET.Program is currently using only one thread to browse and index the site, but in my tests, adding new threads didn’t make big difference or made the application significantly faster. But of course if any of you, programmers out there :), feel like working on this project and playing around with it, you may find the ways to improve it or add additional threading logic to speed up a crawling process.

Anyhow, at its current stage, program has very high execution and running speed and it is also very small in size (72 Kb)and very importantly it’s free of charge to use or completely rework. Basically I don’t care what you do with it under one condition, that my name is mentioned in the application. As of now, as I mentioned already, this project is not very mature, however, it’s an one-man-project and that is why there’re still some bugs and missing features.

You can download the source code of GWebCrawler 1.7 Beta here: www.trytobreak.com/gwebcrawler-source.rar or download the current working version here:  http://www.trytobreak.com/webcrawler.zip - 72kb (No installation required. Just run webcrawler.exe)

If you have any question or suggestion about this project, leave me a comment here or look for my contact at http://www.trytobreak.com/, I’d love to hear from you.

Also, if you have any feature request, you can always do the demand, but there’s no warranty that I’ll implement it.

I hope you enjoy GWebCrawler & Google Sitemap Creator as much as I enjoyed coding it.

Entry Filed under: English Blog