GWebCrawler, is a project IÂ started to workÂ on approx. 1 year or so ago.Â Currently I am busy with other projects, so I have decided toÂ let others work onÂ this Google Sitemap Creator andÂ you can downloadÂ the free source code of this application right here.Â GWebCrawler’s web indexing engine isÂ running under the MS Windows environment andÂ it’s written purelyÂ in VB.NET.Program is currently using only one thread to browse and index the site, but in my tests, adding new threads didn’t make big difference or made the applicationÂ significantly faster. But of course if any of you, programmers out there :),Â feel like working on this project and playing around with it, you may find the ways to improve it or add additional threading logicÂ to speed up a crawling process.
Anyhow, at its current stage, program has very high execution and running speed and it is also very small in size (72 Kb)and very importantly it’s free of charge to use or completely rework. Basically I don’t care what you do with it under one condition, that my name is mentioned in the application. As of now, as I mentioned already, this project is not very mature, however, itâ€™s an one-man-project and that is why thereâ€™re still some bugs and missing features.
You can download the source code of GWebCrawler 1.7 Beta here: www.trytobreak.com/gwebcrawler-source.rarÂ or download the current working version here:Â http://www.trytobreak.com/webcrawler.zip – 72kb (No installation required. Just run webcrawler.exe)
If you have any question or suggestion about this project, leave me a comment here or lookÂ for my contact at http://www.trytobreak.com/,Â Iâ€™dÂ love to hear from you.
Also, if you have any feature request, you can always do the demand, but thereâ€™s no warranty that Iâ€™ll implement it.
I hope you enjoy GWebCrawler & Google Sitemap Creator as much as I enjoyed coding it.