SpeedyBot - Customize IE to make UBot faster

[AKprogrammer 7]

This isn't a UBot, but it's a utility that I use while running my UBots - so I hope no one minds me posting it here.

 

SpeedyBot is a .NET program that can make your botting/spidering faster. It intercepts HTTP traffic, and modifies the HTML of a webpage before your browser reads it. This program can insert or remove any code from a webpage before UBot or any browser loads it.

 

You can:

1. Remove all iframe, flash, or script tags before loading.

2. Insert Jquery, or your own javascript and have that code become part of the page.

3. Remove images from a webpage. Or remove all images except the captcha images(experimental).

 

I'm selling this tool at my website for $40. BlogPost with YouTube Video

 

I'm planning to add more features to SpeedyBot in the future, and any upgrades will automatically be emailed to anyone who buys it now. You can post any questions you have in this thread, or feel free to PM me.

 

Also as a side note... I was busy with php contract work before so couldn't do much botting - but the company I was doing contract-work for went under. If you have a UBot bot you need created, or any custom .NET development done - PM me. Thanks.

[webautomationlab 21]

That is totally awesome. It solves a major issue with browser rendering a lot of urls in succession in IE.

 

You should definitely offer to disable external CSS, which would solve the background images issue, and for the purposes of scraping, unstyled HTML is still fine.

 

I am very impressed.

[AKprogrammer 7]

Thanks for the suggestion.

 

That does seem like an essential option... so I added another checkbox so you can block external CSS files. Anyone who buys SpeedyBot will get the new version.

 

For the most part - IE sucks as a webspider. I think this utility can change that.

[Pete 121]

Ubot struggles some what with xml feeds and fails with Rss feeds which made me return to winautomation could this be used to correct these problems by inserting code? (I found this with the original ubot not the latest updates)

[Net66 54]

This is a good idea to speed up your own bots but quite a lot of us on here sell compiled bots. Maybe there is a market for a version that can be redistributed and reads all the settings from a file. Might be a licencing nightmare though, but something you might want to think about.

 

Andy

[alcr 135]

Nice tool indeed! +1 in rep

[webautomationlab 21]

For the most part - IE sucks as a webspider. I think this utility can change that.

I don't need it now, since I encountered these issues last year and have come up with workarounds like disabling stuff in IE and other software, but when I need it, I am definitely going to buy it. It's a great idea to pre-process.

 

Ubot struggles some what with xml feeds and fails with Rss feeds which made me return to winautomation could this be used to correct these problems by inserting code? (I found this with the original ubot not the latest updates)

UBot has added the ability to download which I have not fully tested yet, but may provide similar functionality to what W/A does.

 

@OP, I'm curious how Speedbot works with rendering xml (or non-HTML) to the screen.

 

This is a good idea to speed up your own bots but quite a lot of us on here sell compiled bots. Maybe there is a market for a version that can be redistributed and reads all the settings from a file. Might be a licencing nightmare though, but something you might want to think about.

If you watch his video, he is already talking about doing that down the line.

[AKprogrammer 7]

This is a good idea to speed up your own bots but quite a lot of us on here sell compiled bots. Maybe there is a market for a version that can be redistributed and reads all the settings from a file. Might be a licencing nightmare though, but something you might want to think about.Andy

 

Yes I was thinking of doing that. It would take some real work though - making settings files, an .exe to distribute that runs silently in the background, another .exe that you call from UBot that uses Named Pipes to communicate with a running instance of the silent version of SpeedyBot... and although I already have a licensing system, this would make it more complex.

 

I was hoping some people would buy the tool now to support future development on this.

 

@OP, I'm curious how Speedbot works with rendering xml (or non-HTML) to the screen.

 

I'm not sure what Zap is trying to do... but right now SpeedyBot doesn't try to intercept non-HTML traffic. If you intercepted an RSS feed I'm not sure what code you could inject to make it easily readable by UBot. The best way to extract info from feeds and use it in UBot is probably to read the feed in an RSS Reader (or download the feed and run a utility that does that).

 

If someone has a specific idea of what I could do with RSS to make UBot handle RSS better, let me know.

[meter 145]

If someone has a specific idea of what I could do with RSS to make UBot handle RSS better, let me know.

 

As always, if Ubot can't do it, the almighty SHELL command plus a little programming on your part can!

 

IN UBOT:

-Take RSS Feed URL, write it to text file called rss.txt.

-Use shell command to run your own custom program, lets call it rss.exe.

 

IN RSS.EXE:

-Read feed, extract the data you want to extract from the feed, export it to output.txt and close program.

 

BACK TO UBOT:

-Open output.txt for rss feed data and load it into a list/variable/whatever... and have fun with it!