Jump to content



HelloInsomnia

Member Since 10 Apr 2012
Offline Last Active Yesterday, 04:17 PM
*****

Topics I've Started

Website Crawler 3 - An Efficient Web Crawler For Ubot

04 December 2018 - 11:31 PM

Website Crawler 3 is a powerful application capable of crawling hundreds of thousands of pages looking for information.

 

 

93PDr.jpg

 

V1 users – how to upgrade.

 

Case Study #1 Powering A Search Engine:

 

 

93PDr.jpg

 

Case Study #2 Domain Hunting For SEO

 

 

Besides the search engine idea, you could rank the recipes and create a site of only the best recipes (like 4.5 stars or higher). You could create your own rating system based on the star rating, how many people made it, how many reviews etc. You could also rate the difficulty based on the number of ingredients and how long it takes to make. This would allow you to make a site with only the easiest recipes (and maybe the highest rated easy recipes).

This is source code. Once you purchase Website Crawler 3 it’s yours to modify, skin, sell – do whatever you want with it.

You should have the Pro or Dev version of Ubot Studio. You should also have the Heopas Plugin and File Management Plugin. Both are free. Links are included in the “Read Me First” file.

  • Capable of scraping 100k+ pages
  • Multithreaded
  • Extremely memory efficient
  • Powerful internal URL detection
  • Easy to modify to scrape whatever you want!
  • Ability to save huge amounts of results without Ubot memory spiking
  • Ability to crawl multiple sites back to back
  • Add in proxies (supports username:password proxies too!)
  • Whitelist filter (must contain) for URL, Title or Innertext
  • Blacklist filter (must not contain) for URL, Title, or Innertext
  • Ability to use both filters at once!
  • Scrape by regex, you don’t even have to modify the code to use this
  • Built-in regular expressions that you can modify for Emails, Phone Numbers, Files, Proxies
  • Ability to remove query if you want (that http://website.com/?this_is=the_query)
  • Powerful extension filter built in which filters pages that can cause problems, you can modify this right in the UI
  • Set a custom user agent if you want

93PDr.jpg


[Tutorial] How To Make Plugins For Ubot Studio

15 November 2018 - 08:39 PM

This covers the basics of making a Ubot Studio plugin. If there's interest I'd like to make a series or maybe even a full course on all of the things I know about plugin development.

 

Enjoy!

 

 

Video Links:

 

Wiki Page: http://wiki.ubotstud...gin_Development

 

Clean Code Ebook: https://elitebotters.com/clean-code


[Sell] Http Post Plugin & Apis Course

13 November 2018 - 08:13 PM

KmIGBTL.png


[Sell] Ubot Studio: Mastering The Fundamentals Course - Learn Ubot!

13 November 2018 - 08:09 PM

OfJs11R.png


Ubot Launcher 2 - Auto Select Ubot Plugins, Code Viewer And More!

04 November 2018 - 11:34 PM

OSKdaoN.png

 

V 2.0 is finally here. Rewritten from scratch. Now with code view.

 

Note that this is "Name Your Price" I explain a bit in the video what that means!

 

 

  • Launches any Ubot version
  • Add as many Ubots as you want
  • New Code view window
  • Auto select plugins in file
  • Change plugins manually before launch
  • Change the browser before launch
  • Counts lines and tabs in your file

93PDr.jpg