pacbotter 1 Posted April 10, 2015 Report Share Posted April 10, 2015 I am not running an automated process, but rather a UI with multiple buttons - each linked to a define to be run on the page - like a button based control panel. The control panel is designed to allow me to run multiple processes on each page to scrape specific information and load the contents of the page into fields in the UI. Each button runs a different define - scraping different info - and placing it into different on-html-window fields... When I run the bot, something strange occurs - the first 5 pages - it works great. I have 3 processes that I run on each page - and for 5 pages - it works s well that I have a button that clicks an automated process that runs the three (3) defines like clockwork. But on the sixth page (on average) the page seems to restrict the first define... the first scrape is delayed although the process begins. On consecutive pages the first define gets so delayed that the second define has an error. Once this happens I can't run the automated process anymore and have to go to a manual button 1 - define - wait until it happens before I can click button 2 for define No 2. Eventually it's impossible to run anything because the first process doesn't work and the bot needs to be closed and re-opened. It's very strange, I can't tell whether it's my code, or the website I am running this on - that makes the bot slow down. I am wondering if anyone can help me - even test and possibly re-code it for me (work for hire). A couple notes:My bot has a lot of code. See the screenshot. The website I am running on - does not like bots. At least I don't think it does - because if it offered what I wanted (without bots) everyone would sign up for it. Quote Link to post Share on other sites
Code Docta (Nick C.) 638 Posted April 10, 2015 Report Share Posted April 10, 2015 Try anyone or combination of these. Should help. With out seeing inside hard to say. comment("use this before each navigation")set user agent("Firefox/37.0.1 FxSync/1.39.0.20150402191859.desktop")comment("ust this at the end of your definewhen you are done with each page")close pagecomment("use this if you dont need to see the images")allow images("No") Can be any current User Agent. Try using 'Large Data' plugin. http://www.ubotstudio.com/forum/index.php?/topic/16308-free-plugin-large-data/ for list or tables Also, try to use more functions to reduce memory.if you have a lot of 'set' variables for the same thing use one in a function instead. Like if you did a lot of copy paste of commands or variables then make a function instead.Each one adds to memory. Finally, stack your code if possible and/or use a function. set(#a, "adam@gmail.comclint@aol.combobby@yahoo.comdanny@gmail.com", "Global")set(#b, $find regular expression(#a, ".*@gmail.com"), "Global")set(#replace, $replace regular expression(#b, "@.*", $nothing), "Global")alert(#replace)comment("\"stacked\" or refactored")set(#all gmails, $replace regular expression($find regular expression(#a, ".*@gmail.com"), "@.*", $nothing), "Global")alert(#all gmails)comment("The functioninstead of makin onr for every emaiprovider thid leaves more universaland gives more functionality.and can run thru a list or use individusally")define $Find and rmv email host(#email host) { return($replace regular expression($find regular expression(#a, ".*@{#email host}"), "@.*", $nothing))}alert($Find and rmv email host("gmail.com"))alert($Find and rmv email host("yahoo.com"))Just a simple demonstration of stacking and of course you can just use replace or just replace regex instead. CD Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.