multithreading in ubot?

[iglow 8]

since i cant find answer. does any version [pro or dev or normal] have the threading? i dont mean the old thread command, that needs to f**k up whole bot to work with, but just a small option that will run the same stuff in few windows without copying whole bot to other thread.

[Seth Turin 223]

wait, how does the thread command f**k up the bot? to run stuff in another window, you just use in sub window. to run the window in a separate thread, you put in sub window inside the thread command. I just touched up both commands and they should work perfectly now.

[marcinek 1]

Seth please example...

[Kreatus (Ubot Ninja) 422]

Seth please example...

check out this thread http://ubotstudio.com/forum/index.php?/topic/5425-pro-and-dev-versions-are-finally-here/page__view__findpost__p__24697

[iglow 8]

wait, how does the thread command f**k up the bot? to run stuff in another window, you just use in sub window. to run the window in a separate thread, you put in sub window inside the thread command. I just touched up both commands and they should work perfectly now.

 

seth what i mean is.

lets say ive got BIG A** bot. that i want to be threaded. currently i have to copy the same 'source' of the BIG A** BOT to every 'sub window' 'thread'. so 1 change in big ass bot = copy paste work over each thread.

 

i asked if theres some kind of workaround that the bot gets runned threaded but without copying 50 times [for 50 threads] the same content.

[Seth Turin 223]

sounds like more of an interface problem than a threading problem. what I would do is move all the commands into a loop (set to 1 loop) and then just copy the loop into where you need it.

[iglow 8]

Okay but if i want to have 100 threads, i have to copy paste 100 times the same, which will make bot bigass big [maybe even kill ubot?]. will there ever be something like 'thread start from here' 'thread end here' and we put it on start of a loop and end, and whole thing is gonna get threaded without copying?

 

its some idea dunno if implementation is hard.

[New Guy 4]

I asked about all of this in this thread http://ubotstudio.com/forum/index.php?/topic/5478-better-thread-control/ but it seems to have been ignored.

 

IMO the thread command could definitely use some work right now as it is almost impossible to create an actual mutli threaded bot that the end user can use.

 

I'm waiting patiently and hoping that there will soon be a fix for this though.

[iglow 8]

yep copy pasting stuff 100 times to get 100 threads is not really threading

since Seth wrote here maybe he wont ignore it this time!

[Psychotix 0]

I've tried to run script inside a thread, inside subwindow it doesnt work.

 

Do I have to put the entire script inside thread>subwindow?

[meter 145]

I think the multithreading implementation is a bit bugged. Will try to put together an example script demonstrating why it is bugged and putting it in the bugs section

[Pete 121]

Working multithreading bot but I have to use all mobile sites or it crashes

So I need some one with a more powerful pc to load non mobile pages into it and test drive it please.

[Pete 121]

Don't worry guys i tested myself now and yes it did work

[Psychotix 0]

This whole ubot multithreading is a myth.

Technically its there, but not really to be actually useful to anyone!

 

At the end of the day, its not what you can claim. Its what it can actually do!

[Abs* 12]

I just tested it on a google serps scraper -

 

i wasnt too sure how to control the loop list total or how it would work so i based it on a maths function dividing the scraped search urls by 4 and have them all loop that many times - i coded 3 threads and one for the application browser.

 

im sure i have done this totally wrong and one loop list total would do the trick -

 

however it was a test to see how it would work

 

so i tested it in the "in-sub window" and also "in-IE window"

 

The in sub command seemed much better as it would pull up a smaller browser - the in ie also did work -

 

however they didnt work concurrently - what happened was that they would all wait for each other to complete then they would navigate to the next list item

 

I guess this is good to get like 3 or 4 things done - but if one completes before the other then it will wait for the others before moving on -

 

This is what I noticed anyway

 

I also know that there have been many notes regarding the cpu usage that the thread commands will use - so what might work on your super hot computer may not work great or work at all on others - not too sure on this one but im sure one of my old laptops will not handle it -

 

But yes I agree - copying each sub into each thread command will certainly be a nightmare for large bots -

 

I currently have bots which work with platforms - users go through thousands of signups a day - threads would be great for this -

 

however each platform on each script - ie the signup script will have a number of subs just for signing up as many sites use different id's even though they are using the same platform -

 

so i may have one sub for navigating to the registration page - then another for adding the usernames and passwords - etc

 

duplicating the subs inside of thread commands will be a nightmare as there already seems to be a big issue with bot sizes which i have faced since i started to build larger scripts -

 

thanks

[Pete 121]

abs from what you are saying

however they didnt work concurrently - what happened was that they would all wait for each other to complete then they would navigate to the next list item

 

I guess this is good to get like 3 or 4 things done - but if one completes before the other then it will wait for the others before moving on -

Possibly you need to rethink how you control the number of threads

 

on my old pc (2G cpu 1GB ram)i can now just about run 3 threads inside a subwindow

 

When i tested it on a higher spec pc it run it very easily

 

But yes I agree - copying each sub into each thread command will certainly be a nightmare for large bots -

 

To make this work on your platform bots I think the whole bot would need to be handled like on big sub

[Abs* 12]

hi zap - yes I agree I need to understand how looping through list totals are handled if you are multithreading -

 

I did try to do it 2 ways -

 

the first was to have a loop list total in the scrape the serps sub - I then created 3 threads all with the scrape the serps run sub in them - then finally left a run scrape the serps sub without a thread -

 

I think it might have automatically split the list total amongst all 4 browsers - but i wasnt sure so tried another way too -

 

Not sure if i explained clear enough how the subs would wait for each other - all sub windows and the application browser would navigate to a search engine page - then scrape the contents - and then all move on together even though after a few times you could see that some sub windows were loading the page faster then others -

 

I guess i will need to test using a bunch of different urls and then create a loop list total-

 

with the platform bot having one sub for the signup then this would make the code very messy and im not sure if running sub within subs a number of times would cause issues on the script - Each section has it own sub and sometimes i need to recall subs so if i was to create one sub i would need to duplicate so much coding -

 

would love to see some multithread bots that others have made

 

thanks

 

abs

[Pete 121]

abs give this a try exe code As my pc would not support more the 3 threads I never did anymore work on this, but in a week or two I should have a new pc

So I may well continual to work on it then

[Abs* 12]

excellent example Zap - I dont think i would ever have thought of the thread control you have created -

 

Im still not sure i fully understand it - just to confirm from what i have understood is that you are cross examining the threads with the thread counter -

 

so if the thread counter and number of threads = the same then it will keep it in the delay 1 sec which will keep continuing and stop more threads from opening ? if it is not the same then it will again loop through and open another thread, - once the sub is complete then the threads and counter will not be the same so it will come out of the delay and let the thread through till they equal the same

 

I think this is a great way to control them - very well thought out -

 

And yes it worked great - all of them -one issue i can see from this is the popup windows opening and closing - so if you was to use a threaded bot then i dont think you will be able to use your computer for anything else as the windows will keep popping up in your face

 

Thanks for clearing things up - Will make it much more fun to play with now -

 

cheers

[Pete 121]

so if the thread counter and number of threads = the same then it will keep it in the delay 1 sec

its if the number of threads = > Greater than x amount

 

-one issue i can see from this is the popup windows opening and closing - so if you was to use a threaded bot then i dont think you will be able to use your computer for anything else as the windows will keep popping up in your face

Yes that’s is a issue unless seth could add a run sub window minimized option

 

Glad it helped you anyway

[Abs* 12]

its if the number of threads = > Greater than x amount

 

 

Yes that’s is a issue unless seth could add a run sub window minimized option

 

Glad it helped you anyway

 

yeh thanks Zap

 

i think the run thread minimized is a good idea or have a split browser similar to how senuke do

 

thanks

[Gogetta 263]

yeh thanks Zap

 

i think the run thread minimized is a good idea or have a split browser similar to how senuke do

 

thanks

 

I think both split browser and the option to minimize is a great idea. It gets to messy having sub windows or IE browser opening up for every thread. It actually interrupts my other work at times.

[UBotBuddy 331]

Thanks for the shared code Zap. +1 sent your way!

 

I have never played much with this feature so this is helpful for mw.

[Abs* 12]

Hi guys - ok been thinking of a way we can keep the sub windows open and have them loop through list totals - this way windows wouldnt popup all over the place -

 

Ive been working with soem java maths commands and thought it would really help in splitting up lists for each thread

 

I have attached the bot below and hope one of you guys can see what im trying to do -

 

Basically the issues of duplicating threads would mean having to duplicate all of the run subs inside of the threads - which im sure we all know will just mean larger bots, something unfortunatly Ubot currently has an issue with -

 

So what ive tried to do is to allow the user to be able to choose the number of threads - then use the maths eval command to divide the number of urls in the list by the number of threads that the user chooses - I do this using the ceiling command to get the highest value needed -

 

I then have a loop placed - which will loop the number entered in the threads field - within that loop there is another loop which will loop the number that the maths command outputs - in this loop it adds the next list item to a list and then saves it to a file outside of the 2nd loop but still in the first and increments the end value - so list1.txt - I have set a variable to 1 and then i have it increment at the end of the loop -

 

What this does is divide the urls loaded into the number of threads required allowing each thread to run through its own list -

 

After the urls are sorted the thread command starts - the way i have it coded is to set a loop for the number of threads chosen - in that loop it adds file xxx{1}.txt ({1} being the incremented value) to a list - threads the sub - increments the number

 

I havnt fully tested but have added the code - hopeing you guys can see how we could eliminate the need to code each thread over and over again if it was possible to add incremented values to lists -

 

so heres the problem -

 

I am using one list - I clear the list each time it opens a new thread - and even though the file sorting is correct before runing the threads they all loop through the list -

 

Not sure if we have a remove from list command and once it navigates to that url it will delete and then the not revisit in another sub -

 

thanks

 

Man - I really need to contact Lilly and have her upgrade my account so i can add more attachments

 

please download from Thread-Test-Abs

 

thanks

[Abs* 12]

Duplicate -