Bob The Builder 62 Posted December 3, 2011 Report Share Posted December 3, 2011 Multi treading in v4 is very buggy and problematic right now if you use more than 5 or so threads. I spent A LOT of time trying to make it work, but still waiting for Eddie to come up with a fix. Quote Link to post Share on other sites
dsc76 6 Posted December 3, 2011 Author Report Share Posted December 3, 2011 Totally stumped and getting frustrated. Here is the most simplified version of the bot I am trying to multithread. Please help. multithreadbot.ubotusernames.txt Quote Link to post Share on other sites
Bob The Builder 62 Posted December 3, 2011 Report Share Posted December 3, 2011 Totally stumped and getting frustrated. Here is the most simplified version of the bot I am trying to multithread. Please help. multithreadbot.ubotusernames.txt Join the club been trying to do it for months now, need to wait for it to be fixed. Quote Link to post Share on other sites
seonid 4 Posted December 3, 2011 Report Share Posted December 3, 2011 I am looking forward to be able to find a fix to this error too. Quote Link to post Share on other sites
seonid 4 Posted December 4, 2011 Report Share Posted December 4, 2011 I've made a few tests to find out what is causing the crash. I've used a new bot and instead of usernames I've used a linklist. I wanted to make sure that each thread will load separately a new url, and they won't overlap with the same data. This is the bot i've tested: ui text box("threads", #threads) ui open file("Upload links : ", #linkstovisit) clear list(%results) divider divider clear list(%linkstovisit) add list to list(%linkstovisit, $list from file(#linkstovisit), "Delete", "Global") set(#linksnr, $list total(%linkstovisit), "Global") clear table(&visited) ui stat monitor("Position :", #position) divider set(#position, 0, "Global") set(#currentlink, $list item(%linkstovisit, #position), "Local") loop(#threads) { thread { in new browser { increment(#position) set(#currenturl, $next list item(%linkstovisit), "Local") set table cell(&visited, #position, 2, #position) set table cell(&visited, #position, 0, #currenturl) navigate(#currenturl, "Wait") wait for browser event("Everything Loaded", "") wait(3) } save to file("{$special folder("My Documents")}\\results.csv", &visited) } } I've used add to table cell function to have a feedback on the bot status. The first error I wanted to avoid is that I didn't wanted it to load the same value in different threads. If I launch manually 5 simultaneous threads it works nice and smooth, the problem is with "loop while" statement because if I include all in loop while function will open too many browsers and the system will crash. Quote Link to post Share on other sites
seonid 4 Posted December 4, 2011 Report Share Posted December 4, 2011 Totally stumped and getting frustrated. Here is the most simplified version of the bot I am trying to multithread. Please help. multithreadbot.ubotusernames.txthttp://nid.ro/bot/solved.jpgSOLVED ui text box("threads", #threads) divider divider clear list(%usernames) add list to list(%usernames, $list from file("{$special folder("My Documents")}\\usernames.txt"), "Delete", "Global") set(#listotal, $list total(%usernames), "Global") divider define multithreader { set(#position, 0, "Global") loop(#threads) { thread { loop while($comparison(#position, "<", $list total(%usernames))) { set(#myusername, $next list item(%usernames), "Local") in new browser { increment(#position) navigate("http://ubotstudio.com/playground/simpleform.aspx", "Wait") type text(<about me textarea>, #myusername, "Standard") wait(8) click(<type="button">, "Left Click", "No") wait for browser event("Everything Loaded", "") } } } } } multithreader() multithreadOK.ubot Quote Link to post Share on other sites
dsc76 6 Posted December 4, 2011 Author Report Share Posted December 4, 2011 Hi Seonid, Thank you for the feedback! I know the troubleshooting must have taken you some time, but I still do not see a solution to my orgional problem. 1) When I run the bot multithreadOK.ubot the same value (list item) appear in each new window. 2) There is no funcionalty that will allow me to cycle through the list more then once. The bot from post #27 ( reattached below) was the exact bot I am trying to use with all the functionalty I want except for the multithread and ui to control it. The ui labeled "cycles" in the bot attached below is the number of times I want to cycle though the list not the amount of threads. multithreadbot.ubotusernames.txt Just to clarify - Is the multithread functionality on ubot not working or am I implementing it incorrectly or both? I am getting mixed feedback. I had no idea this would be so problematic. Thanks again all. -Eric Quote Link to post Share on other sites
Bob The Builder 62 Posted December 4, 2011 Report Share Posted December 4, 2011 Hi Seonid, Thank you for the feedback! I know the troubleshooting must have taken you some time, but I still do not see a solution to my orgional problem. 1) When I run the bot multithreadOK.ubot the same value (list item) appear in each new window. 2) There is no funcionalty that will allow me to cycle through the list more then once. The bot from post #27 ( reattached below) was the exact bot I am trying to use with all the functionalty I want except for the multithread and ui to control it. The ui labeled "cycles" in the bot attached below is the number of times I want to cycle though the list not the amount of threads. multithreadbot.ubotusernames.txt Just to clarify - Is the multithread functionality on ubot not working or am I implementing it incorrectly or both? I am getting mixed feedback. I had no idea this would be so problematic. Thanks again all. -Eric As far as I have seen and I spent many many hours wasted on this over 2 months of trying, multi-threading does not work reliably in ubot. Not when using in new browser, they lock up constantly. I have managed to do multiple threads using 5 threads but after that it locks browsers left and right. I've tried working with Eddie on it many times but it is still doing it and there is no fix currently. I believe Eddie found an in new browser bug but not sure if it is related to this. Quote Link to post Share on other sites
seonid 4 Posted December 5, 2011 Report Share Posted December 5, 2011 As far as I have seen and I spent many many hours wasted on this over 2 months of trying, multi-threading does not work reliably in ubot. Not when using in new browser, they lock up constantly. I have managed to do multiple threads using 5 threads but after that it locks browsers left and right. I've tried working with Eddie on it many times but it is still doing it and there is no fix currently. I believe Eddie found an in new browser bug but not sure if it is related to this. I understand your frustration Bob, but this problem has a solution, and if multithreading works using 5 threads maybe we should use it that way until it gets fixed. I don't think that helps anyone a negative approach. Quote Link to post Share on other sites
seonid 4 Posted December 5, 2011 Report Share Posted December 5, 2011 Hi Seonid, Thank you for the feedback! I know the troubleshooting must have taken you some time, but I still do not see a solution to my orgional problem. 1) When I run the bot multithreadOK.ubot the same value (list item) appear in each new window. 2) There is no funcionalty that will allow me to cycle through the list more then once. The bot from post #27 ( reattached below) was the exact bot I am trying to use with all the functionalty I want except for the multithread and ui to control it. The ui labeled "cycles" in the bot attached below is the number of times I want to cycle though the list not the amount of threads. multithreadbot.ubotusernames.txt Just to clarify - Is the multithread functionality on ubot not working or am I implementing it incorrectly or both? I am getting mixed feedback. I had no idea this would be so problematic. Thanks again all. -Eric I've checked again and I've inserted a step indexer to see if it loads the same usernames in every thread. By step indexer I mean an add to list function of the username in use in the new browser window under the parallel thread. http://nid.ro/bot/debuggershowlist.jpg multithreadOKwithcheck.ubot Quote Link to post Share on other sites
dsc76 6 Posted December 5, 2011 Author Report Share Posted December 5, 2011 Sorry Seonid, I see the same issue as before. When I click on the small new browser windows the same list item is being used in all threads at the same time not a different nextlistitem in each window. Also, functionality to cycle through the list more then once is not included in your test bot so who knows what additional problems that will cause. Perhaps this is bug referred to in previous posts. Add me to the list of people who are waiting for a fix. This should be a simple command that asks "how many threads" and accepts a variable. I suppose I'll wait to see what happens with future updates. Thank you for all the help everyone! -Eric Quote Link to post Share on other sites
JohnB 255 Posted December 5, 2011 Report Share Posted December 5, 2011 Have any of you tried setting a local variable to $next list item (i.e. #myusername=$nextlistitem) and then loading that variable into the form field? It would go inside the while node but before the in new browser node. John Quote Link to post Share on other sites
dsc76 6 Posted December 5, 2011 Author Report Share Posted December 5, 2011 I think Seonid tried something to that effect in the bot from post #31 and #35. Quote Link to post Share on other sites
JohnB 255 Posted December 5, 2011 Report Share Posted December 5, 2011 Ok, just trying to cover as many bases as I can think of... John Quote Link to post Share on other sites
seonid 4 Posted December 5, 2011 Report Share Posted December 5, 2011 Sorry Seonid, I see the same issue as before. When I click on the small new browser windows the same list item is being used in all threads at the same time not a different nextlistitem in each window. Also, functionality to cycle through the list more then once is not included in your test bot so who knows what additional problems that will cause. Perhaps this is bug referred to in previous posts. Add me to the list of people who are waiting for a fix. This should be a simple command that asks "how many threads" and accepts a variable. I suppose I'll wait to see what happens with future updates. Thank you for all the help everyone! -Eric Eric, The information displayed in the right corner is wrong, the bot is working well. I've tryed several times, on other websites also, i've attached a bot with save to a file function to create a different text file named with the variable #username ( so you can check if there are duplicates ) I am really sorry but, I am curious if you really have tested the bot. And please people please do some debugging before posting that it's not working. Only watching the little windows it isn't really debugging and it isn't helping any of us. There seem to be a bug with the display windows - the information displayed is incorrect when I click on it. Regards,multithreadwithwritefile.ubot Quote Link to post Share on other sites
dsc76 6 Posted December 6, 2011 Author Report Share Posted December 6, 2011 Eric, The information displayed in the right corner is wrong, the bot is working well. I've tryed several times, on other websites also, i've attached a bot with save to a file function to create a different text file named with the variable #username ( so you can check if there are duplicates ) I am really sorry but, I am curious if you really have tested the bot. And please people please do some debugging before posting that it's not working. Only watching the little windows it isn't really debugging and it isn't helping any of us. There seem to be a bug with the display windows - the information displayed is incorrect when I click on it. Regards, To satisfy your curiosity, yes, I tested the bot to the best of my abilities and searched for a solution for several hours before posting the thread. You have to respect the fact that not all of us are expert level ubot programmers. Myself and others who are still learning depend on the forum to answer questions, and not every thread will help the community as a whole. With that being said I appreciate your assistance and the time it took you to make the example bots. You demonstrated the value of the debugger to me and I will have to track down a tutorial how to use it properly, as admittedly it was not part of my troubleshooting routine. I suppose up until now the complexity of my bots did not demand its use. The fact that the small windows are showing the wrong information is unacceptable. I depend on the gui being accurate and it is one of the main reasons ubot is so valuable to me. Without the gui, or without it being accurate, I might as well start coding by hand which is a practice that is well beyond my knowledge base or skill level. Please consider this matter resolved or at least on hold until the bugs are fixed. I cannot justify trying to implement or troubleshoot functionally that is not working correctly. Thanks again.-Eric Quote Link to post Share on other sites
Bob The Builder 62 Posted December 6, 2011 Report Share Posted December 6, 2011 I understand your frustration Bob, but this problem has a solution, and if multithreading works using 5 threads maybe we should use it that way until it gets fixed. I don't think that helps anyone a negative approach. Negative approach? I spent countless hours and been waiting two months now and it still does not work. Silence never fixes anything and without stable multi-threading and stable compiling I can't even use the product for what I bought it for. It is not a negative approach, it is just a fact. Quote Link to post Share on other sites
Seth Turin 223 Posted December 6, 2011 Report Share Posted December 6, 2011 Hey guys Just wanted to let you know that we're looking at the issue. I'm talking to Ed about it now to probe deeper into what's going on. He will be posting soon with more information. Quote Link to post Share on other sites
Eddie Waller 158 Posted December 7, 2011 Report Share Posted December 7, 2011 Hello, I am trying to make a multithreaded bot that will also cycle through a list multiple times. For example I have a list of 10 accounts in a .txt file and would like to use 5 threads to login to each account a total of 3 times…in other words cycling through the list total 3 times. When I run the bot attached it seems to open windows totaling the number of threads multiplied by the number of cycles all at one time which is not what I intended. multi thread test.ubot Any feedback is appreciated. Thanks. Hi dsc76, It's a little hard to follow this forum thread at this point, but I can try to address your initial issue. We are planning a new release for the end of this week, and after that I will see what I can do to make limiting the number of threads easier to do . Something to keep in mind is that threading is an advanced topic, and even advanced programmers can struggle with it. I apologize in advance that this runs a little long, I'm doing my best to explain this step by step. If you are just interested in looking at the answer, you can scroll to the end. If I understand correctly, you have X accounts and want to log in to each of them Y times using a maximum of Z threads. Your example you used: X = 10 accountsY = 3 timesZ = 5 threads What the code you posted in your initial post does, is loop Y * Z times opening 15 threads at once. You have no limits to how many threads are allowed to be open at once. You simply loop 3 times, and in each of those loops, you loop 5 times, and in each of those you open a new thread. One thing to keep in mind is that when you run a thread command, the contents of the thread (typing in about me) are told to run separately from the main script. The main script does not wait for the thread to finish before it continues with the main script. That is why it opens all 15 of them at once, without waiting for any of them to finish before opening more. So there are a few things we need to do to correct these issues. Here is the code view of the code you have posted: ui text box("cycles", #cycles) ui text box("threads", #threads) divider ui stat monitor("counter", #counter) divider set(#counter, 0, "Global") divider clear list(%usernames) add list to list(%usernames, $list from file("{$special folder("Application")}\\usernames.txt"), "Delete", "Global") loop(#cycles) { loop(#threads) { thread { in new browser { navigate("http://ubotstudio.com/playground/simpleform.aspx", "Wait") type text(<about me textarea>, $next list item(%usernames), "Standard") wait(8) click(<type="button">, "Left Click", "No") increment(#counter) wait for browser event("Everything Loaded", "") } } } } To keep the maximum threads open limited to (#threads) we are going to need a new variable #threadsOpen, that we can set to 0 to start. set(#threadsOpen, 0, "Global") To keep the username for each thread unique to the thread, so that no thread is reusing the username, we can pass the username into a custom command, and put the about me code we run into a custom command. define about me(#username) { thread { in new browser { navigate("http://ubotstudio.com/playground/simpleform.aspx", "Wait") type text(<about me textarea>, #username, "Standard") wait(8) click(<type="button">, "Left Click", "No") increment(#counter) wait for browser event("Everything Loaded", "") } } } So now our program looks like this (ignoring the ui boxes for simplicity): define about me(#username) { thread { in new browser { navigate("http://ubotstudio.com/playground/simpleform.aspx", "Wait") type text(<about me textarea>, #username, "Standard") wait(8) click(<type="button">, "Left Click", "No") increment(#counter) wait for browser event("Everything Loaded", "") } } } set(#threadsOpen, 0, "Global") clear list(%usernames) add list to list(%usernames, $list from file("{$special folder("Application")}\\usernames.txt"), "Delete", "Global") loop(#cycles) { loop(#threads) { about me($next list item(%usernames)) } } We're still missing a limit on the number of threads that can run at once, which we want to do based on the #threadsOpen variable. What we can do is before calling the about me command, we can add a while loop: loop while(#threadsOpen >= #threads) { wait(1) } This means that while the maximum amount of threads or more are open, we should just wait until they aren't anymore. Right now, #threadsOpen is always 0 though, so where do we increase and decrease it? One idea might be to increment(#threadsOpen) at the beginning of about me, and decrement it at the end of about me: define about me(#username) { increment(#threadsOpen) thread { in new browser { navigate("http://ubotstudio.com/playground/simpleform.aspx", "Wait") type text(<about me textarea>, #username, "Standard") wait(8) click(<type="button">, "Left Click", "No") increment(#counter) wait for browser event("Everything Loaded", "") } } decrement(#threadsOpen) } The problem with this is that it will increment #threadsOpen to 1, and then start the thread, and then decrement #threadsOpen back down to 0. This happens because when we run the thread command, we do not wait for it to finish, we run it at the same time as our main script. Since #threadsOpen would always be 0, the limiting would not work. Another idea would be to put the increment and decrement inside the thread command at the start and end: define about me(#username) { thread { increment(#threadsOpen) in new browser { navigate("http://ubotstudio.com/playground/simpleform.aspx", "Wait") type text(<about me textarea>, #username, "Standard") wait(8) click(<type="button">, "Left Click", "No") increment(#counter) wait for browser event("Everything Loaded", "") } decrement(#threadsOpen) } } This would also be incorrect, since when we call the about me command and run the thread command, the nodes inside the thread command (including the increment) do not run immediately. So it would be possible that we call the thread command 15 times before any of the threads actually start up and increment the #threadsOpen variable, which would lead to all 15 threads running at once. The correct way to do this is to increment #threadsOpen at the beginning of about me, and decrement #threadsOpen at the end of the thread command: define about me(#username) { increment(#threadsOpen) thread { in new browser { navigate("http://ubotstudio.com/playground/simpleform.aspx", "Wait") type text(<about me textarea>, #username, "Standard") wait(8) click(<type="button">, "Left Click", "No") increment(#counter) wait for browser event("Everything Loaded", "") } decrement(#threadsOpen) } } This works because we increment #threadsOpen before starting a thread, and we decrement #threadsOpen when the thread is finished. Putting this all together it ends up looking like this: Note: I also changed to looping of the $list total instead of looping the threads, because we want to loop through each account (X) for (Y) times. This means we shouldn't be looping based on Z (#threads). ui text box("cycles", #cycles) ui text box("threads", #threads) divider ui stat monitor("counter", #counter) set(#counter, 0, "Global") define about me(#username) { increment(#threadsOpen) thread { in new browser { navigate("http://ubotstudio.com/playground/simpleform.aspx", "Wait") type text(<about me textarea>, #username, "Standard") wait(8) click(<type="button">, "Left Click", "No") increment(#counter) wait for browser event("Everything Loaded", "") } decrement(#threadsOpen) } } divider set(#threadsOpen, 0, "Global") clear list(%usernames) add list to list(%usernames, $list from file("{$special folder("Application")}\\usernames.txt"), "Delete", "Global") loop(#cycles) { loop($list total(%usernames)) { loop while(#threadsOpen >= #threads) { wait(1) } about me($next list item(%usernames)) } set list position(%usernames, 0) } Here's a video of it in action with 5 threads, running 3 times on 10 accounts: http://screencast.com/t/DcQoPlUweTkx I've also attached the bot that I am using. Hope this helps clear things up a little bit, and I'm sorry if I lost you at any point. If you have future questions just let me know . Threading Example - Eddie.ubot 9 Quote Link to post Share on other sites
dsc76 6 Posted December 7, 2011 Author Report Share Posted December 7, 2011 Seth ,Thank you for the tutorial and the functioning sctipt. It works absoultly perfect!TYVM everyone +1-Eric Quote Link to post Share on other sites
matflawless 7 Posted January 28, 2012 Report Share Posted January 28, 2012 Thanks a lot! Quote Link to post Share on other sites
southsider 0 Posted March 8, 2012 Report Share Posted March 8, 2012 Thanks for posting The loopwhile ThreadsOpen was my missing link that I couldn't figure out Quote Link to post Share on other sites
Proven 1 Posted December 7, 2012 Report Share Posted December 7, 2012 Seems the attachment got lost - anybody got that download I could borrow? Threading Example - Eddie.ubot Thx Quote Link to post Share on other sites
orbital 15 Posted December 15, 2012 Report Share Posted December 15, 2012 I wanted to add this screencast that TJ made for me to fix my bot. I was trying to increment a #row in a table for account info. TJ does it in a much more elegant way, a way that works . http://screencast.com/t/cO20EHysZx I haven't seen this method any where else on this forum when i was searching for a solution. Thanks again TJ Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.