Jump to content
UBot Underground

Multithread bot question.


Recommended Posts

Totally stumped and getting frustrated. Here is the most simplified version of the bot I am trying to multithread. Please help.

 

multithreadbot.ubot

usernames.txt

 

Join the club been trying to do it for months now, need to wait for it to be fixed.

Link to post
Share on other sites

I've made a few tests to find out what is causing the crash.

 

I've used a new bot and instead of usernames I've used a linklist. I wanted to make sure that each thread will load separately a new url, and they won't overlap with the same data.

 

This is the bot i've tested:

 

ui text box("threads", #threads)
ui open file("Upload links : ", #linkstovisit)
clear list(%results)
divider
divider
clear list(%linkstovisit)
add list to list(%linkstovisit, $list from file(#linkstovisit), "Delete", "Global")
set(#linksnr, $list total(%linkstovisit), "Global")
clear table(&visited)
ui stat monitor("Position :", #position)
divider
set(#position, 0, "Global")
set(#currentlink, $list item(%linkstovisit, #position), "Local")
loop(#threads) {
   thread {
       in new browser {
           increment(#position)
           set(#currenturl, $next list item(%linkstovisit), "Local")
           set table cell(&visited, #position, 2, #position)
           set table cell(&visited, #position, 0, #currenturl)
           navigate(#currenturl, "Wait")
           wait for browser event("Everything Loaded", "")
           wait(3)
       }
       save to file("{$special folder("My Documents")}\\results.csv", &visited)
   }
}

 

I've used add to table cell function to have a feedback on the bot status.

 

The first error I wanted to avoid is that I didn't wanted it to load the same value in different threads.

 

If I launch manually 5 simultaneous threads it works nice and smooth, the problem is with "loop while" statement because if I include all in loop while function will open too many browsers and the system will crash.

Link to post
Share on other sites

Totally stumped and getting frustrated. Here is the most simplified version of the bot I am trying to multithread. Please help.

 

multithreadbot.ubot

usernames.txt

http://nid.ro/bot/solved.jpg

SOLVED

 

ui text box("threads", #threads)
divider
divider
clear list(%usernames)
add list to list(%usernames, $list from file("{$special folder("My Documents")}\\usernames.txt"), "Delete", "Global")
set(#listotal, $list total(%usernames), "Global")
divider
define multithreader {
   set(#position, 0, "Global")
   loop(#threads) {
       thread {
           loop while($comparison(#position, "<", $list total(%usernames))) {
               set(#myusername, $next list item(%usernames), "Local")
               in new browser {
                   increment(#position)
                   navigate("http://ubotstudio.com/playground/simpleform.aspx", "Wait")
                   type text(<about me textarea>, #myusername, "Standard")
                   wait(8)
                   click(<type="button">, "Left Click", "No")
                   wait for browser event("Everything Loaded", "")
               }
           }
       }
   }
}
multithreader()

multithreadOK.ubot

Link to post
Share on other sites

Hi Seonid,

 

Thank you for the feedback!

 

I know the troubleshooting must have taken you some time, but I still do not see a solution to my orgional problem.

 

1) When I run the bot multithreadOK.ubot the same value (list item) appear in each new window.

2) There is no funcionalty that will allow me to cycle through the list more then once.

 

The bot from post #27 ( reattached below) was the exact bot I am trying to use with all the functionalty I want except for the multithread and ui to control it. The ui labeled "cycles" in the bot attached below is the number of times I want to cycle though the list not the amount of threads.

 

multithreadbot.ubot

usernames.txt

 

Just to clarify - Is the multithread functionality on ubot not working or am I implementing it incorrectly or both? I am getting mixed feedback.

 

I had no idea this would be so problematic.

 

Thanks again all.

 

-Eric

Link to post
Share on other sites
Hi Seonid,

 

Thank you for the feedback!

 

I know the troubleshooting must have taken you some time, but I still do not see a solution to my orgional problem.

 

1) When I run the bot multithreadOK.ubot the same value (list item) appear in each new window.

2) There is no funcionalty that will allow me to cycle through the list more then once.

 

The bot from post #27 ( reattached below) was the exact bot I am trying to use with all the functionalty I want except for the multithread and ui to control it. The ui labeled "cycles" in the bot attached below is the number of times I want to cycle though the list not the amount of threads.

 

multithreadbot.ubot

usernames.txt

 

Just to clarify - Is the multithread functionality on ubot not working or am I implementing it incorrectly or both? I am getting mixed feedback.

 

I had no idea this would be so problematic.

 

Thanks again all.

 

-Eric

 

As far as I have seen and I spent many many hours wasted on this over 2 months of trying, multi-threading does not work reliably in ubot. Not when using in new browser, they lock up constantly. I have managed to do multiple threads using 5 threads but after that it locks browsers left and right.

 

I've tried working with Eddie on it many times but it is still doing it and there is no fix currently. I believe Eddie found an in new browser bug but not sure if it is related to this.

Link to post
Share on other sites

As far as I have seen and I spent many many hours wasted on this over 2 months of trying, multi-threading does not work reliably in ubot. Not when using in new browser, they lock up constantly. I have managed to do multiple threads using 5 threads but after that it locks browsers left and right.

 

I've tried working with Eddie on it many times but it is still doing it and there is no fix currently. I believe Eddie found an in new browser bug but not sure if it is related to this.

 

I understand your frustration Bob, but this problem has a solution, and if multithreading works using 5 threads maybe we should use it that way until it gets fixed.

 

I don't think that helps anyone a negative approach.

Link to post
Share on other sites

Hi Seonid,

 

Thank you for the feedback!

 

I know the troubleshooting must have taken you some time, but I still do not see a solution to my orgional problem.

 

1) When I run the bot multithreadOK.ubot the same value (list item) appear in each new window.

2) There is no funcionalty that will allow me to cycle through the list more then once.

 

The bot from post #27 ( reattached below) was the exact bot I am trying to use with all the functionalty I want except for the multithread and ui to control it. The ui labeled "cycles" in the bot attached below is the number of times I want to cycle though the list not the amount of threads.

 

multithreadbot.ubot

usernames.txt

 

Just to clarify - Is the multithread functionality on ubot not working or am I implementing it incorrectly or both? I am getting mixed feedback.

 

I had no idea this would be so problematic.

 

Thanks again all.

 

-Eric

 

 

I've checked again and I've inserted a step indexer to see if it loads the same usernames in every thread.

 

By step indexer I mean an add to list function of the username in use in the new browser window under the parallel thread.

 

http://nid.ro/bot/debuggershowlist.jpg

 

multithreadOKwithcheck.ubot

Link to post
Share on other sites

Sorry Seonid,

 

I see the same issue as before.

 

When I click on the small new browser windows the same list item is being used in all threads at the same time not a different nextlistitem in each window. Also, functionality to cycle through the list more then once is not included in your test bot so who knows what additional problems that will cause. Perhaps this is bug referred to in previous posts.

 

Add me to the list of people who are waiting for a fix. This should be a simple command that asks "how many threads" and accepts a variable. I suppose I'll wait to see what happens with future updates.

 

Thank you for all the help everyone!

 

-Eric

Link to post
Share on other sites

Have any of you tried setting a local variable to $next list item (i.e. #myusername=$nextlistitem) and then loading that variable into the form field? It would go inside the while node but before the in new browser node.

 

John

Link to post
Share on other sites

Sorry Seonid,

 

I see the same issue as before.

 

When I click on the small new browser windows the same list item is being used in all threads at the same time not a different nextlistitem in each window. Also, functionality to cycle through the list more then once is not included in your test bot so who knows what additional problems that will cause. Perhaps this is bug referred to in previous posts.

 

Add me to the list of people who are waiting for a fix. This should be a simple command that asks "how many threads" and accepts a variable. I suppose I'll wait to see what happens with future updates.

 

Thank you for all the help everyone!

 

-Eric

 

 

Eric,

 

The information displayed in the right corner is wrong, the bot is working well. I've tryed several times, on other websites also, i've attached a bot with save to a file function to create a different text file named with the variable #username ( so you can check if there are duplicates )

 

I am really sorry but, I am curious if you really have tested the bot. And please people please do some debugging before posting that it's not working. Only watching the little windows it isn't really debugging and it isn't helping any of us.

 

There seem to be a bug with the display windows - the information displayed is incorrect when I click on it.

 

Regards,

multithreadwithwritefile.ubot

Link to post
Share on other sites

Eric,

 

The information displayed in the right corner is wrong, the bot is working well. I've tryed several times, on other websites also, i've attached a bot with save to a file function to create a different text file named with the variable #username ( so you can check if there are duplicates )

 

I am really sorry but, I am curious if you really have tested the bot. And please people please do some debugging before posting that it's not working. Only watching the little windows it isn't really debugging and it isn't helping any of us.

 

There seem to be a bug with the display windows - the information displayed is incorrect when I click on it.

 

Regards,

 

 

To satisfy your curiosity, yes, I tested the bot to the best of my abilities and searched for a solution for several hours before posting the thread. You have to respect the fact that not all of us are expert level ubot programmers. Myself and others who are still learning depend on the forum to answer questions, and not every thread will help the community as a whole.

 

With that being said I appreciate your assistance and the time it took you to make the example bots.

 

You demonstrated the value of the debugger to me and I will have to track down a tutorial how to use it properly, as admittedly it was not part of my troubleshooting routine. I suppose up until now the complexity of my bots did not demand its use.

 

The fact that the small windows are showing the wrong information is unacceptable.

 

I depend on the gui being accurate and it is one of the main reasons ubot is so valuable to me. Without the gui, or without it being accurate, I might as well start coding by hand which is a practice that is well beyond my knowledge base or skill level.

 

Please consider this matter resolved or at least on hold until the bugs are fixed. I cannot justify trying to implement or troubleshoot functionally that is not working correctly.

 

Thanks again.

-Eric

Link to post
Share on other sites

 

I understand your frustration Bob, but this problem has a solution, and if multithreading works using 5 threads maybe we should use it that way until it gets fixed.

 

I don't think that helps anyone a negative approach.

 

Negative approach? I spent countless hours and been waiting two months now and it still does not work. Silence never fixes anything and without stable multi-threading and stable compiling I can't even use the product for what I bought it for.

 

It is not a negative approach, it is just a fact.

Link to post
Share on other sites

Hello,

 

I am trying to make a multithreaded bot that will also cycle through a list multiple times. For example I have a list of 10 accounts in a .txt file and would like to use 5 threads to login to each account a total of 3 times…in other words cycling through the list total 3 times. When I run the bot attached it seems to open windows totaling the number of threads multiplied by the number of cycles all at one time which is not what I intended.

 

multi thread test.ubot

 

Any feedback is appreciated.

 

Thanks.

 

Hi dsc76,

 

It's a little hard to follow this forum thread at this point, but I can try to address your initial issue.

 

We are planning a new release for the end of this week, and after that I will see what I can do to make limiting the number of threads easier to do :).

 

Something to keep in mind is that threading is an advanced topic, and even advanced programmers can struggle with it. I apologize in advance that this runs a little long, I'm doing my best to explain this step by step. If you are just interested in looking at the answer, you can scroll to the end.

 

If I understand correctly, you have X accounts and want to log in to each of them Y times using a maximum of Z threads.

 

Your example you used:

 

X = 10 accounts

Y = 3 times

Z = 5 threads

 

What the code you posted in your initial post does, is loop Y * Z times opening 15 threads at once. You have no limits to how many threads are allowed to be open at once. You simply loop 3 times, and in each of those loops, you loop 5 times, and in each of those you open a new thread.

 

One thing to keep in mind is that when you run a thread command, the contents of the thread (typing in about me) are told to run separately from the main script. The main script does not wait for the thread to finish before it continues with the main script. That is why it opens all 15 of them at once, without waiting for any of them to finish before opening more.

 

So there are a few things we need to do to correct these issues.

 

Here is the code view of the code you have posted:

 

ui text box("cycles", #cycles)
ui text box("threads", #threads)
divider
ui stat monitor("counter", #counter)
divider
set(#counter, 0, "Global")
divider
clear list(%usernames)
add list to list(%usernames, $list from file("{$special folder("Application")}\\usernames.txt"), "Delete", "Global")
loop(#cycles) {
   loop(#threads) {
       thread {
           in new browser {
               navigate("http://ubotstudio.com/playground/simpleform.aspx", "Wait")
               type text(<about me textarea>, $next list item(%usernames), "Standard")
               wait(8)
               click(<type="button">, "Left Click", "No")
               increment(#counter)
               wait for browser event("Everything Loaded", "")
           }
       }
   }
}

 

To keep the maximum threads open limited to (#threads) we are going to need a new variable #threadsOpen, that we can set to 0 to start.

 

set(#threadsOpen, 0, "Global")

 

To keep the username for each thread unique to the thread, so that no thread is reusing the username, we can pass the username into a custom command, and put the about me code we run into a custom command.

 

define about me(#username) {
   thread {
       in new browser {
           navigate("http://ubotstudio.com/playground/simpleform.aspx", "Wait")
           type text(<about me textarea>, #username, "Standard")
           wait(8)
           click(<type="button">, "Left Click", "No")
           increment(#counter)
           wait for browser event("Everything Loaded", "")
       }
   }
}

 

So now our program looks like this (ignoring the ui boxes for simplicity):

 

define about me(#username) {
   thread {
       in new browser {
           navigate("http://ubotstudio.com/playground/simpleform.aspx", "Wait")
           type text(<about me textarea>, #username, "Standard")
           wait(8)
           click(<type="button">, "Left Click", "No")
           increment(#counter)
           wait for browser event("Everything Loaded", "")
       }
   }
}

set(#threadsOpen, 0, "Global")
clear list(%usernames)
add list to list(%usernames, $list from file("{$special folder("Application")}\\usernames.txt"), "Delete", "Global")
loop(#cycles) {
   loop(#threads) {
       about me($next list item(%usernames))
   }
}

 

We're still missing a limit on the number of threads that can run at once, which we want to do based on the #threadsOpen variable. What we can do is before calling the about me command, we can add a while loop:

 

loop while(#threadsOpen >= #threads) {
   wait(1)
}

 

This means that while the maximum amount of threads or more are open, we should just wait until they aren't anymore.

 

Right now, #threadsOpen is always 0 though, so where do we increase and decrease it? One idea might be to increment(#threadsOpen) at the beginning of about me, and decrement it at the end of about me:

 

define about me(#username) {
   increment(#threadsOpen)
   thread {
       in new browser {
           navigate("http://ubotstudio.com/playground/simpleform.aspx", "Wait")
           type text(<about me textarea>, #username, "Standard")
           wait(8)
           click(<type="button">, "Left Click", "No")
           increment(#counter)
           wait for browser event("Everything Loaded", "")
       }
   }
   decrement(#threadsOpen)
}

 

The problem with this is that it will increment #threadsOpen to 1, and then start the thread, and then decrement #threadsOpen back down to 0. This happens because when we run the thread command, we do not wait for it to finish, we run it at the same time as our main script. Since #threadsOpen would always be 0, the limiting would not work.

 

Another idea would be to put the increment and decrement inside the thread command at the start and end:

 

define about me(#username) {
 thread {
     increment(#threadsOpen)
     in new browser {
         navigate("http://ubotstudio.com/playground/simpleform.aspx", "Wait")
           type text(<about me textarea>, #username, "Standard")
         wait(8)
         click(<type="button">, "Left Click", "No")
         increment(#counter)
         wait for browser event("Everything Loaded", "")
     }
     decrement(#threadsOpen)
 }
}

 

This would also be incorrect, since when we call the about me command and run the thread command, the nodes inside the thread command (including the increment) do not run immediately. So it would be possible that we call the thread command 15 times before any of the threads actually start up and increment the #threadsOpen variable, which would lead to all 15 threads running at once.

 

The correct way to do this is to increment #threadsOpen at the beginning of about me, and decrement #threadsOpen at the end of the thread command:

 

define about me(#username) {
   increment(#threadsOpen)
   thread {
       in new browser {
           navigate("http://ubotstudio.com/playground/simpleform.aspx", "Wait")
           type text(<about me textarea>, #username, "Standard")
           wait(8)
           click(<type="button">, "Left Click", "No")
           increment(#counter)
           wait for browser event("Everything Loaded", "")
       }
       decrement(#threadsOpen)
   }
}

 

This works because we increment #threadsOpen before starting a thread, and we decrement #threadsOpen when the thread is finished.

 

Putting this all together it ends up looking like this:

 

Note: I also changed to looping of the $list total instead of looping the threads, because we want to loop through each account (X) for (Y) times. This means we shouldn't be looping based on Z (#threads).

 

ui text box("cycles", #cycles)
ui text box("threads", #threads)
divider
ui stat monitor("counter", #counter)
set(#counter, 0, "Global")
define about me(#username) {
   increment(#threadsOpen)
   thread {
       in new browser {
           navigate("http://ubotstudio.com/playground/simpleform.aspx", "Wait")
           type text(<about me textarea>, #username, "Standard")
           wait(8)
           click(<type="button">, "Left Click", "No")
           increment(#counter)
           wait for browser event("Everything Loaded", "")
       }
       decrement(#threadsOpen)
   }
}
divider
set(#threadsOpen, 0, "Global")
clear list(%usernames)
add list to list(%usernames, $list from file("{$special folder("Application")}\\usernames.txt"), "Delete", "Global")
loop(#cycles) {
   loop($list total(%usernames)) {
       loop while(#threadsOpen >= #threads) {
           wait(1)
       }
       about me($next list item(%usernames))
   }
   set list position(%usernames, 0)
}

 

Here's a video of it in action with 5 threads, running 3 times on 10 accounts: http://screencast.com/t/DcQoPlUweTkx

 

I've also attached the bot that I am using. Hope this helps clear things up a little bit, and I'm sorry if I lost you at any point. If you have future questions just let me know :).

 

Threading Example - Eddie.ubot

  • Like 9
Link to post
Share on other sites
  • 1 month later...
  • 1 month later...
  • 8 months later...

I wanted to add this screencast that TJ made for me to fix my bot. I was trying to increment a #row in a table for account info. TJ does it in a much more elegant way, a way that works :).

 

http://screencast.com/t/cO20EHysZx

 

I haven't seen this method any where else on this forum when i was searching for a solution.

 

Thanks again TJ

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...