[SIMPLE PROOF] Variables are not thread-safe in Ubot 4/5

[Marton 48]

Dear Ubot Team!

 

This thread was opened in reference to my open letter to you:

EDIT: I sent the letter to Seth personally, since it seems to have been deleted from the forums.

 

This was not my idea at all, I'm just trying to spread the word, and I hope I'll be able to reference the right users (and topics) here.

 

If I'm not mistaken this issue was first brought to our attention by UbotDev here:

http://www.ubotstudio.com/forum/index.php?/topic/15122-must-read-threading-doesnt-work-as-expected-tested-in-v4/

 

He was also the first who was able to provide a fix for this (can't thank you enough UbotDev!):

http://www.ubotstudio.com/forum/index.php?/topic/15441-free-plugin-threads-counter-ubot-v4-threading-fixed/

 

If you need more info about this, I think Dan did a great job at testing and summarizing the current situation here:

http://www.ubotstudio.com/forum/index.php?/topic/16326-threads-and-ubot/

 

There have also been a few bug reports in the tracker about this (and many-many forum conversations):

http://tracker.ubotstudio.com/issues/208

http://tracker.ubotstudio.com/issues/379

 

It definitely took some advanced users to find this issue, but that doesn't mean beginner/intermediate Ubotters aren't affected by this. Many users simply weren't able to figure out the "problem" with their code, since they didn't suspect that the issue lied in the core.

 

Anyway I attached a VERY SIMPLE example that will help you understand our biggest concern about threads. And I will also show you a solution (using both UbotDev's and Pash's plugin). And yes, this is not a real life example (meaning that no one would actually put a bot like that together), but it was simply created to help you understand our concern, and I can assure you this DOES happen in real life examples too (and this has nothing to do with plugins, a simple traffic bot could have this issue running at more than 20-30 threads). Running this code takes less than a second, and you'll see right away what we mean. Obviously in a real life example this anomaly would only show up after a longer amount of time (even though the best possible algorithm is used), but I think we should be able to run multi-threaded applications safely for more than just a few minutes at a time...

 

I kept the code as simple as possible, didn't even add alerts to make sure it doesn't confuse anyone (add them yourself if you wish), everything is checked through the debugger.

 

I only used 10 threads, the bot simply increments the variable 100 times in each thread. Therefore the correct result should be 1000 every time!

I quickly ran all three versions (10 times each), here are the results:

 

UBot_variables.ubot: (0 plugins used, I unchecked them all)

914,998,979,1000,888,892,878,987,1000,982

 

UbotDev_theadscounter.ubot:

1000,1000,1000,1000,1000,1000,1000,1000,1000,1000

 

Pash_threadscounter.ubot:

1000,1000,1000,1000,1000,1000,1000,1000,1000,1000

 

I would like to point out that I have a fairly high-end and well maintained PC with 8 GB RAM (surely better than the average user's PC)

 

I would also kindly ask you not to insult our intelligence again by saying that adding delays would solve this problem. I think this has also been discussed many times before. Adding delays only decreases the chance of this happening, but does NOT eliminate the problem. Because: Most multi-threaded algorithms have at least one variable (example: thread counter) that's updated/written from inside the thread, and if they are updated at the same time (or almost at the same time) you got a problem (it only gets written/updated once)! On the other hand, atomic operation variables could reduce the chance of this happening to almost 0.

 

As you can clearly see, UbotDev's and Pash's plugin were both able to solve the problem. But we need more than just one of these variables for more complex bots! Some very simple examples (there might be better ones, I was just trying to keep it simple yet again) can be read in Dan's thread from above.

 

In short: As far as we know, atomic operation variables are the way to solve this problem (but I'm sure you'll be able to figure out the fix for this better than us). I completely understand that you can't just replace Ubot's variables with atomic ones, since Ubot mas many different data types, but there could be some atomic operation possibilities (at least with integers). Anyway, the concern has been raised by us, but the solution is completely up to you. We believe you will find the right solution!

 

This issue is present in both Ubot 4 and 5, and I think we can all agree it should definitely be fixed in both versions, since Ubot 5 can't be used for complex projects yet.

 

Lists and tables aren't completely thread-safe either, but again: priorities! One thing at a time...

 

Thank you for listening!

Marton

 

p.s.: once again, the point of this thread is NOT to show a nice multi-threaded algorithm, but to provide a simple and fast example that demonstrates the miscounting that could happen when a global variable is accessed (written) inside a thread at the same time from different threads (or almost at the same time), which can happen with even the best algorithms.

UBot_variables.ubot

UbotDev_threadscounter.ubot

Pash_threadscounter.ubot

[Marton 48]

I just found Dan's other bug report about this. I didn't have time to check it out yet, but it's surely useful too:

 

http://tracker.ubotstudio.com/issues/368#change-1194

 

I would also like to point out that the more complex examples in some other threads are way better than my example, but I feel this was needed to provide a "simple" demonstration that can be understood by everyone.

 

All the best,

Marton

[Bot-Factory 602]

I think the problem is that there are different understandings about threading:

 

Ubot Team:

5 threads with shared browser where each thread runs for at least 30 seconds

 

Threading works fine

 

WE:

100 threads in parallel where each thread needs less than 1 second to finish.

Using http plugin or sockets commands. Scraping thousands of sites.

 

Threading doesn't work fine

 

So it all depends on the perspective :-)

[UBotDev 276]

Good that someone brings this topic up again since I think it's really important for fast threading, but neither users and neither UBot team took the issue seriously (no one else supported me when I opened up an issue and it got quickly rejected by the staff).

 

I'm really disappointed and sad to hear that they still think that threading works as it should, when it was clearly demonstrated that it doesn't, with multiple users confirming it.

 

I hope that with more and more complaints they will finally look into this for v5!

[Marton 48]

Good that someone brings this topic up again since I think it's really important for fast threading, but neither users and neither UBot team took the issue seriously (no one else supported me when I opened up an issue and it got quickly rejected by the staff).

 

I'm really disappointed and sad to hear that they still think that threading works as it should, when it was clearly demonstrated that it doesn't, with multiple users confirming it.

 

I hope that with more and more complaints they will finally look into this for v5!

 

I feel your pain bro! I can't believe you actually had to develop a plugin because Ubot wouldn't solve this issue...

 

I also can't believe it's been almost 8(!) months since Ubot 5 came out and it's still useless (love paying for the "updates" though). I just opened up the above Ubot_variables script in v5 (with 0 plugins) and it CRASHED after running it a few times!

 

(In Ubot 4 at least it doesn't crash. Go figure...)

[Bot-Factory 602]

I feel your pain bro! I can't believe you actually had to develop a plugin because Ubot wouldn't solve this issue...

 

I also can't believe it's been almost 8(!) months since Ubot 5 came out and it's still useless (love paying for the "updates" though). I just opened up the above Ubot_variables script in v5 (with 0 plugins) and it CRASHED after running it a few times!

 

(In Ubot 4 at least it doesn't crash. Go figure...)

Hey just add a pause of 1 second     

[Pete 121]

I don't know who made that script "Ubot_Variables" but I would expect it to fail the way it's coded

[Marton 48]

Hey just add a pause of 1 second     

 

  Haha don't even joke about that Dan, the Ubot team will think you're serious   

 

I don't know who made that script "Ubot_Variables" but I would expect it to fail the way it's coded

Yes exactly, that's precisely the problem! The SAME script works perfectly using any of the two plugins mentioned above (instead of using the built-in increment/decrement functions).

 

Unfortunately you don't seem to understand (I saw your code before you edited your post), allow me to clarify again: I created a super simple example script to demonstrate the issue (and how easy it is to solve it). In your code you didn't increment/decrement the variable from INSIDE the thread, and all this has to do with multi-threading (incrementing/decrementing a global variable at the same time; like a thread counter, but it can also be other things).

 

Check out UbotDev's thread (link above) for more information on this, it's a really interesting topic. I'm just trying to raise awareness by showing an even simpler example, so the Ubotteam can understand it too. The proof is in the pudding, just run all three scripts and you'll see what we mean. This fix is a MUST for anyone who would like to create super-fast multi-threaded applications, because this kind of thing can happen in working bots too (just not as often, but even a difference of one in a million can mean a huge problem, you never know when it's going to hit you).

[Pete 121]

@martion  I understand the problem very well, and all history multi threading with ubot, the code I posted was incorrect that's why I had to edited the post as I had to pop out,

 

As far as plugins go I don't like most of them for different reasons,  but one I can't live without is Advanced shell by UbotDev

 

 

Anyway give this code ago

 

 

[Bot-Factory 602]

@martion  I understand the problem very well, and all history multi threading with ubot, the code I posted was incorrect that's why I had to edited the post as I had to pop out,

 

As far as plugins go I don't like most of them for different reasons,  but one I can't live without is Advanced shell by UbotDev

 

 

Anyway give this code ago

 

Now.. remove all the wait commands, so that every thread finishes between 0.1 and 0.5 seconds. 

I don't understand why we have to use wait commands as a workaround? When I use plugins it works perfectly well. Without any wait. Much faster and much more stable. 

 

So using wait to "fix" the native commands is not really a fix. We are working around the issue so that it's less likely for the issue to happen. But it's not a fix. 

 

Dan

[Pete 121]

Only the developer can tell you why we need all the wait commands (I can't see that happening)

Many have tried 50/150 threads but I've yet to see a stable working bot do it, sockets was removed at one time from version 4 because of all the problems.  And it seems version 5 has improved but not enough for running so many socket threads,  

I believe it’s a problem in the ubot core code, and reading between the lines of what Seth has said each define has to work as a queue so yes you can multi thread but not as you are thinking. Remember when ubot was first developed it was a single IE browser then came multi-threading then Firefox then awesomium browser and the updated awesomium browser with version 5, but I doubt the core code has been updated each  time

[Marton 48]

@martion  I understand the problem very well, and all history multi threading with ubot, the code I posted was incorrect that's why I had to edited the post as I had to pop out,

 

As far as plugins go I don't like most of them for different reasons,  but one I can't live without is Advanced shell by UbotDev

 

 

Anyway give this code ago

 

I did give it a go, and this is just another PROOF for the point we are trying to make  (didn't even have to remove the delays)

I don't think it's a great example though, because it's full of bad coding habits, but if you run it, you'll see that "Active Threads" don't go back to 0 at the end (similar to what UbotDev proved before). However if you use the SAME code with UbotDev's counter, it works perfectly again (code attached, check the end result, or you can also watch the debugger).

 

 

Only the developer can tell you why we need all the wait commands (I can't see that happening)

Many have tried 50/150 threads but I've yet to see a stable working bot do it, sockets was removed at one time from version 4 because of all the problems.  And it seems version 5 has improved but not enough for running so many socket threads,  

I believe it’s a problem in the ubot core code, and reading between the lines of what Seth has said each define has to work as a queue so yes you can multi thread but not as you are thinking. Remember when ubot was first developed it was a single IE browser then came multi-threading then Firefox then awesomium browser and the updated awesomium browser with version 5, but I doubt the core code has been updated each  time

 

The solution is already out there and I don't think we are out of line for requesting this change. We are not talking about making Ubot completely thread safe here (as Seth understands it), we just want the variables made as thread-safe as possible (UbotDev proved it can be achieved with atomic operation variables). At least the integers, so we can increase/decrease them safely, because even that would be a great start.

 

p.s.: please understand, this thread is not about sharing the different multi-threaded algorithms that *might* work in some cases (although the one you provided didn't work just as expected). This is just a simple example that proves that variables are indeed NOT thread-safe in Ubot (which makes multi-threading useless for serious projects, because two threads could finish just close enough to each other so that the built-in thread counter variable (or any other variable) won't be decremented (or incremented) properly. It's not a matter of how, but a matter of when!

UBot_variables_zap_ubotdev.ubot

[Pete 121]

I think you're getting a little confused

I also can't believe it's been almost 8(!) months since Ubot 5 came out and it's still useless (love paying for the "updates" though). I just opened up the above Ubot_variables script in v5 (with 0 plugins) and it CRASHED after running it a few times!

To which I replied

I don't know who made that script "Ubot_Variables" but I would expect it to fail the way it's coded

and added my script

 

Now say it failed, but you could in fact run it without it crashing, you then go on to say

but if you run it, you'll see that "Active Threads" don't go back to 0 at the end (similar to what UbotDev proved before). However if you use the SAME code with UbotDev's counter, it works perfectly again (code attached, check the end result, or you can also watch the debugger).

Then go on about  thread-safe & local variable "please point out" where I said ubot was thread-safe?
 

 

 

[Marton 48]

I think you're getting a little confused.

Hi, I wasn't talking about your script (which was deleted). I was talking about my Ubot_variables.ubot script (from the main post). If you try to run it in Ubot 5, it crashes (while it doesn't crash in Ubot 4).

 

But this thread is not about crashing (I just mentioned it once, I was hoping it wouldn't confuse anyone), it's simply about the counting (I'm trying to separate the problems).

 

Now say it failed, but you could in fact run it without it crashing

Your script doesn't crash, but it doesn't decrement the threads back to 0 (as it should). But as soon as you change it to UbotDev's functions, everything works as expected (that's why I attached the fixed version). So in the end your example fits here perfectly (although it does have some bad coding habits), but if I'm not mistaken you were trying to show a working multi-threaded example (instead of mine), which is not the point here (believe me I can write some nicely optimized multi-threaded algorithms myself, I'm way past the "simple multithreading example" that most people use here).

 

 

Then go on about  thread-safe & local variable

 

I never said one word about local variables (this thread is not about those either).

 

Everything is ok, I just didn't know why you shared your script in this thread I hope you aren't offended or anything though!

 

I'm just trying to show the Ubot team how inaccurate the counting is, which could cause serious problems since most multi-threaded algorithms (and I'm talking about the good ones here, not my example -which was intentionally created that way) have at least one variable which is incremented/decremented inside the thread -which could cause problems if they are being updated at the same time (or almost at the same time).

[Pete 121]

@marton

Let me try and explain for you, I will go back to your first post.
You are saying the increment and decrement are not working correctly and you would like this to be fixed by the ubot team.

You then add 3 scripts to show the problem.
But you fail to take into account the ubot team do not support plugins from other developers.
This in turn makes 2 of them irrelevant to a large extent.
This leaves just one script the ubot team could use for testing.

 

In a later post you go on to say “I have just opened up the above ubot script in version 5 and it crashed after running a few times”.
As the original thread was about the increment and decrement commands this makes the script pretty unusable to use for testing purposes in ubot 5. 

It is at this point that I downloaded the variables script, looked at the coding and posted my thoughts (as to my mind the script was encouraging data rush).

I then went on to produce you a script with version 5 that did not crash, did not encourage data rush and I believe was fit for testing purposes.

You then go on to say it did not work because it did not reduce the active threads back to 0.
Which is somewhat confusing as it was coded and runs on ubot 5, it does not crash and it shows the commands failure.

So in my eyes the bot does work for its intended purpose and is the reason I added it to your thread.
Please note I keep quoting ubot version 5 as any ref to version 4 is inappropriate for an updated.
 

[Marton 48]

 

 

So in my eyes the bot does work for its intended purpose and is the reason I added it to your thread.

 

Hey Zap, you're right, thanks!

 

 

 

But you fail to take into account the ubot team do not support plugins from other developers.

Oh, those plugins just show that there actually is an easy solution for this specific issue. We don't need them to support it, we just wish they would finally integrate this, or a similar solution.

 

Anyway, 

It looks like there was a point to all this after all, the Ubot team seems to have accepted the existence of the issue. Let's just hope they will eventually deliver on all those promises!

 

Have a nice weekend,

Marton