Dear Ubot Team!
This thread was opened in reference to my open letter to you:
EDIT: I sent the letter to Seth personally, since it seems to have been deleted from the forums.
This was not my idea at all, I'm just trying to spread the word, and I hope I'll be able to reference the right users (and topics) here.
If I'm not mistaken this issue was first brought to our attention by UbotDev here:
He was also the first who was able to provide a fix for this (can't thank you enough UbotDev!):
If you need more info about this, I think Dan did a great job at testing and summarizing the current situation here:
There have also been a few bug reports in the tracker about this (and many-many forum conversations):
It definitely took some advanced users to find this issue, but that doesn't mean beginner/intermediate Ubotters aren't affected by this. Many users simply weren't able to figure out the "problem" with their code, since they didn't suspect that the issue lied in the core.
Anyway I attached a VERY SIMPLE example that will help you understand our biggest concern about threads. And I will also show you a solution (using both UbotDev's and Pash's plugin). And yes, this is not a real life example (meaning that no one would actually put a bot like that together), but it was simply created to help you understand our concern, and I can assure you this DOES happen in real life examples too (and this has nothing to do with plugins, a simple traffic bot could have this issue running at more than 20-30 threads). Running this code takes less than a second, and you'll see right away what we mean. Obviously in a real life example this anomaly would only show up after a longer amount of time (even though the best possible algorithm is used), but I think we should be able to run multi-threaded applications safely for more than just a few minutes at a time...
I kept the code as simple as possible, didn't even add alerts to make sure it doesn't confuse anyone (add them yourself if you wish), everything is checked through the debugger.
I only used 10 threads, the bot simply increments the variable 100 times in each thread. Therefore the correct result should be 1000 every time!
I quickly ran all three versions (10 times each), here are the results:
UBot_variables.ubot: (0 plugins used, I unchecked them all)
I would like to point out that I have a fairly high-end and well maintained PC with 8 GB RAM (surely better than the average user's PC)
I would also kindly ask you not to insult our intelligence again by saying that adding delays would solve this problem. I think this has also been discussed many times before. Adding delays only decreases the chance of this happening, but does NOT eliminate the problem. Because: Most multi-threaded algorithms have at least one variable (example: thread counter) that's updated/written from inside the thread, and if they are updated at the same time (or almost at the same time) you got a problem (it only gets written/updated once)! On the other hand, atomic operation variables could reduce the chance of this happening to almost 0.
As you can clearly see, UbotDev's and Pash's plugin were both able to solve the problem. But we need more than just one of these variables for more complex bots! Some very simple examples (there might be better ones, I was just trying to keep it simple yet again) can be read in Dan's thread from above.
In short: As far as we know, atomic operation variables are the way to solve this problem (but I'm sure you'll be able to figure out the fix for this better than us). I completely understand that you can't just replace Ubot's variables with atomic ones, since Ubot mas many different data types, but there could be some atomic operation possibilities (at least with integers). Anyway, the concern has been raised by us, but the solution is completely up to you. We believe you will find the right solution!
This issue is present in both Ubot 4 and 5, and I think we can all agree it should definitely be fixed in both versions, since Ubot 5 can't be used for complex projects yet.
Lists and tables aren't completely thread-safe either, but again: priorities! One thing at a time...
Thank you for listening!
p.s.: once again, the point of this thread is NOT to show a nice multi-threaded algorithm, but to provide a simple and fast example that demonstrates the miscounting that could happen when a global variable is accessed (written) inside a thread at the same time from different threads (or almost at the same time), which can happen with even the best algorithms.