Tony5895 2 Posted March 1, 2018 Report Share Posted March 1, 2018 I have a muti-threaded bot that fetches web pages through the HTTP post plugin and saves data to a SQLite database. It also works with add-list-to-list to keep track of URLs of a site, on average between 5.000-15.000 items in the list. It's not a huge program but it eventually crashes every time I run it sooner or later. When I have 2-3 threads it lasts longer, 10-20 threads and it will crash for sure. Memory is always in the 500-800MB range, it doesn't build up, goes up and down all time. CPU usage 10-15%, so computer seems fast enough. Even on 25 threads CPU increases only by about 3%, memory usage stays the same. So i wonder where the problem is. In addition, the program runs slow imho but I have no idea where to improve performance more. I have no idea where exactly the bottleneck is or if that is just the maximum Ubot can do. But that's probably a different question. I wonder if anyone can share tips to avoid crashes. Thanks Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.