Seth Turin
-
Content Count
1185 -
Joined
-
Last visited
-
Days Won
50
Posts posted by Seth Turin
-
-
There's currently no way to do what was described specifically. The preferred methods of modularity are the define command, the include command, bot bank commands, and plugins.
-
Sorry guys, programmers are notoriously bad at estimating release dates, and UBot Stealth is no exception. The UBot Browser is crucial to UBot Studio, and is understandably difficult to replace.
Would i be wrong to think that the team have a list of inhouse bots that are built to login to 20 of the main social media sites, taking their details from a sql database, changing proxy, and then once logged in, perform a socket test action?
With every new release they run this bot, which has error message signals on points that don't work so they can be fixed?
This could be built into one program, and each release they just run this program, if it makes it through, it gets released. Surely this is already a given?!
As far as program logic goes, i don't think ubot can really get any better. I have the odd glitch here are there, but its so rare its easier to just have the code automatically restarting.
We have a term for this in the biz - we call it unit testing, or integration testing in this case. I even wrote a couple blogs on it, here and here. We do have a test suite here to make sure the commands work. Maintaining a test suite to log into specific websites would be harder to maintain, only because websites are prone to change, and the scripts would have to constantly be kept up to date.
Still though, we might end up open sourcing the test suite, so if anyone wants to add anything to them they can.
-
3
-
-
For what it's worth, the stealth browser (still incomplete) is considerably faster than the old ubot browser. It also gives you the option of running headless, which makes it very similar to a socket-based bot in performance, only with all the ease of use of a browser bot.
But kreatus is correct. 5.8.5 is an in between version that we released to solve a few issues while we continue perfecting UBot Stealth.
-
juicehunter is incorrect in his assertion. The cake is not a lie.
-
2
-
-
I posted about this in another thread: http://www.ubotstudio.com/forum/index.php?/topic/18162-well-they-have-definitely-been-working-on-the-browser-crash-issue/?p=112827
If you're seeing specific issues, the quickest way to get them fixed is by adding them to our tracker and letting support know.As for custom error pages, that sounds like a great feature request!
-
1
-
-
I want to state at the outset, to anyone not familiar with this particular issue, that browser crashes in UBot Studio affect mostly fringe cases of high-stress bots. To date, in the 4 or 5 years that we've been hearing about them, we have yet to have a single browser crash bug reported in our bug tracker that we could actually reproduce locally. But it's true, browsers do crash.
To shed some light on what all this means, consider the following links:
Google chrome's bug tracker lists 6018 open bugs that contain the word "crash". Firefox's lists 7037. UBot Studio's browser is based in chrome.
There's nothing we can do to make chrome stop crashing (although the browser update we're working on will help). The real problem was that UBot Studio itself would crash along with it. Now, those crashes are caught and contained. Making good use of logging tools like the 'establish' command will allow you to easily test for browser crashes, and react accordingly if they occur.
Sending a bot out into the internet is a lot like sending a teenager out into the world. There's a lot of stumbling blocks, people who mean them harm, and messy situations. But if you give them a solid enough foundation, they'll know how to tackle all of it without flinching. That foundation makes the difference between the ones that fail and the ones that are wildly successful.
-
1
-
-
Let's make sure it works before requesting new features on a new feature we haven't even been able to use yet
wise words. feature creep kills.
-
Well, I'm pretty excited
-
4
-
-
Welcome to the community Steve!
Let me clear up a bit about the Bot Bank. We first made it as a means for people to quickly share simple website scripts, a means of crowdsourcing. We had some early success with it, but it wasn't getting the amount of activity that we needed, so we sectioned off the website scripts and we essentially repurposed the Bot Bank to house some of UBot Studio's more advanced functionality, the UBot Extended Library. The Website section is still there and should be functional, but it's likely that many of the scripts are out of date, due to how frequently websites change. Our major focus now is on the UBot Extended Library.
There is also a private Bot Bank, which is a means for you to quickly save and retrieve scripts that you use a lot. We're working on redoing our tutorials to reflect the changes that have happened.
Hope this clears some of it up!
-
Yes, this should work as you expect.
-
You guys are awesome. Great answers.
-
Don't get too excited - when you start working with Tumblr in Ubot you will find more
problems. This has been confirmed by Seth who says its something to do with Awesomium.
Any of the popups that appear in Tumblr will not work.
My best bots have been dead in the water for several months due to the problems.
I've yet to see a single problem in tumblr that can't be fixed by switching the user agent. Be careful listening to folks who focus more on problems than solutions. UBot Studio is designed to let you attack problems from 1000s of different angles. If one doesn't work, another surely will! I'm glad that the user agent fix is working for you, GreyHat!
-
2
-
-
Can you tell us which site this is?
-
Just out of curiosity, why sell it instead of keeping it?
-
Great tips! Anyone else have stuff to add? Here's mine:
Get yourself set up with an affiliate program and start talking to affiliates. The right person selling your stuff can easily net you 4 or 5 figures in a single campaign.
-
4
-
-
great post. the software should do this on its own. I'm going to add an issue to the tracker for this.
-
1
-
-
This is something I'd like to add, but I can't make any statement on when. In the mean time, my favorite concept for desktop automation comes from the MIT labs, in a project called sikuli. Also, it's free! Check it out: http://www.sikuli.org/
-
Hi Ubot Team.
The recent downtimes made it very clear that we need 24*7 support for the ubot studio servers and services.
It's not enough to have you guys fixing the servers during your regular business hours in the US.
Ubot studio is a big project and a huge community. People from all over the world are using it.
And when you arrive in the office at 7am, it's already end of business for others.
So if you only stop server errors when you arrive in the office, that's just not fast enough.
There are server providers who offer managed servers with service monitoring.
With 24x7 support. Of course they cost more, but that might be something to consider?
Thanks in advance for your help.
Kindest regards
Dan
Yes, it has become clear that this issue will need a more permanent solution. We are looking into it.
-
3
-
-
I would fill a list with all the possible choices and then use $random list item
-
Yeah, there's no need to argue the counting issue. In either case, thread safe is an issue and we need a better parallel processing system. So getting back to the spirit of this thread, here are my two:
1. Simplified threading system, to make thread safety a non-issue and to make thread complexity easier to manage
2. Object oriented design capabilities, to help organize large bots, but without overcomplicating the basic scripting system for beginners.
-
3
-
-
Yes please...Seth Turin.. here is the challenge:
Goal:
90-100 Threads in parallel. So please no additional waits.
And the list should have exactly 10000 entries at the end.
if you can change this code so that it works in V4 and V5 without any crashes, no threads going down (constantly running 100threads until the end) and exactly 10000 entries in the list, then I will never say a bad word about threading again..
Oh I forgot.. No UI / bot freezes. Constantly updating the stat mons. And no 500+MB memory usage.
Because all of this is working when we use Plugins. But hopefully we are just to stupid to do it right with the native commands.
I really would love to make this working without plugins.
So please.. show us the light!
ui stat monitor("Open Threads:", #threadcount)
ui stat monitor("Total:", $list total(%numbers))
ui button("Stop") {
set(#threadcount, 0, "Global")
clear list(%numbers)
}
ui button("Start High (100Threads / 5000Loop)") {
set(#Loops, 10000, "Global")
set(#threads, 100, "Global")
RunTest1()
alert("Done")
}
define RunTest1 {
clear list(%numbers)
set(#threadcount, 0, "Global")
loop(#Loops) {
loop while($comparison(#threadcount, ">=", #threads)) {
wait(0.1)
}
increment(#threadcount)
RunTest1thread($random text(50))
}
loop while($comparison(#threadcount, ">", 0)) {
wait(0.1)
}
}
define RunTest1thread(#number) {
thread {
wait(0.2)
add item to list(%numbers, #number, "Don\'t Delete", "Global")
decrement(#threadcount)
}
}
Test file:
https://dl.dropboxusercontent.com/u/10322/Threads%20and%20Add%20Items%20%28test%29.ubot
It has all my examples. And it will also show examples how it works with plugins without any problems.
Dan
Ok, I spent a little time playing with this. This one is different than any of the examples I had seen so far. When I run the script, it does actually crash. When I drilled down into it, I can confirm that this is genuinely a thread safety issue.
Now, I still want to draw a distinction between this and counting issues. Kev was correct when he said:
What also makes the term "thread safe" on the forum more confusing is people are actually talking about two different scenarios and packaging it up in one term.
The counting issues are just an artifact of multithreading being genuinely difficult to do correctly. The crashing issue is because lists are not thread safe. And actually, neither are variables.
This is actually kind of a sticky problem to fix, and rather than spend a long time fixing a broken command that is difficult to use in the first place, I'd like to, instead, work on a better threading system from the ground up. The whole point of UBot is to simplify complex ideas, and I have some ideas for how we might do that with threading. In the mean time, if you need to use threading, I suggest doing it in UBot 4.
To anyone reading this thread who isn't a UBot veteran: these guys have all been using UBot for years. Multithreading is one of the most advanced things you can do with UBot, and one of the most difficult. I'd recommend waiting until you're comfortable with all the other commands before messing with threading.
-
2
-
-
This thread should be closed now, its lost the spirit of Dan's original question now, and is becoming a vacuum for complaints.
Ah, Thank you!
Quickly though, I'd like to try and explain what I think you guys are seeing with threading. Imagine a fruit bowl filled with fruit, and a bunch of hungry gorillas. In normal program flow, the gorillas stand in line, each awaiting his turn to take a piece of fruit from the bowl. If you have a zookeeper keeping count of how much fruit there is, he will have no problem doing so. Now imagine the same scenario, but instead of lining up, the gorillas may take fruit whenever they please in a free-for-all. Two gorillas might take fruit at the same time, or several seconds might pass before any fruit is taken, or any other number of unusual fruit counting might happen. To the zookeeper keeping count, it is no longer a clean and simple job. This is multithreading.
If that zookeeper is also adding fruit, it also becomes an issue of how much fruit he adds at a time, whether he does so synchronously or asynchronously. When the zookeeper looks at his tallies at the end of the day, unless he counted very carefully, the numbers might not look quite right. Looking at the situation, it's less likely that the gorillas or fruits are magically disappearing, and more likely that the counting is just a crazy process when it comes to threading.
In 100% of the threading issues I've looked at, I've been able to solve it by moving where the increments/decrements are placed.
-
1
-
-
* I always wanted compiling on our side. When I bought Ubot, I thought I bought a visual compiler like VB. Just imagine my horror when I learnt the compiling is done on Seth's side and they get hold of our source codes. I don't think they will ever build a compiler version for us because that would make them lose the chances to snoop on our source codes. That brings us to the question: Why do they want to view our source codes which is something really top secret ?
* Just how can we convert our bots to more stable language ? What are the steps ? Which members did all this and with which bots ? Where are the links to these bots ?
* Ah! You don't mind about the errors but do customers feel such ?
Server-side compiling actually serves two major functions: Makes UBot harder to crack and keeps you from having to download and install a lot more software, some of which may involve paid licenses. But anyway, we get around 1000 compiles per day, and we definitely have better things to do than mine them for secrets.
I heard on the grapevine that they will be doing this for V6
Where is this grapevine and why was I not invited?! Seriously, our post-Edward dev strategy is still being formulated, and details have not been discussed outside the team.
we would all probably just be happy with a quick porn site link on the toolbar tbh.
lolwut
1. Better browser than awesomium
2. More api for plugin developers.
Awesomium does have some annoying issues here and there. It's built on top of the chromium project (which is what chrome is based on), so it's a great deal better than internet explorer. I've seen a firefox-based browser engine floating around that I'd like to look at as well. All browsers have issues. Ultimately I'd like to implement all 3. The technology involved might make that dream hard to chase down.
Only 2. Keep it short and to the point please.
1. UI get's unresponsive with large bots.
Better in v4. Even worse in v5
Opening new tabs in v5 takes ages. Memory usage goes up over time. UI crashes from time to time.
Talking about bots with 20k+ lines. Most of that comes from HTML UIs in my case.
Or bots with 10 or more tabs.
2. Multithreading
Adding to global lists and tables is not thread safe.
Inc / decrement command is not thread safe.
In v5 I get some crashes with some kind of "boost" library. Not sure what that is.
Multithreading doesn't feel "complete" for me at the moment.
Shared browser with 2-5 windows in parallel works fine most of the time.
But I'm talking about high scaler scrapers with 50-100 threads in parallel. That's when things get starting squishy.
What are your 2 pain in the b.... things?
Dan
There might be some misunderstanding on the definition of 'thread safe'. In programming, if a variable is not thread safe, it means that the program will freeze. All the threading issues that I've seen in support have come from a misunderstanding of how threads work in general. And that's not a dig at anyone. Concurrency is one of the most difficult issues in programming. 60 years of programming language development, and even the experts still aren't very good at threading. I will add that I've noticed that UBot 4 does handle threading slightly differently than UBot 5. Neither version does it incorrectly, they just treat is slightly differently, which may be contributing to the confusion. Simplifying threading is one of my major goals moving forward.
This is a great thread. It's good to see where people's heads are on this stuff. There are some interesting ideas in here that I might have to think about more.
-
2
-
-
Hey Seth, Honest question. Why not hire some of the plugin developers as the core developer of ubot?
You can see how productive they are. Its a win-win between you and your customers.
Honest answer: We designed the plugin api to abstract away most of the complexity of the UBot Core. We purposefully designed it so that beginning programmers could be highly productive. The UBot Core is a whole different monster. It would be like asking a masseuse (even a talented, productive masseuse) to perform open-heart surgery.
Updated To Latest Version 5.85
in General Discussion
Posted
Thanks, I really appreciate that. I built pretty much this entire version with my own two hands. After deconstructing what worked and didn’t work in 5, I went through and created 5.5, and it’s been great improving it up to 5.8.5.
It's been a labor of love. And hate. And yelling at the computer at 3am, but mostly love.