Ubot Dev Crashing 50+ Times A Day

[Cnotey 3]

Ok, so I'm submitting this here because it's obviously not a bug.

 

A little about my system:

 

Windows 10 x64

AMD 8350 @ 4.6 ghz oc'd

OCZ razor 16gb ram

Ubot Developer

Using Chrome 39 in Ubot

 

So I guess I'm just checking in with my peers to see if I'm the only one with this issue.  Working on project, my Ubot will crash at a minimum once every 15-20 minutes.  Usually when going into node mode to set an attribute to scrape, but that's not the only time.

 

Seriously, WTF?

 

I learned how to program about a year ago in C# and Golang.  But I continue to use Ubot for most bots because it's simply more efficient.

 

I'm beginning to wonder if I took the custom route if I would end up with a higher efficiency factor given the lost time in Ubot.

 

Any thoughts?  Anyone else know how to keep it from crashing when I breathe too deep?

[Anonym 53]

If it is any time, perhaps you should check the available disk space. I don't use Windows 10 and I probably won't the nearest 5 years or so (until Win7 support is gone), but I have read all over that Windows10 is causing a lot of problems with different programs.

A memory leak perhaps!? I read somewhere that Frank had come up with an idea on how to release memory in a certain case (hmmm... loading a empty HTML page (?) - search for it here in the forums)

 

 

If this is during startup:

 

1)  Disable all your plugins and trace it by enabling one at a time

2)  If that still doesn't work, uninstall UBS and re-install it again.

 

Just two weeks ago I got really strange crashes every time I started UBS, and the advise I got from the support was the above. I had to go to step 2, which I by the way thought would never work.

 

 

Good luck!

[cүвεя_נυηкιε 68]

I was fine until the last update (5.9 Rev 6)

Ill try the un-install 

 

but ill probably end up reverting back to the last version which was far more stable

 

 

Ill let you know if the re-install makes a difference

 

CJ

[cүвεя_נυηкιε 68]

Re-install did nothing for me, 

 

Un-installed again and re-installed V5.9.4 and all is groovy again

 

 

If i had to stab in the dark i would have said that the latest version was stalling/crashing when downloading an image, 

ran fine elsewhere, 

 

 

no matter, 5.9.4 is good enough for me

[jason 101]

Unclear from the two posts by Cyber Junkie and CNotey if these are related issues.

 

CNotey, UBot is definitely not supposed to crash every 20 minutes, and doesn't for most people. Can you open a support ticket with more information? 

[cүвεя_נυηкιε 68]

Unclear from the two posts by Cyber Junkie and CNotey if these are related issues.

 

CNotey, UBot is definitely not supposed to crash every 20 minutes, and doesn't for most people. Can you open a support ticket with more information? 

 

 

Hey Jason

 

My case wasnt as severe as CNotey's 

but i did notice that it was stalling at regular intervals, my PC is a bit of a power house so the chances are it would have crashed, 

for an i7 with 32GB RAM I dont normally see it struggle over anything as you can imagine,

but i did notice it struggle in particular when it was in the process of solving a captcha, 

so i figured it could have been the downloading the image or one of the http requests,

 

I did try multiple things, you know the usual procedure, un-install as mentioned above, delete UBot folder to let is download the files again, 

disabled AV/Firewall etc etc etc etc etc .. 

 

but the only thing that resolved it was reverting to 5.9.4, 

although i might add that since using UBot version 5.8.5 was lightening fast at my end, ... like ridiculously smooth quick lol

 

 

anywho, thought i would share the extra info

 

 

Cheers

CJ

[jason 101]

So the update released last week created a specific amount of slowdown, potentially during captcha solving?

 

Weird!

 

If we see any reports of this in the tracker we'll be sure to check it out. 

 

 

Thanks for the explanation!

[cүвεя_נυηкιε 68]

So the update released last week created a specific amount of slowdown, potentially during captcha solving?

 

Weird!

 

If we see any reports of this in the tracker we'll be sure to check it out. 

 

 

Thanks for the explanation!

 

Hey Jason

 

I had to come back to this thread and just let you know that the latest version just released (5.9.8 I think) has resolved a shed load at my end

The "like ridiculously smooth quick" that i mentioned before is back, ... and to be frank, .. Im pretty chuffed lol

 

Ive just sat here and had major project brain fry so decided to test a couple of things in the latest UBot instead lmao, 

I tested 5 of my BIG bots, you know just to push the boat out haha

All worked great, .. maybe even a little better than before, 

There was no stalling, bare in mind that 3 of the bots i tested are multi-threaded and all seemed to cope pretty damn good solving 50 captchas simultaneously

There were no downloaded images missing (i get this a lot normally and have to code around it) 

 

The only problem i noticed (still) is that if set to "Chrome 39" as suggested by MiriamMB in another thread somewhere regarding cookie control,

then a few things started to go wrong, ..

Most of any captcha images downloaded where actually captcha "sized" images with a random section of the page the captcha was on, 

this is the same for downloading most images using Chrome39, ..

Some of the tags dont get recognised and end up either skipping or timing out (like wait for element, if element, scrape element)

But the second i switch back to Chrome 21 all is back to groovy 

 

 

 

either way, .... still happy 

 

 

 

 

Cheers

CJ

[Cnotey 3]

Sorry I forgot about this thread.

 

The issue still appears to be tied to Chrome 39.  I set it back to Chrome 21 and it fixed which is why I forgot about the thread.  Which was a previous fix a few revisions ago to get around another issue!

 

However, I should note that a lot of websites aren't working with Chrome 21 so this isn't really a viable fix anymore.

[jason 101]

Hey Jason

 

I had to come back to this thread and just let you know that the latest version just released (5.9.8 I think) has resolved a shed load at my end

The "like ridiculously smooth quick" that i mentioned before is back, ... and to be frank, .. Im pretty chuffed lol

 

Ive just sat here and had major project brain fry so decided to test a couple of things in the latest UBot instead lmao, 

I tested 5 of my BIG bots, you know just to push the boat out haha

All worked great, .. maybe even a little better than before, 

There was no stalling, bare in mind that 3 of the bots i tested are multi-threaded and all seemed to cope pretty damn good solving 50 captchas simultaneously

There were no downloaded images missing (i get this a lot normally and have to code around it) 

 

The only problem i noticed (still) is that if set to "Chrome 39" as suggested by MiriamMB in another thread somewhere regarding cookie control,

then a few things started to go wrong, ..

Most of any captcha images downloaded where actually captcha "sized" images with a random section of the page the captcha was on, 

this is the same for downloading most images using Chrome39, ..

Some of the tags dont get recognised and end up either skipping or timing out (like wait for element, if element, scrape element)

But the second i switch back to Chrome 21 all is back to groovy 

 

 

 

either way, .... still happy 

 

 

 

 

Cheers

CJ

Awesome, glad to hear it!

 

If you've got time to open a tracker about the issue you're experiencing in Chrome 39 that'd be really great. The number of issues reported about it is pretty low so we're relying on anyone who has one to let us know so we can knock it out!

[Cnotey 3]

I have further nailed down the exact issue that is causing the crash.

 

The crash is being caused using Chrome 39 when using the element selector "<>".

 

If I use the element selector, select an element and then try and re-select another element, it crashes.

 

Pretty much making Ubot un-usable for me.

 

Hopefully we can see a fix in the next revision.

 

Thanks,

 

Cnotey

[mparempi 0]

I have further nailed down the exact issue that is causing the crash.

 

The crash is being caused using Chrome 39 when using the element selector "<>".

 

If I use the element selector, select an element and then try and re-select another element, it crashes.

 

Pretty much making Ubot un-usable for me.

 

Hopefully we can see a fix in the next revision.

 

Thanks,

 

Cnotey

 I have found exactly the same problem and it's Chrome 39 that causes the crash because Chrome 21 works fine.

[cүвεя_נυηкιε 68]

Hey Jason (and indeed all reading)

 

 

Im back with an update, 

im afraid its not so good news from my end .. 

 

The smoothness lasted a day or 2 then things started to go wrong, 

every couple of minutes the freezing was back, if left for like 5 mins it would eventually come back to normal but again short lived and it freezes again or crashes completely,

 

Also in V 5.9.4, 5.9.6 and 5.9.8 if Headless is used when Chrome39 is selected then there is a good 2 -5 second delay before anything happens, 

i should point out this is in a multi-thread environment running only 5 simultaneous. 

 

 

Im thinking that this new CEF browser will solve a lot of problems, 

mainly browser crashes, 

 

however, Ive tried Chrome 39 in 5.9.4, 5.9.6 and 5.9.8 and I am having major problems downloading images, 

mainly captcha images but I tested with a few other just in case but with the same results using Chrome39

 

here is an example :

http://i68.tinypic.com/aazncz.jpg

 

Ive tried all possible alternate <tags> but same result

 

the only way any of the stuff mentioned above works is if i switch back to Chrome21,

 

  

 

I will report it to the bug tracker at one point when i get some time, but with my current project list and workload time is merely a distant memory at the mo, .. 

hmmm, .. time, ..... Its funny how i can sit down thinking "easy job, ill get this done in 30 mins", ... 8 hours later XD

 

Anywho, .. 

 

Out of interest is there a better way to download a captcha image that i dont know about ?

I did try socket and http download but only get a weird transparent error image :/

 

 

 

 

Hope i didnt bore you with the details, 

 

Cheers

CJ

[Sebastian Rooks 3]

Hi,

 

I see that this thread is a few months old now, but I'm here because I'm having what, I think, is a similar problem.

 

 

Using the advanced element selector has crashed Ubot for me....somewhere in the neighborhood of 100 times total.

 

I can't predict what's going to cause it or when, but it always happens the same way.

 

1. I select an element to scrape.

2. click the gear icon to open the advanced element selector.

3. Sometimes it works, but other times... it says:

                         "no element found", then the advanced element selector pops up completely blank, then "Ubot has stopped working".

 

.......around 100 times, often several times in a row.

 

If anyone else has experienced this and has some sort of a solution, I'd really like to know what it is.

 

Thanks.

[cүвεя_נυηкιε 68]

Hi,

 

I see that this thread is a few months old now, but I'm here because I'm having what, I think, is a similar problem.

 

 

Using the advanced element selector has crashed Ubot for me....somewhere in the neighborhood of 100 times total.

 

I can't predict what's going to cause it or when, but it always happens the same way.

 

1. I select an element to scrape.

2. click the gear icon to open the advanced element selector.

3. Sometimes it works, but other times... it says:

                         "no element found", then the advanced element selector pops up completely blank, then "Ubot has stopped working".

 

.......around 100 times, often several times in a row.

 

If anyone else has experienced this and has some sort of a solution, I'd really like to know what it is.

 

Thanks.

 

 

Hi Sebastian, 

Yep, this is something that just happens, although your experience sounds excessive :/

 

There are measures you can take to "try" and reduce resource draining/crashing like splitting your bot into tabs and keep your defines away from the tab the bot is actually running, 

but to be honest once your bot becomes a certain size no amount of extra tabs will make much difference, 

I know when its about to happen at  my end because UBot starts lagging like crazy, and my PC doesnt lag, .. ever 

 

 

I think if you are seeing this error that much then submitting to the bug tracker might be a good idea in case there is an underlying reason 

 

 

 

 

 

Cheers

CJ

[Sebastian Rooks 3]

Thanks, it' nice to know that I'm not alone here.

 

Unfortunately, on my end, there doesn't seem to be a correlation between bot size, use of defines, and advanced element selector crashing.

It happened many times working on a big messy bot, and it's happened I think 5 times today on a really small bot, with no defines.

 

ehh, such is life.

[Ricx 3]

When I switch chrome 39 to 21 while my actual chrome is open, ubot fails to launch, so if you have chrome open try closing it there could be a clash.

 

Also after disabling: http://prntscr.com/aabz5f

 

I've noticed my element selector seems to be far more efficient. If it glitches i just switch to a new tab then back and it works fine.

[Bot-Factory 602]

Thanks, it' nice to know that I'm not alone here.

 

Unfortunately, on my end, there doesn't seem to be a correlation between bot size, use of defines, and advanced element selector crashing.

It happened many times working on a big messy bot, and it's happened I think 5 times today on a really small bot, with no defines.

 

ehh, such is life.

 

From my experience it depends on the website you are working with. With some websites the build in browser is not able to handle everything properly. 

Even the new one has some limitations on sites with a lot of active content, scripts and stuff.

 

One of the reasons why I'm using Google Chrome in all my bots now. Much more stable. A bit more work to build the expressions, because there is no easy element selector. 

But you can copy the necessary Xpath Expressions very easily. And yeah.. you need ExBrowser Plugin for that.

 

If you want to test it, please let me know. 

 

Dan

[Sebastian Rooks 3]

I do actually have a couple of instances of chrome open. I'm not sure if I did all the other times or not, but it's quite possible.

 

That's something new to try that I never would have guessed, kind of like deselecting that checkbox I didn't know about.

 

Thanks man, this has been an ongoing frustration for the past couple of months. I'm feeling hopeful. 

[Sebastian Rooks 3]

From my experience it depends on the website you are working with. With some websites the build in browser is not able to handle everything properly. 

Even the new one has some limitations on sites with a lot of active content, scripts and stuff.

 

One of the reasons why I'm using Google Chrome in all my bots now. Much more stable. A bit more work to build the expressions, because there is no easy element selector. 

But you can copy the necessary Xpath Expressions very easily. And yeah.. you need ExBrowser Plugin for that.

 

If you want to test it, please let me know. 

 

Dan

 

So far my problems (and my experience) have been limited to Twitter, and today, trying to scrape the keyword ideas from Google's keywords planner because up until about 6 minutes ago, I was apparently too stupid to just download the .csv and get it to open properly.

 

I've read good things about the Xpath plugin, and I'm definitely interested. But I'm still fairly new to everything, and I'm afraid Xpath is still outside of my understanding. 

[Chainsaw 9]

Just trying out Chrome 39 because Chrome 21 can't run Chrome extensions. However when I switch to Chrome 39, Ubot can't even render a single web page. Just displays blank white pages.

 

Anyone else had this issue?

[Bot-Factory 602]

Just trying out Chrome 39 because Chrome 21 can't run Chrome extensions. However when I switch to Chrome 39, Ubot can't even render a single web page. Just displays blank white pages.

 

Anyone else had this issue?

 

Do you have .net 4.5 installed?

Dan