[Sell] Exbrowser Now On Sale

[cүвεя_נυηкιε 68]

Hi dan,

 

I ran into a unusual  issue bcause of youtube changing html.

 

You had created a video on howtocode xbrowser in of your videos where you scrape the youtube URL and the xpath for that is:

 

x://h3[@class=yt-lockup-title]/a

 

This worked fine for 15 days or so until my bot stopped scraping the title URL, so on checking my bot program, I had to change xpath to

 

x://h3[@class=yt-lockup-title contains-action-menu]/a

 

Now all of sudden when I checked today the new xpath didnt work

 

So I had to revert back to;

 

x://h3[@class=yt-lockup-title]/a

 

 

So now the software is working again with regards to exbrwoser scraping title information.

 

So what if the class element changes again, would I need to keep updating like this again and again by constantly checking or should I use a "or" command as

below so that whichever class is working the exbrowser will use that]

 

On running OR command as below didn't work in xpath builder Pro;(maybe i am doing something wrong with OR statement]

 

x://h3[@class=yt-lockup-title" or @class="yt-lockup-title contains-action-menu]/a

 

 

Hey cob007

 

I havent scraped from YT so not really qualified to offer any "solid" help

however, I have had similar issues of sites altering tags resulting in bots not functioning ....

 

I normally resolve this by selecting multiple tags, 

One that is ALWAYS present (even if duplicate exists on the same page)

and 1 or 2 more that actually target the tag but with wildcards

 

again i must stress that 90% of the projects i work on probably dont have the security and/or budget that YT has to continuously alter things, 

but if the right combination of direct "EXACT" tag search and "WildCard" tag searches can result in a more accurate/reliable response 

 

 

 

another way i get around similar problems if the above method fails too many times, 

If using wildcards and a scrape returns loads of other elements that are similar on the page then scrape to a list ..  until your scraped out, ... or your mouse breaks,

9 times out of 10 if you scrape to a list the value your looking for will normally appear in the same location so easy to fish out by location or using regex or xpath

 

 

 

you never know something above might spark some inspiration and you might stumble across your solution 

 

 

Good luck

CJ

[Bot-Factory 602]

Exbrowser V1.1.9 Beta 4 is available now

 

SendSpecial Keys Bugfix

Clipboard to Element Bugfix

"Strg + Text" in "CTRL + Text" renamed.

Send special Key (Select All then Delete)

Send special Key (Delete)

[Kreatus (Ubot Ninja) 422]

 

Exbrowser V1.1.9 Beta 4 is available now

 

SendSpecial Keys Bugfix

Clipboard to Element Bugfix

"Strg + Text" in "CTRL + Text" renamed.

Send special Key (Select All then Delete)

Send special Key (Delete)

 

 

Nevermind got it.

[Bot-Factory 602]

I'm finalizing V1.1.9.0

 

Any final feature requests, bug reports or improvements?

support(ad)bot-factory.com

 

Send me your ideas. And I'll see what I can do.

 

Cheers

Dan

 

[Kreatus (Ubot Ninja) 422]

 

I'm finalizing V1.1.9.0

 

Any final feature requests, bug reports or improvements?

support(ad)bot-factory.com

 

Send me your ideas. And I'll see what I can do.

 

Cheers

Dan

 

 

 

 

Got a suggestion.

Command to get the current browser ID so we can kill it afterwards that will work on threading.

Would be useful so we are not going to use the cleanup browser command that affects other working bot that have exbrowser.

This way we only need to kill the current browsers opened on separate bots.

[Bot-Factory 602]

Got a suggestion.

Command to get the current browser ID so we can kill it afterwards that will work on threading.

Would be useful so we are not going to use the cleanup browser command that affects other working bot that have exbrowser.

This way we only need to kill the current browsers opened on separate bots.

 

ExBrowser is using the ThreadID to identify the browser. You can use close browser within the thread to close the related browser.

But there is no way to close a specifiy browser from the outside. This would require a big change in the plugin and everyone would have to update his code. 

 

You can use cleanup light, which will not impact other bots, because it will not kill chromedriver.exe and phantomjs.exe

The downside is, that those processes might still run on your pc if you don't close your bots properly or if something crashes. 

 

Dan

[Bot-Factory 602]

Common Question:

Is there a Money Back Guarantee for ExBrowser Plugin?

 

Yes, 30 days. Managed automatically via Zaxaa.

 

More Questions? support(xx)bot-factory.com

[awpramono 1]

can exbrowser launch private firefox?

[Bot-Factory 602]

can exbrowser launch private firefox?

 

Hi. What do you mean by private firefox exactly?

You can specify the executable location. So if you download a stand alone FF version that can run without installation, then yes you can use it.

 

Dan

[awpramono 1]

Hi. What do you mean by private firefox exactly?

You can specify the executable location. So if you download a stand alone FF version that can run without installation, then yes you can use it.

 

Dan

 

FF private browser window

 

[cob007 19]

Hi dan,

 

You had created a video somewhere where you shwoed how to make exbrowser return an empty item if nothing is to be scraped , because currently in my bot it is not returning an empty item while it is looping through a list of youtube URL's. So if for a particular URL there is nothing to  be returned it should return an empty element so that list column stays consistent. I think you recorded a video for that if im not mistaken.

[Bot-Factory 602]

FF private browser window

 

 

Ok understand. 

 

Quick question, what's the reason for opening a private browsing FF instance?

ExBrowser will always launch a clean profile and as soon as you close the browser that profile is deleted. So there are no cookies or cache files that are stored after the browser is closed.

 

But if you tell me the exact usecase for this, I can definately take a look and see how this can be done.

 

Regards

Dan

[cob007 19]

Hi dan,

 

You had created a video somewhere where you shwoed how to make exbrowser return an empty item if nothing is to be scraped , because currently in my bot it is not returning an empty item while it is looping through a list of youtube URL's. So if for a particular URL there is nothing to  be returned it should return an empty element so that list column stays consistent. I think you recorded a video for that if im not mistaken.

 

To give u an example. xbrowser is going in a loop to list of youtube urls and scraping the vidoes which have ads beginning in them using xpath

 

//div[@class=videoAdUiAttribution]

 

the scraping is working fine but what if a video does not have ads, it does not return an empty item for that video oto the list, so the synchonozing is wrong.

 

Actually you had recorded a  video for this somewhere in the forum replying to somebody but I dont seem to find it. I cant find it  in members area as well. So can you show me that video again

[Bot-Factory 602]

To give u an example. xbrowser is going in a loop to list of youtube urls and scraping the vidoes which have ads beginning in them using xpath

 

//div[@class=videoAdUiAttribution]

 

the scraping is working fine but what if a video does not have ads, it does not return an empty item for that video oto the list, so the synchonozing is wrong.

 

Actually you had recorded a  video for this somewhere in the forum replying to somebody but I dont seem to find it. I cant find it  in members area as well. So can you show me that video again

 

This is a feature of the generic Xpath parser. And it's covered in the xpath plugin tutorial:

https://www.youtube.com/watch?v=Nw1U-QRJnJQ

 

Because that's exactly the same feature.

 

This will NOT help you in the case an element is NOT there. It will only return empty lines in a list result if the element is there but an attribute is empty.

Because if scrape a list of elements, but sometimes you have 20 and sometimes 17, there is no way to automatically identify which element is missing. 

Is it the first one? The last one or one in the middle. 

 

But there is probably another way to approach issues like that. But it's not possible to give a generic answer. Because it completely depends on the html code and what you try to archive.

 

If you give a real life example, we might be able to help you to find the correct solution.

 

Dan

[cob007 19]

Hi dan,

 

To give you a real life example, see the screenshots as below.. I am trying to scrape the videos which have ads beginning in them, 2. Which have a optin box showing up. So my bot will tell which videos have ads or which videos have opt-in box.

 

Now the exbtrowser   loops through a list and play each of these youtube videos..... while looping it scrapes the informaton of videos which have ads or optins

 

Now I am using the xpath expressions for these as: //div[@class=videoAdUiAttribution] and 2.
//a[@class=iv-promo-txt iv-click-target]

 

Now the xpath expression worked fine but not all videos are going to have opt-in or an advert at the beginning infact 80% of them don't.

 

So xpath should return an empty listing for this.

 

The problem is i dont think I can use document text as an input because these ads and optins dont show in the html of the page. ( Not sure if this is what is called as iframes as to whatever data i am trying to scrape)

 

So although the scraping happens it doesn't show for what videos exactly these ads and optins dont exist.

 

 

[SaHiN 14]

....

[Bot-Factory 602]

Now I am using the xpath expressions for these as: //div[@class=videoAdUiAttribution] and 2.

//a[@class=iv-promo-txt iv-click-target]

 

 

Now it depends a bit on the ExBrowser Command you use.

You probably use the single item scrape commands or the element exists command right?

 

And then you can use an IF then else to determine what to write into your list?

 

Basically you do (not correct syntax):

var1 = element exist //div[@class=videoAdUiAttribution]

var2 = element exist /a[@class=iv-promo-txt iv-click-target]

 

if var1 and var2 = true

set listelemet to BOTH

 

if var1 = true and var2 = false

set listelemet to AD

 

if var1 = false and var2 = true

set listelemet to PROMO

 

Maybe I'm a bit to tired now, but could you please explain what exactly the problem is here? 

What is not working. 

 

Cheers

Dan

 

 

 

 

if var1 and var2 = true

set listelemet to BOTH

[Bot-Factory 602]

What's new with ExBrowser V1.1.9.0:

http://www.bot-factory.com/exbrowser-v1-1-9-0-whats-new/

[Bot-Factory 602]

Just in case you still struggle with Xpath:

http://www.bot-factory.com/xpath-video-tutorials/

[cob007 19]

Hi dan,

 

Thanks, the example below worked for me for the query i asked you, but when I tried as per screenshot according to the video (xpath plugin tutorial) of how to use generic xpath parser nothing was returned as u see in screenshot.

 

 

 

Now it depends a bit on the ExBrowser Command you use.

You probably use the single item scrape commands or the element exists command right?

And then you can use an IF then else to determine what to write into your list?
 

Basically you do (not correct syntax):

var1 = element exist //div[@class=videoAdUiAttribution]
var2 = element exist /a[@class=iv-promo-txt iv-click-target]

 

if var1 and var2 = true

set listelemet to BOTH

 

if var1 = true and var2 = false

set listelemet to AD

 

if var1 = false and var2 = true

set listelemet to PROMO

 

Maybe I'm a bit to tired now, but could you please explain what exactly the problem is here? 
What is not working. 

 

Cheers

Dan

 

 

 

 

if var1 and var2 = true

set listelemet to BOTH

[HelloInsomnia 1103]

Hi dan,

 

Thanks, the example below worked for me for the query i asked you, but when I tried as per screenshot according to the video (xpath plugin tutorial) of how to use generic xpath parser nothing was returned as u see in screenshot.

 

It looks like you need to hit okay to close the node before running, I'm surprised that even was able to run.

[cob007 19]

I wanted to take a screenshot, that's why I opened it to show all parameters I added in inside. Let me record a video for it, i am sure im doing something wrong but I did exactly what video showed.

[Kreatus (Ubot Ninja) 422]

 

What's new with ExBrowser V1.1.9.0:

http://www.bot-factory.com/exbrowser-v1-1-9-0-whats-new/

 

 

Is this already released? Im still getting the old version on my account here http://licensing.bot-factory.com/user

[Bot-Factory 602]

Is this already released? Im still getting the old version on my account here http://licensing.bot-factory.com/user

 

Beta4 is the latest one. If you need the link: support(ad)bot-factory.com

 

Dan

[Bot-Factory 602]

How to use ExBrowser with FF Portable:

http://www.bot-factory.com/use-exbrowser-plugin-firefox-portable/

 

The FF portable Executable is:

FirefoxPortable_47.0.1_English.paf.exe

 

Enter it into google and you will find some download locations. 
Do NOT use version 48.x with Exbrowser at the moment. Won't work!