cob007
-
Content Count
352 -
Joined
-
Last visited
-
Days Won
6
Posts posted by cob007
-
-
Hi dan,
To give you a real life example, see the screenshots as below.. I am trying to scrape the videos which have ads beginning in them, 2. Which have a optin box showing up. So my bot will tell which videos have ads or which videos have opt-in box.
Now the exbtrowser loops through a list and play each of these youtube videos..... while looping it scrapes the informaton of videos which have ads or optins
Now I am using the xpath expressions for these as: //div[@class=videoAdUiAttribution] and 2.
//a[@class=iv-promo-txt iv-click-target]Now the xpath expression worked fine but not all videos are going to have opt-in or an advert at the beginning infact 80% of them don't.
So xpath should return an empty listing for this.
The problem is i dont think I can use document text as an input because these ads and optins dont show in the html of the page. ( Not sure if this is what is called as iframes as to whatever data i am trying to scrape)
So although the scraping happens it doesn't show for what videos exactly these ads and optins dont exist.
-
Hi dan,
You had created a video somewhere where you shwoed how to make exbrowser return an empty item if nothing is to be scraped , because currently in my bot it is not returning an empty item while it is looping through a list of youtube URL's. So if for a particular URL there is nothing to be returned it should return an empty element so that list column stays consistent. I think you recorded a video for that if im not mistaken.
To give u an example. xbrowser is going in a loop to list of youtube urls and scraping the vidoes which have ads beginning in them using xpath
//div[@class=videoAdUiAttribution]
the scraping is working fine but what if a video does not have ads, it does not return an empty item for that video oto the list, so the synchonozing is wrong.
Actually you had recorded a video for this somewhere in the forum replying to somebody but I dont seem to find it. I cant find it in members area as well. So can you show me that video again
-
Hi dan,
You had created a video somewhere where you shwoed how to make exbrowser return an empty item if nothing is to be scraped , because currently in my bot it is not returning an empty item while it is looping through a list of youtube URL's. So if for a particular URL there is nothing to be returned it should return an empty element so that list column stays consistent. I think you recorded a video for that if im not mistaken.
-
I know this a old post but can somebody tell me what will be the javascript or jquery code to get the colour picker to choose colour in UI?
-
Hi dan,
I ran into a unusual issue bcause of youtube changing html.
You had created a video on howtocode xbrowser in of your videos where you scrape the youtube URL and the xpath for that is:
x://h3[@class=yt-lockup-title]/a
This worked fine for 15 days or so until my bot stopped scraping the title URL, so on checking my bot program, I had to change xpath to
x://h3[@class=yt-lockup-title contains-action-menu]/a
Now all of sudden when I checked today the new xpath didnt work
So I had to revert back to;
x://h3[@class=yt-lockup-title]/a
So now the software is working again with regards to exbrwoser scraping title information.
So what if the class element changes again, would I need to keep updating like this again and again by constantly checking or should I use a "or" command as
below so that whichever class is working the exbrowser will use that]
On running OR command as below didn't work in xpath builder Pro;(maybe i am doing something wrong with OR statement]
x://h3[@class=yt-lockup-title" or @class="yt-lockup-title contains-action-menu]/a
-
Can this work with stripe? I am planning to integrate stripe for payment processing, as soon as someone purchases, I want to be able to create same license key and same license email for one buyer for all the softwares in the store.
-
Hi dan,
Thanks for all fast customer support, one question I wanted to ask is can a website come to know that some automation is going on and people are using the website to get some data if i distribute my software to lot of customers using exbrowser plugin?
For example let us say we have a website like xyz that allows to scrape tags and and lot of people use their bot programmed by me with exbrwowser to get data using headless browsing i.e phantom JS from the website , so can the website owner know in any way that some sort of automation is going on?
-
Okay,the same thing is happening with twitter as well? I have asked the question here;
http://network.ubotstudio.com/forum/index.php/topic/19355-twitter-connction-issue/
And are there any tutorials on how to use fiddler with phantom JS?
-
Hi dan,
I am able to use exbrowser in incognito mode in chrome( because else fb shows shows a pop-up to enable or disable notification) to post a status update fb. However on switching to headless browing in phantom it didnt work... This is the code which works with chrome but doesnt with phantom, what could be issue.
ui text box("Username",#Username)
ui text box("Password",#Password)
plugin command("ExBrowser.dll", "ExBrowser CleanUp")
plugin command("ExBrowser.dll", "ExBrowser Launcher", "Chrome", "", " {$plugin function("ExBrowser.dll", "$ExBrowser Incognito Mode")}")
plugin command("ExBrowser.dll", "ExBrowser Delete All Cookies")
plugin command("ExBrowser.dll", "ExBrowser Navigate", "https://facebook.com")
wait(2)
plugin command("ExBrowser.dll", "ExBrowser Wait For Element", "//*[@id=\"email\"]", "Appear", 20)
plugin command("ExBrowser.dll", "ExBrowser Type Text (Human Delay)", "x://*[@id=\"email\"]", #Username, 100, 150)
plugin command("ExBrowser.dll", "ExBrowser Type Text (Human Delay)", "x://input[@type=\"password\"]", #Password, 100, 150)
plugin command("ExBrowser.dll", "ExBrowser Send Special Key", "x://input[@type=\"password\"]", "Enter")
if($plugin function("ExBrowser.dll", "$ExBrowser Element Exist", "x://*[@id=\"loginform\"]/div[4]/a[1]")) {
then {
alert("try again!")
plugin command("ExBrowser.dll", "ExBrowser Close")
}
else {
wait(1)
plugin command("ExBrowser.dll", "ExBrowser Type Text (Human Delay)", "x://textarea[@title=\"What\'s on your mind?\"]", #hello, 100, 150)
plugin command("ExBrowser.dll", "ExBrowser Click", "x://button[@class=\"_1mf7 _4jy0 _4jy3 _4jy1 _51sy selected _42ft\"]")
}
}
ui text box("hellp",#hello) -
Okay i was able to get to post update woth chrome but then ive to run it on incognito mode or else the chrome facebook status notification appears. but if i try to do same thing with phantom js it doesnt work
-
Hi TJ,
I followed your video and also was able to get the exbrowser to post tweet with CHROME as well but as soon as I switched to phantom JS it is not posting tweet. You can see my code below. I even changed the user agent to mozilla firefox and it still didnt post the tweet with phantom JS. What could be the issue?
ui text box("Username",#Username)
ui text box("Password",#Password)
plugin command("ExBrowser.dll", "ExBrowser CleanUp")
plugin command("ExBrowser.dll", "ExBrowser Launcher", "PhantomJS", "", $plugin function("ExBrowser.dll", "$ExBrowser Set User Agent", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36"))
plugin command("ExBrowser.dll", "ExBrowser Delete All Cookies")
plugin command("ExBrowser.dll", "ExBrowser Navigate", "https://twitter.com")
wait(2)
plugin command("ExBrowser.dll", "ExBrowser Click", "x://*[@id=\"doc\"]/div[1]/div/div[1]/div[2]/a[3]")
plugin command("ExBrowser.dll", "ExBrowser Wait For Element", "x://input[@name=\"session[username_or_email]\"]", "Appear", 20)
wait(2)
plugin command("ExBrowser.dll", "ExBrowser Type Text (Human Delay)", "x://input[@name=\"session[username_or_email]\"]", #Username, 100, 150)
plugin command("ExBrowser.dll", "ExBrowser Type Text (Human Delay)", "x://input[@name=\"session[password]\"]", #Password, 100, 150)
plugin command("ExBrowser.dll", "ExBrowser Send Special Key", "x://input[@name=\"session[password]\"]", "Enter")
if($plugin function("ExBrowser.dll", "$ExBrowser Element Exist", "x://button[@class=\"submit btn primary-btn\"]")) {
then {
alert("try again!")
plugin command("ExBrowser.dll", "ExBrowser Close")
}
else {
wait(1)
plugin command("ExBrowser.dll", "ExBrowser Wait For Element", "x://*[@id=\"global-new-tweet-button\"]", "Appear", 20)
plugin command("ExBrowser.dll", "ExBrowser Click", "x://*[@id=\"global-new-tweet-button\"]")
plugin command("ExBrowser.dll", "ExBrowser Wait For Element", "x://*[@id=\"tweet-box-global\"]", "Appear", 20)
plugin command("ExBrowser.dll", "ExBrowser Type Text (Human Delay)", "x://*[@id=\"tweet-box-global\"]", #hello, 100, 150)
plugin command("ExBrowser.dll", "ExBrowser Click", "x://button[@class=\"btn primary-btn tweet-action tweet-btn js-tweet-btn\"]")
}
}
ui text box("hellp",#hello) -
I am having trouble finding the xpath for status update, Do you guys now what will be xpath for writing a status update?
-
thanks pash, great customer service!
-
thanks TJ and pash that worked.Iam trying to find a command that clears the log file, so that it doesn't overwrite it each time but starts fresh but unable to find any command.. How do you clear a file that is in your hardrive as log.txt?
-
Hi TJ,
I used as you said the command like this but it still didn't work: I even added a 2 second wait, I am looping for each list item and adding log file for each list item.
set(#thumbs,0,"Global")
loop($list total(%thumbnails)) {
plugin command("Advanced Systems.dll", "create custom log file", "Thumbnails", " Success : {$list item(%thumbnails,#thumbs)}", "{$special folder("Desktop")}\\test.log")
wait(2)
increment(#thumbs)
} -
Hi pash,
Sorry I was not clear..
What I meant was...
I used your command like this in image...
It only prints time and success only for the first list item. For remaining list items it is not showing success or time.
Thumbnails
06/12/2016 20:08:59:
Success : https://i.ytimg.com/...bA/hqdefault.jp [Note: time and success is showing only for first list item,,)
https://i.ytimg.com/.../maxresdefault. [ Note: remaining list items are not having time or success appeded to them)
https://i.ytimg.com/.../maxresdefault.So if i have scraped a list of items and i want each list item to have time and success message before it, how do I do that
-
Hi pash,
after scraping a list of thumbnails and using your command like this to create log file, the time is shown only for the first image and after that no time is shown for remaining images.
plugin command("Advanced Systems.dll", "create custom log file", "Thumbnails", " Success : {%thumbnails}", "{$special folder("Desktop")}\\test.log")
==========================================================================================
Thumbnails
06/12/2016 20:08:59:
Success : https://i.ytimg.com/vi/JvnABGut5bA/hqdefault.jp
https://i.ytimg.com/vi/r39Fqh2sNPg/maxresdefault.
https://i.ytimg.com/vi/wk5CjMCDAOA/maxresdefault.
-
This one works for me
I wanted to confirm the above method worked for me as well. Thanks pash and everyone for help
-
can we create log file like the attached image using this plugin?
-
Hi CJ,
Thanks a ton. I did use a event button but still the right-click datagrid open URL did not work, can you share code of what u did exactly if at all its okay with you.
-
1
-
-
Thanks CJ,
I hope its sorted as I dont want to add another button inside UI
-
I have 2 datagrids attached to my XAML UI. The datagrid context menu function of opening a URL(right-clicking and open URL) is not working with XAML UI. Also the delete rows is not working. Is anyone else facing this issue?
define open_website {
shell("cmd.exe /c start {$plugin function("DataGridUI.dll", "$datagridUI get value", $plugin function("DataGridUI.dll", "$datagridUI selected index"), "URL")}")
} -
Hi itexpert,
I tried to find your twitter thread but I couldn't find it, so I am replying here. Moreover I sent you a PM almost a week back, regarding issues with twitter software and updates needed for both twitter and linkedin for which I haven't received any reply. So I would appreciate if you can look into it and get back , I can even record a video showing the issue. Also thanks for putting up great products and helping ubot community.
-
Hi pash,
I tried both ur examples of saving app setting from advanced dialog-2 and they are not working as per ur demo videos. although config.dat file did get created. Has there been any updates? When i try to restore. no data is restored to my app
[Sell] Exbrowser Now On Sale
in Plugins
Posted
Hi dan,
Thanks, the example below worked for me for the query i asked you, but when I tried as per screenshot according to the video (xpath plugin tutorial) of how to use generic xpath parser nothing was returned as u see in screenshot.