CharlieTango 3 Posted January 19, 2011 Report Share Posted January 19, 2011 Hey guys. I'm still a bit new to ubot, but I have been through every single tutorial and I still can't quite figure this problem out. I would really appreciate a kick in the right direction. There is a UL element on the page, which contains some specific data that I want to be scraped. The information I want to be scraped is "data-id="(whatever is inside here)", for a specified data-author that I will define in a UI text box. However, The UL element is full of variables that are subject to change everytime a new page is loaded. Here's an example of the code... <LI class=comment data-tag="R" data-removed="False" data-flagged="False" data-blocked="" data-pending="0" data-author="mitchelmcmaken" data-score="151" data-id="6_VWTn-l0CGcYKrAHHl7tRe1wkiQXcScyfsvdvd-qcI" data-author-viewing="False"><DIV class=content><DIV dir=ltr class=comment-text><P>What ever happened to finding free boat rides in your cereal box?</P><P>I miss those days..</P></DIV><DIV class=metadata-inline><A class=author title=mitchelmcmaken href="/user/mitchelmcmaken">mitchelmcmaken</A> <SPAN class=time>17 hours ago</SPAN> <SPAN class=comments-rating-positive>151 <IMG class="master-sprite comments-rating-thumbs-up" src="//s.ytimg.com/yt/img/pixel-vfl3z5WfW.gif"> </SPAN></DIV></DIV><DIV class=metadata><A class=author title=mitchelmcmaken href="/user/mitchelmcmaken">mitchelmcmaken</A> <SPAN class=time>17 hours ago</SPAN> <SPAN class=comments-rating-positive>151 <IMG class="master-sprite comments-rating-thumbs-up" src="//s.ytimg.com/yt/img/pixel-vfl3z5WfW.gif"> </SPAN></DIV></LI><LI class=comment data-tag="R" data-removed="False" data-flagged="False" data-blocked="" data-pending="0" data-author="TheDirtysouthboi" data-score="49" data-id="6_VWTn-l0CG7rReMdzxiKm8LvD8-1i6-8D8phwqRSWw" data-author-viewing="False"><DIV class=content><DIV dir=ltr class=comment-text><P>do u thin t-pain is tryna be a ghetto mad hatter?????</P></DIV><DIV class=metadata-inline><A class=author title=TheDirtysouthboi href="/user/TheDirtysouthboi">TheDirtysouthboi</A> <SPAN class=time>16 hours ago</SPAN> <SPAN class=comments-rating-positive>49 <IMG class="master-sprite comments-rating-thumbs-up" src="//s.ytimg.com/yt/img/pixel-vfl3z5WfW.gif"> </SPAN></DIV></DIV><DIV class=metadata><A class=author title=TheDirtysouthboi href="/user/TheDirtysouthboi">TheDirtysouthboi</A> <SPAN class=time>16 hours ago</SPAN> <SPAN class=comments-rating-positive>49 <IMG class="master-sprite comments-rating-thumbs-up" src="//s.ytimg.com/yt/img/pixel-vfl3z5WfW.gif"> </SPAN></DIV></LI>[/Quote] When I use the page scrape feature, to select the "data-id" information what i want the left and right sides are full of information that will not look the same when the page reloads, so i cant use this feature. When i use the "scrape by attribute" feature, I can only seem to effectively scrape the entire outerhtml/innerhtml, and not the preferred specific content within. I hope I have explained myself okay here. Would appreciate any input that sets me in the right direction, thanks! Quote Link to post Share on other sites
UBotBuddy 331 Posted January 20, 2011 Report Share Posted January 20, 2011 Give us a website to look at and maybe we can help you. Quote Link to post Share on other sites
Praney Behl 314 Posted January 20, 2011 Report Share Posted January 20, 2011 Yes A website or a webpage will certainly help to look into it.. Quote Link to post Share on other sites
CharlieTango 3 Posted January 20, 2011 Author Report Share Posted January 20, 2011 Give us a website to look at and maybe we can help you. Sure. I should've mentioned that. Looking at any youtube video, with comment ratings enabled will do. I'm basically trying to write a bot for youtube videos, which will scrape the information inside "data-id="(this information)" from a specific comment on youtube. So, the final idea is to have a bot that will search for a specified username in the comments, and then scrape the information inside data-id="" for that comment, and save that information to a file. Quote Link to post Share on other sites
Praney Behl 314 Posted January 20, 2011 Report Share Posted January 20, 2011 There you go mate done.. easy peezie Attached the sample with the post. Just enter the username you want to scrape in the UI and done. Cheers! Praneysample_Charlie_Tango.ubot 1 Quote Link to post Share on other sites
Praney Behl 314 Posted January 20, 2011 Report Share Posted January 20, 2011 Oh forgot to mention that it is just a straight and simple code. for what you need to do.You will need to navigate to the YT URL first to get results.. Let me know how you went with it. Praney 1 Quote Link to post Share on other sites
CharlieTango 3 Posted January 20, 2011 Author Report Share Posted January 20, 2011 There you go mate done.. easy peezie Attached the sample with the post. Just enter the username you want to scrape in the UI and done. Cheers! Praney Wow, thank you so much for your help. This must've been an incredibly easy one for you, haha. I'm looking over what you've done right now, to try and understand it. Quote Link to post Share on other sites
Praney Behl 314 Posted January 20, 2011 Report Share Posted January 20, 2011 Hey sorry there was a bug in the first one it was scraping the "data-id" for the first comment. I have fixed that. Please try the updated script in with post.. Cheers! Praneysample_Charlie_Tango_Updated.ubot 2 Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.