Frank 177 Posted October 6, 2011 Report Share Posted October 6, 2011 Hey all, I'm trying to program in version 3.5 of ubot and I'm up against a bit of a problem. I'm trying to scrape data from Google Local. I've got everything with on exception - the owner verified or not info. What google is doing is they are using javascript to re-write the style="display:none;" to style="" dynamically when the page loads. Now here's the rub. If you scan the text and try to match with that, you won't. The document text will show all the style="display:none;" in place. Here's an excerpt of the code in question from a google local page: <DIV class=end-edge-links jstcache="0"> <A id=pp-edit-link-lbc class=link-text href="javascript:void(0)" jsaction="ssedit.showLbcEdit" jstrack="1" jsdisplay="topbar_config.show_lbc_badge&&!topbar_config.show_edit_link" jstcache="5">Edit this place</A> <A style="DISPLAY: none" id=pp-edit-link class=link-text href="http://maps.google.ca/maps//maps/place?cid=1290796031657788252&q=computer+repair+calgary&hl=en&gl=ca&authuser=0&continue=/maps/place%3Fcid%3D1290796031657788252%26q%3Dcomputer%2Brepair%2Bcalgary%26hl%3Den%26gl%3Dca%26authuser%3D0&mode=edit&ved=0CFQQgQY&sa=X&ei=6DOOTs69FIeaywS5jqXcAQ" jsattrs="href: topbar_config.edit_link_url" jsdisplay="topbar_config.show_edit_link" jstcache="6" __jsproperties_parsed="true">Edit this place</A> <SPAN jsdisplay="(topbar_config.show_lbc_badge||topbar_config.show_edit_link)&&(topbar_config.show_claim_link||topbar_config.show_lbc_badge)" jstcache="7">- </SPAN> <A style="DISPLAY: none" class="link-text pp-lbc-link-text" href="http://www.google.com/local/add/choice?hl=en&gl=CA&latlng=1290796031657788252&q=&near=&utm_campaign=ww&utm_source=placepage-claim&utm_medium=et&ved=0CFUQggY&sa=X&ei=6DOOTs69FIeaywS5jqXcAQ" jsattrs="href: topbar_config.claim_link_url" jsdisplay="topbar_config.show_claim_link" jstcache="8" __jsproperties_parsed="true">Business owner?</A> <A href="http://www.google.com/local/add/businessCenter?hl=en&gl=CA&utm_campaign=ww&utm_source=placepage-badge&utm_medium=et&ved=0CFkQgQY&sa=X&ei=ZzWOTuCIGoPyzASU64GfAQ" jsattrs="href: topbar_config.lbc_badge_url" jsdisplay="topbar_config.show_lbc_badge" jstcache="9" __jsproperties_parsed="true"><IMG class=checkmark alt="" src="//maps.gstatic.com/mapfiles/transparent.png" jstcache="0"> <SPAN class=link-text jstcache="0">Owner-verified listing</SPAN> </A></DIV> Now, I"m pretty sure that I need to capture the values from these two javascript variables: topbar_config.show_lbc_badge and topbar_config.show_claim_linkAny of you guys have any idea how to capture these values and figure out what the verified listing value really is? Cheers, Frank Quote Link to post Share on other sites
Pete 121 Posted October 6, 2011 Report Share Posted October 6, 2011 Not sure if this will help or not <A href(.*)topbar_config(.*)</ Quote Link to post Share on other sites
Frank 177 Posted October 7, 2011 Author Report Share Posted October 7, 2011 nope, the code looks the same for either a confirmed or not confirmed. Frank Quote Link to post Share on other sites
localleadr 3 Posted October 7, 2011 Report Share Posted October 7, 2011 Try searching page for: href="http://www.google.com/local/add/choice? It's only present on unclaimed listings - hope this helps. 1 Quote Link to post Share on other sites
Frank 177 Posted October 7, 2011 Author Report Share Posted October 7, 2011 thanks but truly I'd love to figure this one out. I'm about to just write it in v4, even though it's still pretty buggy just to use the picture recognition feature. Frank Quote Link to post Share on other sites
localleadr 3 Posted October 7, 2011 Report Share Posted October 7, 2011 Hey Frank - here is an image of how I determine if a listing is claimed or not. Took me way to long to find a simple solution. The link to the places signup page only exists on unclaimed listings. http://screencast.com/t/wXdwxHfxS35E Hope this helps. 1 Quote Link to post Share on other sites
poging69 0 Posted October 7, 2011 Report Share Posted October 7, 2011 Try the version 4 too. I saw a lot of fixes on it from 3.5. Worth the shot. Quote Link to post Share on other sites
Frank 177 Posted October 7, 2011 Author Report Share Posted October 7, 2011 It has the ability to look for something on the page, so that may be the ticket. It's just soo darn buggy yet. Nothing against the Eddie, I know it takes time. Frank Quote Link to post Share on other sites
Frank 177 Posted October 7, 2011 Author Report Share Posted October 7, 2011 Hey Frank - here is an image of how I determine if a listing is claimed or not. Took me way to long to find a simple solution. The link to the places signup page only exists on unclaimed listings. http://screencast.com/t/wXdwxHfxS35E Hope this helps. All I have to say is that you SO ROCK!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Did I say that loud enough? Thank you very much. I really didn't understand your first post but the screen capture screams clear and loud!!! Now I don't have to fool with version 4!!! Yay, I was going to pull my hair out on that one alone. Too goofy for me yet. But I must say that it must have taken you a while to figure that one out. Great fix! Frank Quote Link to post Share on other sites
firionicable 0 Posted October 8, 2011 Report Share Posted October 8, 2011 This problem is driving me el polo loco as well. That is why I hate Google Local. It seems like they give limits to everything. Quote Link to post Share on other sites
Frank 177 Posted October 8, 2011 Author Report Share Posted October 8, 2011 Yea, but you have to agree - some definite potential there for $$$$. Frank Quote Link to post Share on other sites
localleadr 3 Posted October 8, 2011 Report Share Posted October 8, 2011 Glad I could help. Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.