ksistrunk 0 Posted December 21, 2015 Report Share Posted December 21, 2015 Hi Guys, I am a relative newbie so a little green sorry. But dollartree is a beast to scrape. The categories are not showing up clearly and are covered with javascript or something. Can anybody help me to see what I should be doing to get the category links. 2 hours and still no success. Thanks, Kris Quote Link to post Share on other sites
luis carlos 94 Posted December 21, 2015 Report Share Posted December 21, 2015 post your code in order to see your advance and the url. Luis Carlos Quote Link to post Share on other sites
ksistrunk 0 Posted December 21, 2015 Author Report Share Posted December 21, 2015 navigate("https://www.dollartree.com/household/500/index.cat","Wait")clear list(%CategoryScrape)add list to list(%CategoryScrape,$scrape attribute(<class="">,"innertext"),"Delete","Global") Basically like this. I am trying to get the categories on the left side. Thanks, Kris Quote Link to post Share on other sites
Bill 7 Posted December 21, 2015 Report Share Posted December 21, 2015 Is this what your after? navigate("https://www.dollartree.com/household/500/index.cat","Wait")wait for browser event("Everything Loaded","")clear list(%CategoryTitle)clear list(%CategoryUrl)add list to list(%CategoryTitle,$find regular expression($replace($scrape attribute(<id="left_nav_data">,"outerhtml"),"&",$nothing),"(?<=\\\"\\>).*?(?=\\<\\/a\\>)"),"Don\'t Delete","Global")add list to list(%CategoryUrl,$find regular expression($scrape attribute(<id="left_nav_data">,"outerhtml"),"(?<=href\\=\\\"\\/).*?(?=\\\"\\>)"),"Delete","Global") Quote Link to post Share on other sites
ksistrunk 0 Posted December 22, 2015 Author Report Share Posted December 22, 2015 Wow that's it. Sorry I did not learn about using the "$find regular expressions" way of getting it out. Thanks so much! Quote Link to post Share on other sites
Bill 7 Posted December 22, 2015 Report Share Posted December 22, 2015 This is a worth while course http://network.ubotstudio.com/forum/index.php/topic/18236-sell-ubot-web-scraping-course-its-finally-here/ Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.