Trying To Scrape The Categories Of Dollartree.com And Its A Beast. Please Help

[ksistrunk 0]

Hi Guys,

 

I am a relative newbie so a little green sorry.

 

But dollartree is a beast to scrape.  

 

The categories are not showing up clearly and are covered with javascript or something.

 

Can anybody help me to see what I should be doing to get the category links.

 

2 hours and still no success.

 

Thanks,

 

Kris

[luis carlos 94]

post your code in order to see your advance and the url.

 

 

Luis Carlos

[ksistrunk 0]

navigate("https://www.dollartree.com/household/500/index.cat","Wait")
clear list(%CategoryScrape)
add list to list(%CategoryScrape,$scrape attribute(<class="">,"innertext"),"Delete","Global")

 

 

Basically like this.  I am trying to get the categories on the left side.

 

Thanks,

 

Kris

[Bill 7]

Is this what your after?

 

navigate("https://www.dollartree.com/household/500/index.cat","Wait")
wait for browser event("Everything Loaded","")
clear list(%CategoryTitle)
clear list(%CategoryUrl)
add list to list(%CategoryTitle,$find regular expression($replace($scrape attribute(<id="left_nav_data">,"outerhtml"),"&",$nothing),"(?<=\\\"\\>).*?(?=\\<\\/a\\>)"),"Don\'t Delete","Global")
add list to list(%CategoryUrl,$find regular expression($scrape attribute(<id="left_nav_data">,"outerhtml"),"(?<=href\\=\\\"\\/).*?(?=\\\"\\>)"),"Delete","Global")

[ksistrunk 0]

Wow that's it.

 

Sorry I did not learn about using the "$find regular expressions" way of getting it out.

 

Thanks so much!

[Bill 7]

This is a worth while course http://network.ubotstudio.com/forum/index.php/topic/18236-sell-ubot-web-scraping-course-its-finally-here/