How do I scrape pages for their links

[alabamafreddie 1]

I am trying to scrape all these links off a page and the scrape table command is not working.

 

I use the id selector and it still wont save it to my table.

 

 

 

[UBotDev 276]

"scrape table" command will only work if you are trying to scrape data from HTML table, which looks like "<table>...</table>".

 

I think you don't have HTML table there, that's why you should use "$scrape attribute" and add all URLs into the list first, then add list to a table.

[alabamafreddie 1]

"scrape table" command will only work if you are trying to scrape data from HTML table, which looks like "<table>...</table>".

 

I think you don't have HTML table there, that's why you should use "$scrape attribute" and add all URLs into the list first, then add list to a table.

 

 

 

Yea I had a feeling but wasnt sure.

 

So there are about over 150 links there, I have to add all of them manually to the list, then to a table

 

Because what is $scrape attribute for exactly ?

Edited August 24, 2013 by alabamafreddie

[UBotDev 276]

$scrape attribute will return all of the URLs (150), you just need to choose the right element ("element to scrape").

[the_way 52]

you have a few options to get scrape the urls, 

 

you can use regex, which removes all the extra crap and leaves you with the urls needed.

 

you can also scrape the urls into one variable, and then split the variable with the $ list from text function,

and then screen out each list item into a final list.

 

if the urls from the page follow a predetermined format, you can use a loop while to scrape each of the urls, one at a time, and then add them to a list.

[alabamafreddie 1]

$scrape attribute will return all of the URLs (150), you just need to choose the right element ("element to scrape").

Right and how do I choose that element...I know firebug is recommended as an element inspector.

 

But in Ubot how do I manually type to use a certain element.

 

I have found the currect element and its #div_bodytext.

 

So what do I put $scrape attribute in ??

[UBotDev 276]

It's hard to tell what you should use without seeing seeing the HTML code...

 

Also, have you checked the tutorials yet? ...if not I strongly suggest you go through all.

 

Here is the part of video tutorial that should answer your question:

http://www.youtube.com/watch?v=e09XGjRBpTM&feature=player_detailpage#t=629