Still can't scrape to table?

[tooltrainer 12]

I really thought this would be addressed by now but I see no mention of it in the forum, and the tables tutorial doesn't cover this at all. Seems like this would be one of the most commonly needed things!

 

I regularly find myself wanting to scrape tables of data from web pages, and manipulate them as tables. Here's a common hypothetical example... say I wanted to grab the table of proxies found here:

 

http://www.samair.ru/proxy/type-01.htm

 

I want to grab all the columns and rows of data, shove them into a table, and then be able to iterate through them etc. in UBot. This way I can have UBot automatically see data in one column, based on data from another column in the same row. Like for example, choose a proxy only if it is in the US and listed as anonymous... that sort of thing.

 

How is this possible with the new tables system? Seems like it should be so dead easy but without a "$scrape to table" function it still seems way more complicated than it should be.

 

Suggestions?

 

Thanks!

 

Jonathan

[Seth Turin 223]

I like the idea of having a $scrape table kind of function, to where you can choose the html table and scrape it to a ubot table. it might be a bit tricky to implement, because tables often don't have any information in the tags, but I'm going to look into it.

[tooltrainer 12]

Yeah especially when data is laid out in the APPEARANCE of a table, but without using a <table>... like when everything is done with DIV and SPAN tags, for example. But hell even if all it could do was scrape "real" tables, that would be absolutely tremendously useful!!!

 

Also as discussed once before in another thread on this subject, it would be good to have both $scrape table as well as a $scrape to table cell function.

 

Thanks for looking into this Seth!

 

Jonathan