RayMue 0 Posted March 9, 2011 Report Share Posted March 9, 2011 Hey Guys, I have got the following "problem". I want to scrape tables fro e.g. 10 pages.Goto Page 1Scrape TableGoto Pag 2Scrape TableGoto .... Save tables in a file. Unfortunately it seems not to be possible to add a scraped table to an existing table? Could someone help me ? Quote Link to post Share on other sites
UBotBuddy 331 Posted March 9, 2011 Report Share Posted March 9, 2011 That is correct. Once you are done with your initial scrapes then you can execute a shell command to append the CSV file together and then load that file. If I have time I may play with this to see if there are other options. What website are you using? Quote Link to post Share on other sites
RayMue 0 Posted March 9, 2011 Author Report Share Posted March 9, 2011 I'm scraping a SEO website for Pageranks, indexed pages etc. for several Domains Quote Link to post Share on other sites
JohnB 255 Posted March 9, 2011 Report Share Posted March 9, 2011 Another way (albeit not the best way) is to save each table to a file like such: set variable ---> #file = 1 save to file ---> i.e. Desktop\tables\table{1}.csv inc file Once you are done you can load the tables and merge them through another process. John Quote Link to post Share on other sites
RayMue 0 Posted March 9, 2011 Author Report Share Posted March 9, 2011 This was another idea of mine. set counter = 1 loop .... save blabla{counter} inc counter But how to merge 20 files into 1 ? Quote Link to post Share on other sites
RayMue 0 Posted March 9, 2011 Author Report Share Posted March 9, 2011 Found it - good old DOS - copy copy file1.txt +file2.txt result.txt Is it possible to add an separator line at the end of a table e.g. ----------- Quote Link to post Share on other sites
JohnB 255 Posted March 9, 2011 Report Share Posted March 9, 2011 I would probably loop through each column and add it to a list. When the list is done save it to a new table as a column, etc (do for each column). John Quote Link to post Share on other sites
HarryPotter 9 Posted April 21, 2012 Report Share Posted April 21, 2012 Unfortunately it seems not to be possible to add a scraped table to an existing table? oh no!!! But how to merge 20 files into 1 ? how do i do this if I have 1000s of files to merge... there must be a better way, is it possible to add a scraped table to an existing table SOMEHOW? Quote Link to post Share on other sites
LoWrIdErTJ - BotGuru 904 Posted April 21, 2012 Report Share Posted April 21, 2012 get the total rows of table you want to add to, then loop total number of new scraped table, adding each row to the new table, starting from the total number of rows from the table your adding to. it sounds confusing i know, but read it a couple times will make since.. Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.