Woolie 0 Posted February 1, 2011 Report Share Posted February 1, 2011 Hey all. I am trying to make a bot that will scrape a table on page 1, click the 'Next Page' link and then scrape the table on page 2. If I save after each scrape the file getas overwritten so I thought I would try to write the data to a list after each scrape which didn't seem to work as the final write to file gave me a blank file. So am I right in thinking that you cannot make a list out of scraped tables and if so, how can I save to file two or three scraped tables as my bot goes round a loop? ThanksWoolie! Quote Link to post Share on other sites
UBotBuddy 331 Posted February 1, 2011 Report Share Posted February 1, 2011 What version of UBot do you have Basic/Pro/Dev? There is a Tables process whereby you can scrape a table but I believe that is available in the Pro & Dev versions. But there is no limit to how many you can scrape so long as you can support the logic and keep them managed. Quote Link to post Share on other sites
JohnB 255 Posted February 1, 2011 Report Share Posted February 1, 2011 Are you using a loop? If so, set a variable outside the loop, and use it as the file name: set--->#filename (set it to a number like 100) Then your save to file should look like this: save to file--->path to file\{1}.csv where {1} is #filename Make sure you place the increment node at the bottom of the loop so that the first file is 100.csv, the second cycle is 101.csv, etc does that make sense? John Quote Link to post Share on other sites
clyde 0 Posted February 14, 2011 Report Share Posted February 14, 2011 JohnB, what I'm trying to do is scrape them all into one big table instead. Is there a way? I'm really scratching my head here. lol Quote Link to post Share on other sites
Woolie 0 Posted February 14, 2011 Author Report Share Posted February 14, 2011 Cheers for the replies all - I was hoping something along the lines of 'add to list' would be available for adding tables together. I will go with the filename(random number) solution and then mush them altogether at the end Thanks again. Quote Link to post Share on other sites
JohnB 255 Posted February 14, 2011 Report Share Posted February 14, 2011 The problem you face is if each table is independent of one another they need to be scraped independently. There are ways to scrape the data within them though. PM me with the site information so I can take a look at the tables. John Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.