Scraping multiple tables and then saving

[Woolie 0]

Hey all.

 

I am trying to make a bot that will scrape a table on page 1, click the 'Next Page' link and then scrape the table on page 2.

 

If I save after each scrape the file getas overwritten so I thought I would try to write the data to a list after each scrape which didn't seem to work as the final write to file gave me a blank file.

 

So am I right in thinking that you cannot make a list out of scraped tables and if so, how can I save to file two or three scraped tables as my bot goes round a loop?

 

Thanks

Woolie!

[UBotBuddy 331]

What version of UBot do you have Basic/Pro/Dev?

 

There is a Tables process whereby you can scrape a table but I believe that is available in the Pro & Dev versions.

 

But there is no limit to how many you can scrape so long as you can support the logic and keep them managed.

[JohnB 255]

Are you using a loop? If so, set a variable outside the loop, and use it as the file name:

 

set--->#filename (set it to a number like 100)

 

Then your save to file should look like this:

 

save to file--->path to file\{1}.csv where {1} is #filename

 

Make sure you place the increment node at the bottom of the loop so that the first file is 100.csv, the second cycle is 101.csv, etc

 

does that make sense?

 

John

 

 

[clyde 0]

JohnB, what I'm trying to do is scrape them all into one big table instead.

 

Is there a way? I'm really scratching my head here. lol

[Woolie 0]

Cheers for the replies all - I was hoping something along the lines of 'add to list' would be available for adding tables together.

 

I will go with the filename(random number) solution and then mush them altogether at the end

 

Thanks again.

[JohnB 255]

The problem you face is if each table is independent of one another they need to be scraped independently. There are ways to scrape the data within them though. PM me with the site information so I can take a look at the tables.

 

John