Scraping pages but can't get the last page

[Jpizzo 0]

Hey all again. I scrape a yellowpages web site gathering addresses and phone numbers but the bit won't scrape the last page because I run the add list to list command inside of a loop while statement and the loop while looks for the next tag. When it gets to the last page it fails to scrape it. I somehow remember a trick was said about how it could be scraped but I can't find it

[HelloInsomnia 1103]

Maybe in the loop while set a second variable, say #lastPage = false for example.

 

Then when the page is scraped and the next button is not found you set that variable to true and the bot still continues that iteration.

 

On the next iteration your comparison statement will say that there is no next element and #lastPage = true therefore you can finally exit the loop while.