How To Scrap Website Difrent Pages Automatic

[Mohammad 0]

how to scrap website different pages automatic, like my website status.pk i want ubot auto page navigation diffrents pages

[addy196 9]

What exactly do you want to scrape from your website ?

Edited December 3, 2015 by addy196

[HelloInsomnia 1103]

If you are talking about pagination then usually there is a next button you can have the program look for that button and if it exists then click it and if it doesn't then it is the last page.

[deliter 203]

Are you looking for a scraper that just crawls sites?
 
if so here is a truly aweful and useless one,but if it helps you then it was worth it

define clickLink(#myLink) {
    navigate(#myLink,"Wait")
    wait for browser event("DOM Ready","")
    wait for browser event("Page Loaded","")
    wait for browser event("Everything Loaded","")
    add item to list(%pageScrape,$document text,"Don\'t Delete","Global")
    clear list(%hrefBefore)
    add list to list(%hrefBefore,$scrape attribute(<tagname="a">,"href"),"Delete","Global")
    set(#beforePos,0,"Global")
    set(#host,$eval("location.host"),"Global")
    set(#host,$find regular expression(#host,"\\..+?\\.\\w+$"),"Global")
    loop($list total(%hrefBefore)) {
        if($comparison($find regular expression($list item(%hrefBefore,#beforePos),"^/|{#myHost}"),">",$nothing)) {
            then {
                add item to list(%href,$list item(%hrefBefore,#beforePos),"Delete","Global")
                increment(#beforePos)
            }
            else {
                increment(#beforePos)
            }
        }
    }
}
define webCrawl {
    set(#url,$prompt("Crawl Website URL"),"Global")
    clickLink(#url)
    set(#position,0,"Global")
    loop while($comparison(#position,"<",$list total(%href))) {
        clickLink($list item(%href,#position))
        increment(#position)
    }
}
webCrawl()