Google scrape Title + URL in table, what's wrong with my script?

[Hagen 0]

Try it yourself ... it leaves out Titles, so the final table is false.

 

### script:

 

ui stat monitor("Results", $list total(%results))

ui stat monitor("ResultsURL", $list total(%resultsurl))

clear list(%results)

clear list(%resultsurl)

clear table(&final)

navigate("http://www.google.de/#hl=de&cp=3&gs_id=b&xhr=t&q=ayurveda&pf=p&sclient=psy-ab&site=&source=hp&pbx=1&oq=ayu&aq=0&aqi=g4&aql=&gs_sm=&gs_upl=&bav=on.2,or.r_gc.r_pw.,cf.osb&fp=368126ad50e1de32&biw=830&bih=489", "Wait")

loop(10) {

add list to list(%results, $scrape attribute(<outerhtml=w"<h3 class=\"r\">*</h3>">, "innertext"), "Delete", "Global")

add list to list(%resultsurl, $scrape attribute(<class="l">, "href"), "Delete", "Global")

click(<outerhtml=w"<a id=\"*\" class=\"*\" style=\"*\" href=\"*\"><span class=\"*\" style=\"*\"></span><span style=\"*\"><b>Weiter</b></span></a>">, "Left Click", "No")

wait for browser event("Everything Loaded", "")

wait($rand(3, 6))

}

add list to table as column(&final, 0, 1, %results)

add list to table as column(&final, 0, 2, %resultsurl)

save to file("{$special folder("Desktop")}/ergebnis.txt", &final)

[Kreatus (Ubot Ninja) 422]

Try this:

ui stat monitor("Results", $list total(%results))
ui stat monitor("ResultsURL", $list total(%resultsurl))
clear list(%results)
clear list(%resultsurl)
clear table(&final)
navigate("http://www.google.de/#hl=de&cp=3&gs_id=b&xhr=t&q=ayurveda&pf=p&sclient=psy-ab&site=&source=hp&pbx=1&oq=ayu&aq=0&aqi=g4&aql=&gs_sm=&gs_upl=&bav=on.2,or.r_gc.r_pw.,cf.osb&fp=368126ad50e1de32&biw=830&bih=489", "Wait")
loop(10) {
   add list to list(%results, $scrape attribute(<outerhtml=w"<h3 class=\"r\">*</h3>">, "innertext"), "Don\'t Delete", "Global")
   add list to list(%resultsurl, $scrape attribute(<class="l">, "href"), "Don\'t Delete", "Global")
   click(<outerhtml=w"<a id=\"*\" class=\"*\" style=\"*\" href=\"*\"><span class=\"*\" style=\"*\"></span><span style=\"*\"><b>Weiter</b></span></a>">, "Left Click", "No")
   wait for browser event("Everything Loaded", "")
   wait($rand(3, 6))
}
add list to table as column(&final, 0, 1, %results)
add list to table as column(&final, 0, 2, %resultsurl)
save to file("{$special folder("Desktop")}/ergebnis.csv", &final)

 

I just changed your add to list to not delete the duplicates. Im sure there's a duplicate title in the result thats why it doesn't match with the urls. Check it.

[Guest klauzser]

Nice script. I would like to try this one myself.

[JohnB 255]

Hi Hagen.

 

Please let is know if that resolved your issue. Thank you.

 

John