Scrapping google result

[ddlx 4]

Hi all

 

I would like to scrappe the google serp urls.

 

This code from 'search result scrapper' (by Kreatus) was working fine.

But it not working now (i  think google have change his page) 

 

    if($comparison(#current total, "<", #total results)) {
        then {
            if($search page("onmousedown=\"return rwt(this")) {
                then {
                    add list to list(%result urls, $scrape attribute(<outerhtml=w"<a href=\"*\" class=\"*\" onmousedown=\"return rwt(this*)\">*</a>">, "href"), "Delete", "Global")
                }
                else {
                    add list to list(%result urls, $page scrape("<a href=\"/url?q=", "&sa="), "Delete", "Global")
                }
            }
            click(<style="background-position:-96px 0;width:71px">, "Left Click", "No")
            wait for browser event("DOM Ready", "")
        }
        else {
        }
    }

 

I think the wrong code is there :

$scrape attribute(<outerhtml=w"<a href=\"*\" class=\"*\" onmousedown=\"return rwt(this*)\">*</a>">, "href")

 

Is someone got the solution ?

[Walter1 8]

Same problem here, seems they dont have that specific class anymore.

[Kreatus (Ubot Ninja) 422]

This should work.

 

clear list(%result urls)
clear list(%final url)
if($comparison(#current total, "<", #total results)) {
    then {
        if($search page("onmousedown=\"return rwt(this")) {
            then {
                add list to list(%result urls, $scrape attribute(<tagname="h3">, "outerhtml"), "Delete", "Global")
            }
            else {
                add list to list(%result urls, $page scrape("<a href=\"/url?q=", "&sa="), "Delete", "Global")
            }
        }
        click(<style="background-position:-96px 0;width:71px">, "Left Click", "No")
        wait for browser event("DOM Ready", "")
    }
    else {
    }
}
loop($list total(%result urls)) {
    add item to list(%final url, $find regular expression($next list item(%result urls), "(?<=href=\").*?(?=\" onmousedown)"), "Delete", "Global")
}
load html($replace($text from list(%final url, "
"), "
", "<br>"))

[ddlx 4]

Cool, i ll try to insert this code in my boot... 

 

Will tel you if it work fine. 

[ddlx 4]

Hi, i am back

 

There is somethink wrong in :

 

loop($list total(%result urls)) {
    add item to list(%final url, $find regular expression($next list item(%result urls), "(?<=href=\").*?(?=\" onmousedown)"), "Delete", "Global")
}

 

It does not correctly find the "http... etc"

[ddlx 4]

Hum, i was wrong, it's working...   ... strange computer ...

[iceman007 0]

Only fairly new to Ubot so can someone explain the code above if it is working for some people as there appears to be no navigation module so there must be something missing?

[ddlx 4]

Arf lol, this is a partial code to scrape google results.

 

I ll post the complète code when finished my bot