Ez Google Scraper (Free Script By Helloinsomnia) Table Related Question

[christojuan 5]

Hi,

I'm trying to modify a free Google Scraper example script by Nick (Hellloinsomnia).  http://imautobots.co...oogle-scraper/  
(note that in another post there is a fix/update (http://network.ubotstudio.com/forum/index.php/topic/21896-ez-google-scraper-given-key-not-present-error/?do=findComment&comment=133868) that you may find necessary if you decide to download and help.

 

There are 4 tabs and one is named Google and is where the scrape is executed:

 

The script is awesome, but I'm trying to dump the url results from each keyword query search into a table where

row 1/col 1 is keyword 1 url result 1 > row 1/col 2 is keyword 1 url result 2 > row 1/col 3 is keyword 1 url result 3 > etc

row 2/col 1 is keyword 2 url result 1 > row 2/col 2 is keyword 2 url result 2 > row 2/col 3 is keyword 2 url result 3 > etc.

row 3/col 1 is keyword 3 url result 1 > etc.

 

Here is the original section that I am focused on:

https://www.screencast.com/t/Ig9X28k49lY

 

And here is the change, I have tried (new lines 12-24

 

    if($comparison($plugin function("LocalDictionary.dll", "$local dictionary get", "google_response"),"=",200)) {
        then {
            set(#number,0,"Global")
            loop($list total(%input_keywords)) {
                increment(#number)
                add item to list(%results,#number,"Delete","Global")
                add list to table as row(&Table,#number,0,%results)
                add list to list(%results,$plugin function("HTTP post.dll", "$xpath parser", $plugin function("LocalDictionary.dll", "$local dictionary get", "google_page_html"), "//h3[@class=\'r\']/a", "href", "HTML"),"Delete","Global")
                return($plugin function("LocalDictionary.dll", "$local dictionary get", "last_page"))
                clear list(%results)
            }
        }
        else {

 

Unfortunately, my result not what was intended:

https://www.screencast.com/t/Uvvmqq5q

 

Could anyone (or Nick himself would be awesome/appreciated) give me some insights on how I would achieve the intended result?

 

Thanks very much!

Chris

 

[iamddr 12]

first use xpath you got

url1

url2

url3

second use regex replace you got

kw,url1

kw,url2

kw,url3

third use add table to table

[christojuan 5]

Hey - thanks for your response. Unfortunately, I don't understand what you are suggesting

 

Let me try asking from another angle:

 

The google scraper part of this bot is here: https://www.screencast.com/t/QUmxmt0fnr70 

 

I added the highlighted section to try to get the script to take the 2 input keywords %keyword_inputs and place them in the first column of the table next to the associated url result.

 

It should look like this

kw1, url 1

kw1, url 2

kw1, url 3

... kw1, url 30

 

AND 

 

kw2, url 1

kw2, url 2

kw2, url 3

...kw2, url 30

 

all of the above in the same table 

 

The problem is that my result is as follows:

https://www.screencast.com/t/DitbMkJZ

 

I think the answer is to create a new temporary list at the completion of each keyword search, then count the number of results, then add that keyword to column 0, then move on to the next keyword... but I'm at a loss as to how to achieve that.

define $Google Results(#keyword, #proxy, #page_num, #count, #timeout) {
    plugin command("LocalDictionary.dll", "local dictionary add", "google_page_html", $plugin function("HTTP post.dll", "$http get", "https://www.google.com/search?q={$replace(#keyword," ","+")}&num={#count}&start={#page_num}", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0", "", #proxy, #timeout))
    plugin command("LocalDictionary.dll", "local dictionary add", "google_response", $plugin function("HTTP post.dll", "$http response", "response status"))
    if($plugin function("HTTP post.dll", "$xpath exists", $plugin function("LocalDictionary.dll", "$local dictionary get", "google_page_html"), "//a[@id=\'pnnext\']")) {
        then {
            plugin command("LocalDictionary.dll", "local dictionary add", "last_page", "false")
        }
        else {
            plugin command("LocalDictionary.dll", "local dictionary add", "last_page", "true")
        }
    }
    if($comparison($plugin function("LocalDictionary.dll", "$local dictionary get", "google_response"),"=",200)) {
        then {
            add list to table as column(&Table,0,0,%input_keywords)
            add list to table as column(&Table,0,1,%results)
            add list to list(%results,$plugin function("HTTP post.dll", "$xpath parser", $plugin function("LocalDictionary.dll", "$local dictionary get", "google_page_html"), "//h3[@class=\'r\']/a", "href", "HTML"),"Delete","Global")
            return($plugin function("LocalDictionary.dll", "$local dictionary get", "last_page"))
        }
        else {
            return("retry")
        }
    }
}

Can someone please point me in the right direction to enable each url in col 1 to have the associated keyword in col 0

 

Again, help would be appreciated.

 

Thanks!
Chris
 

Edited April 12, 2018 by christojuan