Google Scrape Quickstart

[myhq 0]

hi, 

 

I have two list of keywords, and i want to search for all possible variations, how is this done best?

 

example: 

 

list 1:

keyword 1

keyword 2

keyword 3

 

list 2:

city 1

city 2

city 3

 

searches:

keyword 1 city 1

keyword 1 city 2

keyword 1 city 3

keyword 2 city 1

keyword 2 city 2

etc etc

Note: list 2 could be empty for a list of regular searches..

 

Next step would be to scrape X urls per keyword combination and add it to a table:

keyword combo, url, domain

keyword combo, url, domain

>> lets say X=50, then the keyword combo would repeat 50 times, with 50 different urls, before heading to the next keyword combo

>> url should be trimmed to domain

>> duplicate root urls from different keyword combos should be skipped.

I thought the best way to do this is to directly navigate to search query URLS (something like: http://screencast.com/t/3DlTrS2Exmo ) ,but i need some help how to walk through all the keywords variations and paging in the url.

 

 

Cheers!

Edited March 10, 2016 by myhq

[Code Docta (Nick C.) 638]

ok, had some problems.

This script is actually crashing ubot 5 and it should not.

 

clear list(%keywords)
clear list(%cities)
clear list(%joined)
add list to list(%keywords,$list from text("keyword 1
keyword 2
keyword 3
keyword 4
keyword 5",$new line),"Delete","Global")
add list to list(%cities,$list from text("city 1
city 2
city 3",$new line),"Delete","Global")
loop($list total(%cities)) {
    set(#city NLI,$next list item(%cities),"Global")
    loop($list total(%keywords)) {
        add item to list(%joined,"{$next list item(%keywords)} {#city NLI}","Don\'t Delete","Global")
    }
}

 

but that is the idea,

see attached file

 

tracker issue

http://tracker.ubotstudio.com/issues/1006

if u experience the same thing please plus one it, thanks +1

 

CD

join-keyword-city-lists.ubot

[Code Docta (Nick C.) 638]

here is an updated versiom

I got it to run with a wait inside first loop as suggested by support in tracker.

 

then added set list position to 0 to fix a  bug in my script.

 

 

join-keyword-city-lists - 2.ubot