GPL 0 Posted November 27, 2013 Report Share Posted November 27, 2013 I try to - scrap vine.co/v/ urls + description of them- scrap a selected number of urls from google search result and save them to mysql db and my code till now is here ui text box("Enter Search", #src) navigate("https://www.google.com/", "Wait") wait(3) type text(<name="q">, #src, "Standard") click(<name="btnG">, "Left Click", "No") wait(3) add list to list(%urls, $scrape attribute(<(before="<a href="\" AND after="\" onmousedown=")>, %urls), "Delete", "Global") save to file("", %urls) what's next ? Quote Link to post Share on other sites
GPL 0 Posted November 27, 2013 Author Report Share Posted November 27, 2013 I try to - scrap vine.co/v/ urls + description of them- scrap a selected number of urls from google search result and save them to mysql db and my code till now is here ui text box("Enter Search", #src) navigate("https://www.google.com/", "Wait") wait(3) type text(<name="q">, #src, "Standard") click(<name="btnG">, "Left Click", "No") wait(3) add list to list(%urls, $scrape attribute(<(before="<a href="\" AND after="\" onmousedown=")>, %urls), "Delete", "Global") save to file("", %urls) what's next ? Quote Link to post Share on other sites
Sirfrank 17 Posted November 27, 2013 Report Share Posted November 27, 2013 next is looping your list while uploading to DB 1 Quote Link to post Share on other sites
Edward_2 85 Posted November 27, 2013 Report Share Posted November 27, 2013 try using regex to scrape google. add list to list(%results, $find regular expression($scrape attribute(<class="r">, "innerhtml"), "(?<=href\\=\\\")http.*?(?=\\\")"), "Delete", "Global") 1 Quote Link to post Share on other sites
GPL 0 Posted November 27, 2013 Author Report Share Posted November 27, 2013 and how do i set the script to go to next page and do same as i did in first page and so on .. then .. to same them in a results.txt for example I want to select how many pages to navigate .. let's say 10 pages and after this to save what he found there in results.txt Quote Link to post Share on other sites
GPL 0 Posted November 27, 2013 Author Report Share Posted November 27, 2013 ui drop down("Pages to navigate", "10,20,30,40,50,100,200,300,500,1000", #nav) ui text box("Enter Search", #src) navigate("https://www.google.com/", "Wait") wait(3) type text(<name="q">, #src, "Standard") click(<name="btnG">, "Left Click", "No") wait(3) add list to list(%results, $find regular expression($scrape attribute(<class="r">, "innerhtml"), "(?<=href\\=\\\")http.*?(?=\\\")"), "Delete", "Global") wait(3) Quote Link to post Share on other sites
Edward_2 85 Posted November 27, 2013 Report Share Posted November 27, 2013 you set inside a loop. 1 Quote Link to post Share on other sites
GPL 0 Posted November 28, 2013 Author Report Share Posted November 28, 2013 I really don't know how to do this Quote Link to post Share on other sites
GPL 0 Posted November 28, 2013 Author Report Share Posted November 28, 2013 my appologies .. i dont know how to do this ! Quote Link to post Share on other sites
Sirfrank 17 Posted November 29, 2013 Report Share Posted November 29, 2013 send me your skype name in PM, will help you ot 2 Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.