VentureOnline 49 Posted April 26, 2012 Report Share Posted April 26, 2012 Sorry for all the questions lately guys but the answers have helped me learn so much. So I am trying to get this bot to scrape Google and cycle each keyword entered into the text box through the list of footprints in footprints.txt. Now for some reason my main loop which is reading the %keywordlist total is opening extra browsers/threads if I place more then 1 keyword in. So if I put one keyword in the list it will all work fine. But when I put 2 keywords it'll open 2 browsers and if I set it to 2 threads there will then be 4 browsers. From what I can see its not finishing the inside loop before relooping the main loop. I ideally would want Keyword 1 to cycle through all footprints using the specified number of threads. Then restart the whole process for keyword 2 and so on. I am totally confused here and just spent about 3 hours trying to figure this out to no avail. Thanks for any help guys. clear list(%urls) clear list(%KeywordList) clear list(%Footprints) ui block text("Keywords", #Keywords) ui drop down("Threads", "1,2,3,4,5,6,7,8,9,10", #Threads) ui stat monitor("Total Scraped", $list total(%urls)) add list to list(%KeywordList, $list from text(#Keywords, $new line), "Delete", "Global") add list to list(%Footprints, $list from file("{$special folder("Application")}\\Footprints.txt"), "Delete", "Global") set(#Position, 0, "Global") loop($list total(%KeywordList)) { set(#FootprintPosition, 0, "Global") loop(#Threads) { thread { in new browser { loop($list total(%Footprints)) { navigate("www.google.com", "Wait") type text(<name="q">, "{$next list item(%Footprints)} {$list item(%KeywordList, #Position)}", "Standard") wait(1) click(<name="btng">, "Left Click", "No") wait(2) loop(2) { add list to list(%urls, $scrape attribute(<outerhtml=w"<a href=\"*\" class=\"l\" onmousedown=\"return rwt(this,\'\',\'\',\'\',\'*\',\'*\',\'\',\'*\',null,event)\">*">, "href"), "Delete", "Global") click(<id="pnnext">, "Left Click", "No") wait(4) save to file("{$special folder("Application")}/urls.txt", %urls) increment(#FootprintPosition) } } } } } increment(#Position) } Quote Link to post Share on other sites
Pete 121 Posted April 26, 2012 Report Share Posted April 26, 2012 Sorry I don’t have the time to rewrite the bot again but I’m pretty sure you can work all that out from the example Quote Link to post Share on other sites
VentureOnline 49 Posted April 27, 2012 Author Report Share Posted April 27, 2012 Im a bit confused. What does the "search for" do? I am just looking to be able to enter the keywords in the text box and have google search the 1st keyword + each footprint in the footprint.txt file and use the amount of threads specified. Then do the same for each subsequent keyword entered. So if the I did 2 keywords "dog training" + "dog collars" and specified 3 threads. The bot would do. thread 1 - dog training footprint1thread 2 - dog training footprint2thread 3 - dog training footprint3 When they finished thread 1 - dog training footprint4thread 2 - dog training footprint5thread 3 - dog training footprint6 And so on till all footprints are used. Then repeat this process for the "dog collars" keyword. Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.