carlstarus 2 Posted October 19, 2013 Report Share Posted October 19, 2013 (edited) Hi guy's, Today I have another weird one... I need to track the indexing of a growing number of websites. I normally use the "site:domainname.com" command in Google to see how many pages are indexed for a particular domain on any given day. I can no longer afford to do this by hand and built the following script... ui stat monitor("Number of sites to check: {$list total(%SitesList)}", "")ui stat monitor("Current site: ", $replace(#CurrentSite, "site:", $nothing))set(#SitesToCheck, "{$nothing}{$replace($get files("{$special folder("Desktop")}\\INDEXING", "No"), ".csv", $nothing)}", "Global")add list to list(%SitesList, $list from text(#SitesToCheck, ""), "Delete", "Global")set(#SitesToCheck, $nothing, "Global")navigate("http://google.com", "Wait")wait for element(<name="q">, "", "Appear")set(#CurrentSite, "site:{$list item(%SitesList, 0)}", "Global")loop($list total(%SitesList)) { set(#CurrentSite, "site:{$next list item(%SitesList)}", "Global") wait($rand(1, 3)) navigate("http://google.com", "Wait") type text(<name="q">, #CurrentSite, "Standard") click(<name="btnG">, "Left Click", "No") set(#results, $scrape attribute(<id="resultStats">, "innertext"), "Global") set(#results, $replace($replace regular expression(#results, "About (.\{1,\}) results.\{1,\}", "$1"), ",", $nothing), "Global") wait(1) alert(#results)}divider Now here's the really, really strange bit... When I run the script "step by step", it pop ups an alert box with the number of pages indexed, yet whenever I try to run the full script the popup (alert box), is empty. I don't think it's a problem with memory either. I have an 8 Gig Windows 7 Machine. Does this kind of thing happen more often ? What could be the problem here? All suggestions are welcome Thanks for reading my post. Edited October 19, 2013 by carlstarus Quote Link to post Share on other sites
pftg4 102 Posted October 19, 2013 Report Share Posted October 19, 2013 after you click the button you probably need a wait before you scrape the results try that not at home right now pftg4 2 Quote Link to post Share on other sites
carlstarus 2 Posted October 19, 2013 Author Report Share Posted October 19, 2013 (edited) Wow... that did the trick... Thanks pftg4. Edited October 19, 2013 by carlstarus 1 Quote Link to post Share on other sites
pftg4 102 Posted October 19, 2013 Report Share Posted October 19, 2013 np glad to help Quote Link to post Share on other sites
Black Automation 20 Posted October 19, 2013 Report Share Posted October 19, 2013 Yes a simple "wait" should fix this issue. Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.