Urgent Enquirer 0 Posted September 4, 2021 Report Share Posted September 4, 2021 (edited) Hi, I am trying to scrape the search result links from this page: https://www.google.com/search?q=covid+vaccine Snippet: add list to list(%urls,$scrape attribute(<outerhtml=w"<h3 class=\"LC20lb DKV0Md\">*</h3>">,"fullhref"),"Delete","Global") Note the above code is totally unrelated to the SERP page links (page 1, page 2, page 3 and so on). So if it is unrelated to such SERP links then why is Ubot scraping these SERP links ? Wrongfully is scraping the Search Result Page links (page 1, page 2, page 3 and so on) that you see at the bottom of the Google search result page). Test the following code out and see for yourselves. Code Context: ui text box("Keywords",#ui textbox_keywords) ui drop down("Max Results","10,20,30,40,50,60,70,80,90,100",#ui dropdown_max results) ui save file("Save To",#ui save file_save to) ui stat monitor("Total Results",$list total(%urls)) divider divider set(#max results,#ui dropdown_max results,"Global") set(#keywords,#ui textbox_keywords,"Global") set(#save to,#ui save file_save to,"Global") divider navigate("https://www.google.com/search?q={#keywords}","Wait") wait($rand(1,5)) clear list(%urls) loop while($both($comparison($list total(%urls),"<= Less than or equal to",#max results),$either($exists(<id="pnnext">),$exists(<outerhtml="<span style=\"display:block;margin-left:53px\">Next</span>">)))) { add list to list(%urls,$scrape attribute(<outerhtml=w"<h3 class=\"LC20lb DKV0Md\">*</h3>">,"fullhref"),"Delete","Global") if($exists(<id="pnnext">)) { then { comment("Click for \"Next\" Result.") click(<id="pnnext">,"Left Click","No") } else { if($exists(<outerhtml="<span style=\"display:block;margin-left:53px\">Next</span>">)) { then { comment("Click for \"Next\" Result.") click(<outerhtml="<span style=\"display:block;margin-left:53px\">Next</span>">,"Left Click","No") } else { alert("ERROR ALERTt: Cannot detect \"Link to NEXT Page\"! Quitting the session.") divider } } divider } } divider wait for browser event("Page Loaded","") } save to file(#save to,%urls) Edited September 5, 2021 by Urgent Enquirer Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.