George 1 Posted February 20, 2012 Report Share Posted February 20, 2012 (edited) OK so I created a sweet little bot (my second overall) that loops through a text file containing keywords and scrapes the Google results then saves them to a text file. Works GREAT! Except the results are always limited to 10. I know I can go in and do it manually at http://www.google.com/preferences but every time my compiled bot launches it's back to 10. I can of course get the bot to click the checkbox to disable suggestions which allows you to move the slider to 100. But for the life of me I can't figure out how to get ubot to move the slider. My ultimate goal is to have a bot I can set to run daily and have it go in and set it to 100 results then do its thing. Any suggestions. EDIT: Thanks Kreatus. I guess I have to o something to add the fact that I have a license to my profile here but it's not exactly readily available information around here or in the welcome email... In any case I can't download what you attached (I can't even reply to my own thread.) but I did figure a crude method out. I simply put a click command in a loop and selected the element "<style="width:6px">". Then I just clicked advanced, made sure it was set to Left Mouse Click and changed Click Using Location to YES. Ran the loop 5 times. It worked like a charm. Edited February 20, 2012 by George Quote Link to post Share on other sites
Kreatus (Ubot Ninja) 422 Posted February 20, 2012 Report Share Posted February 20, 2012 Hi, check this code below. This is the google scraper source I uploaded on bot bank but hasn't approve yet.It will scrape up to 1000 google result urls. google scraper.ubot 1 Quote Link to post Share on other sites
Legend 181 Posted February 21, 2012 Report Share Posted February 21, 2012 Hey Kreatus, Thanks for the bot but it doesn't work for me, it freezes up when it gets to: loop(5) { click(<image="___IMAGE___0___IMAGE___">, "Left Click", "No")} Not sure if anyone else is having the same problem but I wanted to let you know. http://ubotstudio.com/forum/public/style_emoticons/default/smile.gif 1 Quote Link to post Share on other sites
a2mateit 395 Posted February 21, 2012 Report Share Posted February 21, 2012 Freezing on loop(5) {click(<image="___IMAGE___0___IMAGE___">, "Left Click", "No")} for me to Quote Link to post Share on other sites
Kreatus (Ubot Ninja) 422 Posted February 21, 2012 Report Share Posted February 21, 2012 Not sure what happened to my code but it should be an image on that click command.. Here's the updated one google scraper.ubot 1 Quote Link to post Share on other sites
JohnB 255 Posted February 21, 2012 Report Share Posted February 21, 2012 George, send me your username and I will be able to lift the forum restrictions for you. John Quote Link to post Share on other sites
Legend 181 Posted February 21, 2012 Report Share Posted February 21, 2012 Ah... much better! Thanks! http://ubotstudio.com/forum/public/style_emoticons/default/smile.gif Quote Link to post Share on other sites
George 1 Posted February 22, 2012 Author Report Share Posted February 22, 2012 George, send me your username and I will be able to lift the forum restrictions for you. JohnThe person that called me to validate the license took care of me. Thanks. Quote Link to post Share on other sites
turbolapp 5 Posted March 22, 2012 Report Share Posted March 22, 2012 I'm trying to get 100 results as well but the methods talked about here aren't working for me. Anyone have a current work around? Edit: nm. When trying George's method I missed click using location "yes". Wow that is some impressive duct tape!! Quote Link to post Share on other sites
Kreatus (Ubot Ninja) 422 Posted March 22, 2012 Report Share Posted March 22, 2012 I'm trying to get 100 results as well but the methods talked about here aren't working for me. Anyone have a current work around? Edit: nm. When trying George's method I missed click using location "yes". Wow that is some impressive duct tape!! Check this google scraper.ubot this should fix the problem from my last upload. Quote Link to post Share on other sites
turbolapp 5 Posted March 22, 2012 Report Share Posted March 22, 2012 Check this google scraper.ubot this should fix the problem from my last upload. Your bot doesn't work. Gets shut down actually. But's it's fine I'm doing George's work around. HOWEVER I can't for the life of me click the goddamn save button. Any ideas? Quote Link to post Share on other sites
Kreatus (Ubot Ninja) 422 Posted March 22, 2012 Report Share Posted March 22, 2012 Your bot doesn't work. Gets shut down actually. But's it's fine I'm doing George's work around. HOWEVER I can't for the life of me click the goddamn save button. Any ideas? Ok. Well it is working perfectly on my side. Quote Link to post Share on other sites
turbolapp 5 Posted March 22, 2012 Report Share Posted March 22, 2012 I didn't see where it clicked the save button. How are you doing this? Quote Link to post Share on other sites
Kreatus (Ubot Ninja) 422 Posted March 22, 2012 Report Share Posted March 22, 2012 By clicking the side of the save button since every country have different save button text. Quote Link to post Share on other sites
turbolapp 5 Posted March 22, 2012 Report Share Posted March 22, 2012 I'm having the same problem that the two posters above you had. When I copy your module and paste it into mine I get what they get: a frozen bot. When I just run your bot as is it freezes. When I try to examine the element to try to find this image module of yours, all i get to work with is: <class="goog-inline-block jfk-button jfk-button-action jfk-button-hover"> which does nothing. At my wits end here over one stupid button. Quote Link to post Share on other sites
Kreatus (Ubot Ninja) 422 Posted March 22, 2012 Report Share Posted March 22, 2012 I'm having the same problem that the two posters above you had. When I copy your module and paste it into mine I get what they get: a frozen bot. When I just run your bot as is it freezes. When I try to examine the element to try to find this image module of yours, all i get to work with is: <class="goog-inline-block jfk-button jfk-button-action jfk-button-hover"> which does nothing. At my wits end here over one stupid button. Not sure why it's freezing at your end but it is running normally on my computer. Try this googlescraper.ubot Quote Link to post Share on other sites
turbolapp 5 Posted March 22, 2012 Report Share Posted March 22, 2012 Alright, looks like I finally found a way to click it: click(<outerhtml="<div role=\"button\" class=\"goog-inline-block jfk-button jfk-button-action\" tabindex=\"0\" style=\"-webkit-user-select: none; \">Save</div>">, "Left Click", "No") Wasted a morning on it, but I win cause I figured it out. Right?...Right? Edit: Kreatus...maybe it's something to do with your dev licences vs my standard? dunno. but thanks for trying. Quote Link to post Share on other sites
turbolapp 5 Posted March 22, 2012 Report Share Posted March 22, 2012 Ahhh the google gods tease. As you stated above it won't work in other countries. So I guess I'll spend the rest of my day on this because I'm just too bullheaded to freakin stop. (btw, your latest bot still has the same problem. I get that pop up from ubot telling me ubot has stopped working and must be closed. I think it probably is the different versions problem because that image module is foreign to me) Anyways, I'm attaching the bot I have so far if anyone wants to take a gander at it.help.ubot Quote Link to post Share on other sites
a2mateit 395 Posted March 22, 2012 Report Share Posted March 22, 2012 Turbolapp, I just took a look at your code and this is working for me to click save: click(<innertext="Save">, "Left Click", "No") And here is your bot with it set up:change google preferences to 100.ubot HTHJustin Quote Link to post Share on other sites
turbolapp 5 Posted March 22, 2012 Report Share Posted March 22, 2012 Still the fundamental problem of not being able to scrape in other countries. Which is why Kreatus went with the image and why I would if I could. Edit: I think maybe this time I got it by making my previous solution a wild card just taking out save. (I thought I had already done this but obviously not.) anyways tried it on a russian proxy and it worked just like on the us proxy. So fingers crossed. Thanks for watching me ramble all day Quote Link to post Share on other sites
jiggi21 0 Posted January 31, 2013 Report Share Posted January 31, 2013 None of these links work, am I doing something wrong? Quote Link to post Share on other sites
VaultBoss 310 Posted January 31, 2013 Report Share Posted January 31, 2013 Assuming I would be searching results in Google for the KeyWord "dog treats", I would get a SERP (Search Engine Results Page) URL that would seem to be quite complicated with lots of parameters, depending on user agent, language selected, etc... HOWEVER, the important parameters to set the number of results per page, as well as what page exactly to appear are easy to set. By default. Google would show the first page, with 10 organic results per page, numbered from 0 upto 9. You could navigate to the corresponding search url like this: First 2 results ONLY: http://www.google.com/search?q=dog+treats&hl=en&start=0&num=2First 10 results (default):http://www.google.com/search?q=dog+treats&hl=en&start=0Results from 20 to 29 (3rd page):http://www.google.com/search?q=dog+treats&hl=en&start=20&num=10First 100 results on a single page:http://www.google.com/search?q=dog+treats&hl=en&start=0&num=100 By playing with the 'start' and 'num' parameters you can display ANYTHING you want... Hope this helps you... Quote Link to post Share on other sites
AutomationNinja 194 Posted January 31, 2013 Report Share Posted January 31, 2013 hmmmm.... I didn't get 100 results when I clicked on it http://www.google.co...start=0&num=100 Quote Link to post Share on other sites
HelloInsomnia 1103 Posted January 31, 2013 Report Share Posted January 31, 2013 Assuming I would be searching results in Google for the KeyWord "dog treats", I would get a SERP (Search Engine Results Page) URL that would seem to be quite complicated with lots of parameters, depending on user agent, language selected, etc... HOWEVER, the important parameters to set the number of results per page, as well as what page exactly to appear are easy to set. By default. Google would show the first page, with 10 organic results per page, numbered from 0 upto 9. You could navigate to the corresponding search url like this: First 2 results ONLY: http://www.google.com/search?q=dog+treats&hl=en&start=0&num=2First 10 results (default):http://www.google.com/search?q=dog+treats&hl=en&start=0Results from 20 to 29 (3rd page):http://www.google.com/search?q=dog+treats&hl=en&start=20&num=10First 100 results on a single page:http://www.google.com/search?q=dog+treats&hl=en&start=0&num=100 By playing with the 'start' and 'num' parameters you can display ANYTHING you want... Hope this helps you... I thought the same thing but I don't think this works in a new session try it in incognito mode and it wont work for example (to display the 100 results that is) Quote Link to post Share on other sites
VaultBoss 310 Posted January 31, 2013 Report Share Posted January 31, 2013 Well... I don't know why it doesn't work for you guys... It works for me. I am using Firefox, language set to English and 10 results / page (default) in the settings, but when I'm changing the num param to 100 it shows 100 results here... just as when I change to 2 (which is not even available as a value in settings) I get only 2 results. Dunno why it doesn't work for you, but it works for me (and it worked so ever since I learnt the trick a couple or more years back) Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.