Pete 121 Posted June 17, 2010 Report Share Posted June 17, 2010 Would it be possible to tap into the search suggestion drop down the big 3 are all using now days? It seems to send each keystroke Quote Link to post Share on other sites
alcr 135 Posted June 17, 2010 Report Share Posted June 17, 2010 What big 3? the search suggestion drop can be scraped, yes. Quote Link to post Share on other sites
Pete 121 Posted June 18, 2010 Author Report Share Posted June 18, 2010 What big 3? google yahoo bing The google/bing search suggestion drop down box is not a standard dropdown as you type your search so it sends the keystrokes one by one and updates the suggestions for each keystroke. Type ubot and the suggestions are (ubot, ubot for kids, ubot game,and so on)However type ubot and just add the character d and get (ubot download, ubot discount, developer) I would like grab all those words Yes you can choose the attribute for the drop down box but how to add it to a list1. The box will disappear before you add it to a list2. The box updates on the fly as you type so how to pin point results after you have typed in your keyword and then added a character Quote Link to post Share on other sites
Kreatus (Ubot Ninja) 422 Posted June 18, 2010 Report Share Posted June 18, 2010 Use this tool http://tools.seobook.com/general/keyword-information/ you can scrape the keywords there. though its for google suggest only . Quote Link to post Share on other sites
Pete 121 Posted June 18, 2010 Author Report Share Posted June 18, 2010 Thanks for the reply Kreatus But that’s the problem the whole world is using adwords keyword tool and you have little control with it I think alcr reply was incorrect as I believe he did not check the type of dropdown box in question And was referring to a run of the mill dropdown box Ubot is by designed to manipulate webpage’s but it seems this could be beyond ubot This is very significant as it would open up a lot more possibilities in data mining Quote Link to post Share on other sites
adamv 0 Posted June 19, 2010 Report Share Posted June 19, 2010 Jon Leger is coming out with a keyword tool in a couple of days that will scrape keywords from the suggestions the big three search engines give. I have no idea what the cost will be yet and I would also be interested in learning if ubot is capable of scraping that data. Quote Link to post Share on other sites
Frank 177 Posted June 20, 2010 Report Share Posted June 20, 2010 I too would like to learn how to do just this. I cannot seem to find that code that is firing that box. Hints, tips, ideas anyone? Frank Quote Link to post Share on other sites
Joe118 3 Posted June 20, 2010 Report Share Posted June 20, 2010 I'm very close to releasing a product based on ubot that does exactly what Zap describes. It will be a WSO in the warrior forum, so you can learn about it there. Quote Link to post Share on other sites
The_Brit 13 Posted June 21, 2010 Report Share Posted June 21, 2010 I created a bot this morning to see if it was doable. Anyone want to buy it? Dave EDIT: I can provide a demo of it, restricted to returning only one of the items in the list. I'll have to email it as the compiled version is too big to upload here. Results are in c:\google_search_suggestions.txt. Quote Link to post Share on other sites
Hugo 2 Posted June 22, 2010 Report Share Posted June 22, 2010 I too would like to learn how to do just this. I cannot seem to find that code that is firing that box. Hints, tips, ideas anyone? Frank yeah, I don't know nearly enough about javascript to try to decipher through G's source, but experimenting this evening, I was able to get the suggestion box to fire by using "send keys chosen" - which isn't very effective as that command sends every keystroke 3 times, but not by using "send keys field chosen" any idea why the two commands behave so differently? -Hugo Quote Link to post Share on other sites
Hugo 2 Posted June 22, 2010 Report Share Posted June 22, 2010 Replying to my own post is weird, but I just tried something that worked, so I figured I'd share: This works on Google - After you "send keys field chosen" (I bet change chosen attribute would work as well but it's untested) to enter the search string you're looking for, "send keys chosen" and set the text to "$special keys: down" Then the query will look like "[search Term]\\" and the resulting suggestion box will give you the same result that you get in a browser using "[search Term] ". Still haven't figured out how to select and scrape the resulting suggestion box. Anyone have any ideas??? Thanks, Hugo Quote Link to post Share on other sites
Pete 121 Posted June 22, 2010 Author Report Share Posted June 22, 2010 Sounds to me like you need to know a fair amount of javascript to pull this off Quote Link to post Share on other sites
Hugo 2 Posted June 22, 2010 Report Share Posted June 22, 2010 Sounds to me like you need to know a fair amount of javascript to pull this off Nope. Find a working sample attached. I'm sure it's not nearly as functional as what some people here have built, but it works. (Note: Right now it only returns the first ten suggestions. You'll have to modify it and add loops etc. to make it into a viable keyword research tool) Hope this helps -Hugo P.s. I see that there are people selling working bots. Those are likely better than what I've provided. I am simply sharing my bot sample to help people who are looking to create bots for their PERSONAL USE. PLEASE do not take the sample I am providing and use my work to create a commercial bot to sell. Thankssuggestions-test.ubot 2 Quote Link to post Share on other sites
alcr 135 Posted June 22, 2010 Report Share Posted June 22, 2010 Nope. Find a working sample attached. I'm sure it's not nearly as functional as what some people here have built, but it works. (Note: Right now it only returns the first ten suggestions. You'll have to modify it and add loops etc. to make it into a viable keyword research tool) Hope this helps -Hugo P.s. I see that there are people selling working bots. Those are likely better than what I've provided. I am simply sharing my bot sample to help people who are looking to create bots for their PERSONAL USE. PLEASE do not take the sample I am providing and use my work to create a commercial bot to sell. ThanksNice share Hugo! That was precisely what I was talking about earlier. Quote Link to post Share on other sites
Pete 121 Posted June 22, 2010 Author Report Share Posted June 22, 2010 Nice one Hugo Simple solution is always the best there was me playing about with JavaScript Visibility And yes it works in a loop Thanks for all your work Quote Link to post Share on other sites
Kreatus (Ubot Ninja) 422 Posted June 22, 2010 Report Share Posted June 22, 2010 Nice one hugo! Keep it up! Quote Link to post Share on other sites
Frank 177 Posted June 23, 2010 Report Share Posted June 23, 2010 Darn! I didn't know about the special keys that you can send, as a keydown! I had all the rest figured out and was stuck trying to get that stupid box to show up. I had resolved that I'd have to pick through the google cryptic js. Thanks very much for that! Helps me in a lot of places. Frank Quote Link to post Share on other sites
Frank 177 Posted June 26, 2010 Report Share Posted June 26, 2010 Here's a stronger challenge for you all. Try to scrape the keyword suggestions out of yahoo! The code on the page is obsured and changes so scraping for me has been pretty much impossible so far. But I shall not give up!! Frank Quote Link to post Share on other sites
The_Brit 13 Posted June 26, 2010 Report Share Posted June 26, 2010 I dodn't go through the code on the samples above, but you can achieve getting the drop-down to appear in Google by sending a space after your keyword as well. I used this method some time ago to scrape keywords from the suggestion list and as far as I know it still works. Quote Link to post Share on other sites
The_Brit 13 Posted June 26, 2010 Report Share Posted June 26, 2010 Here's a stronger challenge for you all. Try to scrape the keyword suggestions out of yahoo! The code on the page is obsured and changes so scraping for me has been pretty much impossible so far. But I shall not give up!! FrankDid you try page scraping using /STRONG> for the left and </A></LI> for the right and then just combine your search term with a space before the word you scraped? I tried it and it worked for me. Dave EDIT: You need to have LI as the starting attribute for the page scrape and not not A or STRONG which are a couple of the choices I have seen it default to. Quote Link to post Share on other sites
Frank 177 Posted June 28, 2010 Report Share Posted June 28, 2010 Did you try page scraping using /STRONG> for the left and </A></LI> for the right and then just combine your search term with a space before the word you scraped? I tried it and it worked for me. Dave EDIT: You need to have LI as the starting attribute for the page scrape and not not A or STRONG which are a couple of the choices I have seen it default to. Hey Dave, I tried just about everything and came up with nothing. Not sure if it's a ubot glitch or a very well crafted webpage. Frank Quote Link to post Share on other sites
Biks 9 Posted July 1, 2010 Report Share Posted July 1, 2010 I'm interested in knowing if UBot can scrape the suggested keywords from the Google keyword tool along with Local search volumes per month. Then filter out the ones that are below a certain threshold. (below 3,000 per month) Is this what you guys were talking about? Quote Link to post Share on other sites
Frank 177 Posted July 5, 2010 Report Share Posted July 5, 2010 Hey Biks, the next step would be to head over to the external keyword research tool from google and do just that. Pump the keywords in and pull out volume keywords. Frank Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.