UBotBuddy 331 Posted May 20, 2011 Report Share Posted May 20, 2011 Go to this site http://www.yell.com/ucs/UcsSearchAction.do?keywords=Plumbers&location=Manchester&scrambleSeed=18075788&searchType=advance&M=&bandedclarifyResults=&ssm=1 I have developed a new technique for scraping the company name and I want to compare my logic to other attempts. The goal is to scrape the top 13 results (including any duplicates) to a list and then save that list to a text file on the desktop. I will share my code with you if you are a Pro/Dev user. I am not certain if Std users could see my code. But you can try if you want to. Quote Link to post Share on other sites
Frank 177 Posted May 20, 2011 Report Share Posted May 20, 2011 Here you go Buddy. I'm using sockets to hide my tracks Just be forewarned that it will save the results in a folder in your my documents folder called 'botbuddy'. You'll get a few files a raw page dump, a pre-cleaned list and a cleaned list. Some of the listings don't show a url, so they come up blank. Enjoy! Ah, and you do need dev or pro to open it for sure. With the premise in this code, you could pull just about any information from each listing. Frank botbuddy-test.ubot Quote Link to post Share on other sites
JohnB 255 Posted May 20, 2011 Report Share Posted May 20, 2011 Well Frank...I got my head bit off earlier doing this with Buddy LOL! He wants just the first 13 out of the 18...(I got 10 running yours) He has truly mastered some of the advanced context menu commands... So anyhow, Buddy called me a "cheater" because I scraped 18 and removed the rest from the list leaving me with 13! lol plumbers.ubot But I stand my ground on a couple of fronts: 1) I can scrape ANY 13 using the list and math functions(odds, evens, the last 13, etc)2) This code is extremely simple and usable in all licenses John Quote Link to post Share on other sites
Praney Behl 314 Posted May 20, 2011 Report Share Posted May 20, 2011 I will give it a shot when I am on my PC sounds interesting. Been playing with sockets and almost completed google search suggestions scraper and keyword analysis bot in sockets last night, will try to implement multi-threading and to it and make it super sonic. Currently, it gets about 200 suggestions and keyword analysis in about 2 minutes, my aim is to get it down to 20-30 secs. With bot will go on the BOT Store, to say thanks to Seth and the Dev team for there efforts. Quote Link to post Share on other sites
Frank 177 Posted May 20, 2011 Report Share Posted May 20, 2011 Ah Google search with Sockets is MAGIC. Very fast and I can pull 1,000 results in literally about 10 seconds or less! Frank Quote Link to post Share on other sites
UBotBuddy 331 Posted May 20, 2011 Author Report Share Posted May 20, 2011 Frank, What can I say? You and John must be cut from the same cloth. lol I ran your bot and you were further off than John was. lol Your unclean-urls list had 8 Company Names and your clean-urls list had no Company Names. Quote Link to post Share on other sites
JohnB 255 Posted May 21, 2011 Report Share Posted May 21, 2011 I'm sorry...I thought 13 was on the money...in fact, I suggest we choose a neutral party to provide us a fresh site...You do your method, I'll do mine... I will code it, make a sandwich, watch a movie, knit a sweater, take a nap and celebrate my birthday and Christmas before you are done evaluating the source code of the web page... Deal? LOL My code is usable for every level of coder...Not only is it right on, it requires no explanation! Buddy, Buddy, Buddy... *sigh* he he (For those that are not familiar with us, this is NOT only friendly, but pretty much a daily thing for us!) John Quote Link to post Share on other sites
UBotBuddy 331 Posted May 21, 2011 Author Report Share Posted May 21, 2011 That's ok John. I realize that my statement to scrape 13 (to you) meant scrape every thing and then run it thru a loop to strip off the excess equates to my instruction. I also understand your inability to accept defeat. But that's ok. You're still my friend. Yes, we do this almost on a daily basis. lol Just so you all know the score is Buddy 100 and John 0. Quote Link to post Share on other sites
Eddie Waller 158 Posted May 21, 2011 Report Share Posted May 21, 2011 I think this should work, let me know.bothead-eddie.ubot Quote Link to post Share on other sites
UBotBuddy 331 Posted May 21, 2011 Author Report Share Posted May 21, 2011 I tip my hat to you! You sort of leap frogged what I was going to do. I would call you a showoff but I think there are some in here ganging up on me. LOL I am going to study what you did because without a doubt that is pretty cool. Quote Link to post Share on other sites
Eddie Waller 158 Posted May 21, 2011 Report Share Posted May 21, 2011 Haha at least one thing to take away from that is that if you go to the print view it is a lot easier to scrape. I kind of cheated by using javascript to get me the results, but you could do it pretty easily from that page in regulat ubot code as well. Just thought it'd be funny to give a solution with 2 nodes . Quote Link to post Share on other sites
JohnB 255 Posted May 21, 2011 Report Share Posted May 21, 2011 Geez Eddie...I just got through telling Buddy I think you could restore a '67 Shelby GT350 with nothing but javascript! Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.