meter 145 Posted June 30, 2010 Report Share Posted June 30, 2010 Hey there Winston, Started the same bot as you real fast. This is what I threw together quicklike. Should help you getting started (note this is just the beginnings of a Yellow Pages scraper).yp_sample.ubot Quote Link to post Share on other sites
meter 145 Posted June 30, 2010 Report Share Posted June 30, 2010 Update, untested. Scrapes all results for first business category into an unformatted file yp_sample.ubot Quote Link to post Share on other sites
Anewbist 18 Posted June 30, 2010 Author Report Share Posted June 30, 2010 Thanks Meter The samples that you provided are awesome…… OK where do we begin. The first step is go over the code and see what was done. But before that we need to label Each version of the YP scraper and where it came from. Take a look at all the bots that are created on YP and go over the scripts. Because they might be some things that you want to understand If you want to program them But for you business buff’s and Entrepreneur's. You have to understand what kind of work goes into the programming of the bots. So you can contract a programmer to create what you need. Now... With the advance users putting up and showing the creation that they have. You will see the person(s) that are capable of doing what you need in a short period of time. And plus the two things that I would do to help out the contributors. One is give them reputation point to help out with they ranking in the forum plus it shows the appreciation of the work that they provided. So if other members look at the members section they can view their Rank Contribution Their post To make a choice if they would like to get in contact with them to complete a project And the other is to bookmark this person as a friend request for help and contracting purposes. So until we learn the system of Ubot ourselves these are the people that we can do business with. To find out how to outsource work you can visit elance, odesk, guru, vworker and get a copy, or look over the non disclosure and contracting agreement. We give them the option first…. Why because we see the work that is being produced. And then to the general public in the ubot forum. Quote Link to post Share on other sites
Anewbist 18 Posted June 30, 2010 Author Report Share Posted June 30, 2010 OK so we code the saples for winston it will be YP scraper_winston The brit was the first to respond in the support section I have bookmarked him as a friend and contacted him for some projects. He is an understanding and pleasant person to communicate withfor the brit it will be YP Scraper_thebritfor meter it will be YP Scraper_meter1 Yp Scraper_meter2 bookmark them as friends for references for contract work and joint ventures. create folders for skeletons scripts And we now ask, and deconstruct the script so far as to how they did it any input form the creator of the script is appreciated. so we now look at deconstruction. meter have the bot created in subs. so now we look at the reason and logic behind it.and hopefully he will explain the functions behind it. the second bot that meter have created is awesome.... with this we look to compare the old coding that Aaron have. and the other yellow pages scraper to find out the difference from the new UB and the old UB.Right of the back we can see that from the video that Aaron created the slight difference UI=user interface. so we can input what we want to scrape. from picking the city to the topicC= Counter of how many pages that was scraped businesses while it is scraping. Quote Link to post Share on other sites
Anewbist 18 Posted June 30, 2010 Author Report Share Posted June 30, 2010 to the mission now is deconstruction of the sample bots.comparing it to otherslooking at the componentshow to put it togetherhow each sub work in unison with each other mapping out the processand then reconstructing the bot marking the contributor to contact them for assignments pushing the + sign at the button of the post to give them creditworking as a team to understand the process then once you get this you can apply this system to other local directories thanks meter Quote Link to post Share on other sites
Anewbist 18 Posted June 30, 2010 Author Report Share Posted June 30, 2010 We also ran into a problem with meter 2nd version of the bot An error has accrued unable to save to filecommand save to fileSub: Nav and ScrapeBot: YP sampe meter 2 Quote Link to post Share on other sites
meter 145 Posted June 30, 2010 Report Share Posted June 30, 2010 Best you highlight the sub in question. But (only guessing here) I suspect you didn't update basepath. Give me more information Quote Link to post Share on other sites
JohnB 255 Posted June 30, 2010 Report Share Posted June 30, 2010 I can only assume you spent time in the Marine Corps! http://ubotstudio.com/forum/public/style_emoticons/default/rolleyes.gifBAMCIS process Begin the Planning sketch" out the botArrange Reconvisit and plan your bots function at the siteMake Recon See how other Ubot to scrape the Yellow pages business nameaddresscitystatezipphoneemailwebsite [*]Complete Planningorganize Your bots in big subs usually.InitializeUIStartupLoopShutdownInitialize clears cookies, clears list, sets constant variables.UI defines all of the UI elementsStartup is my initial NAV, list processing, initial counter value setting etc that should be done before the main Loop.Loop includes the bulk of my bot. If the bot does not require any looping, then this sub is redundant.Shutdown takes list data and saves it, does completion NAV or pops up Javascript "finished" alert boxes and all other post loop processing before the bot stops. Issue Orderrun and test the botSupervise Quote Link to post Share on other sites
meter 145 Posted June 30, 2010 Report Share Posted June 30, 2010 ^Always faithful Quote Link to post Share on other sites
Anewbist 18 Posted June 30, 2010 Author Report Share Posted June 30, 2010 it is the second bot that you createdi renamed it yp sample meter 2.ubot Quote Link to post Share on other sites
Anewbist 18 Posted June 30, 2010 Author Report Share Posted June 30, 2010 jonb that is a yes Quote Link to post Share on other sites
JohnB 255 Posted June 30, 2010 Report Share Posted June 30, 2010 Winston, I am also trying to follow along, but I am in a Masters program for software engineering right now, and it is very time consuming...But I am keeping up on the reading for now! John Well Semper Fi then...0811 here! Quote Link to post Share on other sites
Anewbist 18 Posted July 1, 2010 Author Report Share Posted July 1, 2010 0311,0331,32, 0317, formerly 8541 Quote Link to post Share on other sites
JohnB 255 Posted July 1, 2010 Report Share Posted July 1, 2010 0331,23, 0311 ,0317, formerly 8541 For anyone following this...THAT is USMC Binary output! Quote Link to post Share on other sites
meter 145 Posted July 1, 2010 Report Share Posted July 1, 2010 Respects to both of you for serving so that soft asses like me can sit around in college all day programming bots for you @Winston: No need to upload the bot again, I was wondering what node Ubot stopped at when you tried executing my bot. A screenshot would do Quote Link to post Share on other sites
meter 145 Posted July 1, 2010 Report Share Posted July 1, 2010 @Winston: You probably need to edit this node to a valid filepath in order to get my script working on your computer: http://i49.tinypic.com/2pspq83.jpg Quote Link to post Share on other sites
Anewbist 18 Posted July 1, 2010 Author Report Share Posted July 1, 2010 ok but first let address max question......... Max Andrews it best to have many people as possible to look at you script. that's why I pointed you in the direction of the support forum. then come back here and up load it so the advance members that understand programing can look at it and to see if they can answer your questions. then we examine the other bot and compare the script to see if it have the same type of coding. and if it does, and it is better.... use it. We are not trying to reinvent the wheel in this case study. but use the best methods that work to give us the results that we want. After we get the results then we torque it a little to what you want.... Remember it is a case study that is based on Aaron s bot. I know that he is swamped in coding and projects... so we are just trying to help out and learn as we go along. and the finish product will be given back to him for his review and to do with it as he please. because He gave first of his time and of the bot. not to mention the other bots that are out dated out their.. Quote Link to post Share on other sites
Anewbist 18 Posted July 1, 2010 Author Report Share Posted July 1, 2010 ok meter so edit the node to work on my system.. ok but first I am going to run the system again it went through every thing but would not save to the file. and meter your welcome..... the programming friends that i had taught me quite a bit about the difference in people when i got out. and friendship Quote Link to post Share on other sites
meter 145 Posted July 1, 2010 Report Share Posted July 1, 2010 Hey there Winston. You should be editting the node before you run the script. Just make it point to any text file Quote Link to post Share on other sites
Anewbist 18 Posted July 1, 2010 Author Report Share Posted July 1, 2010 ok will give a shot ..... got to step away for a minute Quote Link to post Share on other sites
Anewbist 18 Posted July 1, 2010 Author Report Share Posted July 1, 2010 ok made the changes ..... seems to be running ok Quote Link to post Share on other sites
Anewbist 18 Posted July 1, 2010 Author Report Share Posted July 1, 2010 Just want to take the time out to the readers out their viewing this post the comment that i made about friendship. I spent some time down in gainesville where the university of Florida is and now santa fe college, it used to be santa fe community college. but the story goes some thing like this. I was out of touch when I got back from overseas and I did not have much or any friends around me. I meet some programmers when i was at Georgia segal hall that goes into downtown and right by the colleges.... By the way many veteran are out of touch with mainstream if they spent a year or more in a concentrated situation.long story short.... they helped me to adjust to be a little bit more human. for some of the vet's that are out their I know that you can relate. Quote Link to post Share on other sites
Anewbist 18 Posted July 1, 2010 Author Report Share Posted July 1, 2010 ok meter I made the adjustments and it worked like a charm. now we have to take a look at what you did... If you can can you walk us through the scripts that you used and in what format that you put them in. the reason and logic behind it. once we get the insight on that and understanding the programming it is time to torque it to do what we want it to do... and get the same results of the original so we need to add a few thingsUI's for search topic and geo locationUI for visual counter and we need it just to extract the name,address,city,state,zip,phone,email,website so we can import it to a excel spreadsheet or a csv file. we already got the outline all we have to do now explain the process for every one to understand the logic and the Flow that Seth talked about in the videos on the Ubotstudios site in the training section Quote Link to post Share on other sites
meter 145 Posted July 1, 2010 Report Share Posted July 1, 2010 Hey Winston, I'm finishing the GUI and CSV support as we speak. I'll add it and comments later, maybe this weekend. I have a statistics midterm this Friday that I am totally fucked for that I need to study for Cheers, -meter Quote Link to post Share on other sites
meter 145 Posted July 1, 2010 Report Share Posted July 1, 2010 Weee sample CSV Will be posting more when I get the chance. http://www.filesend.net/download.php?f=131aaff28f33b7cf7b9db588de670819 Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.