Case Study

[meter 145]

Hey there Winston,

 

Started the same bot as you real fast. This is what I threw together quicklike. Should help you getting started (note this is just the beginnings of a Yellow Pages scraper).

yp_sample.ubot

[meter 145]

Update, untested. Scrapes all results for first business category into an unformatted file

yp_sample.ubot

[Anewbist 18]

Thanks Meter

 

 

 

The samples that you provided are awesome…… OK where do we begin.

 

The first step is go over the code and see what was done. But before that we need to label

 

Each version of the YP scraper and where it came from.

 

Take a look at all the bots that are created on YP and go over the scripts.

 

Because they might be some things that you want to understand If you want to program them

 

But for you business buff’s and Entrepreneur's. You have to understand what kind of work goes into the programming of the bots. So you can contract a programmer to create what you need. Now... With the advance users putting up and showing the creation that they have.

 

You will see the person(s) that are capable of doing what you need in a short period of time. And plus the two things that I would do to help out the contributors. One is give them reputation point to help out with they ranking in the forum plus it shows the appreciation of the work that they provided. So if other members look at the members section they can view their

 

Rank

 

Contribution

 

Their post

 

To make a choice if they would like to get in contact with them to complete a project

 

And the other is to bookmark this person as a friend request for help and contracting purposes.

 

 

 

So until we learn the system of Ubot ourselves these are the people that we can do business with.

 

 

 

To find out how to outsource work you can visit elance, odesk, guru, vworker and get a copy, or look over the non disclosure and contracting agreement.

 

 

 

We give them the option first…. Why because we see the work that is being produced. And then to the general public in the ubot forum.

[Anewbist 18]

OK so we code the saples

 

for winston it will be YP scraper_winston

 

The brit was the first to respond in the support section I have bookmarked him as a friend and contacted him for some projects. He is an understanding and pleasant person to communicate with

for the brit it will be YP Scraper_thebrit

for meter it will be YP Scraper_meter1

Yp Scraper_meter2

 

bookmark them as friends for references for contract work and joint ventures.

 

create folders for skeletons scripts

 

And we now ask, and deconstruct the script so far as to how they did it

 

any input form the creator of the script is appreciated.

 

so we now look at deconstruction. meter have the bot created in subs.

so now we look at the reason and logic behind it.and hopefully he will explain the functions behind it.

 

the second bot that meter have created is awesome.... with this we look to compare the old coding that Aaron have. and the other yellow pages scraper to find out the difference from the new UB and the old UB.

Right of the back we can see that from the video that Aaron created the slight difference

 

UI=user interface. so we can input what we want to scrape. from picking the city to the topic

C= Counter of how many pages that was scraped businesses while it is scraping.

[Anewbist 18]

to the mission now is deconstruction of the sample bots.

comparing it to others

looking at the components

how to put it together

how each sub work in unison with each other

mapping out the process

and then reconstructing the bot

 

marking the contributor to contact them for assignments

pushing the + sign at the button of the post to give them credit

working as a team to understand the process

 

then once you get this you can apply this system to other local directories

 

thanks meter

[Anewbist 18]

We also ran into a problem with meter 2nd version of the bot

 

 

An error has accrued

 

unable to save to file

command save to file

Sub: Nav and Scrape

Bot: YP sampe meter 2

[meter 145]

Best you highlight the sub in question. But (only guessing here) I suspect you didn't update basepath. Give me more information

[JohnB 255]

I can only assume you spent time in the Marine Corps! http://ubotstudio.com/forum/public/style_emoticons/default/rolleyes.gif

BAMCIS process

• Begin the Planning

sketch" out the bot

• Arrange Recon
  visit and plan your bots function at the site
• Make Recon
  See how other Ubot to scrape the Yellow pages
  • business name
  • address
  • city
  • state
  • zip
  • phone
  • email
  • website

  [*]Complete Planning

• organize Your bots in big subs usually.
• Initialize
• UI
• Startup
• Loop
• Shutdown
• Initialize clears cookies, clears list, sets constant variables.
• UI defines all of the UI elements
• Startup is my initial NAV, list processing, initial counter value setting etc that should be done before the main Loop.
• Loop includes the bulk of my bot. If the bot does not require any looping, then this sub is redundant.
• Shutdown takes list data and saves it, does completion NAV or pops up Javascript "finished" alert boxes and all other post loop processing before the bot stops.

• Issue Order
  run and test the bot

• Supervise

[meter 145]

^Always faithful

[Anewbist 18]

it is the second bot that you created

i renamed it

 

 

 

yp sample meter 2.ubot

[Anewbist 18]

jonb that is a yes

[JohnB 255]

Winston, I am also trying to follow along, but I am in a Masters program for software engineering right now, and it is very time consuming...But I am keeping up on the reading for now!

 

John

 

Well Semper Fi then...0811 here!

[Anewbist 18]

0311,0331,32, 0317, formerly 8541

[JohnB 255]

0331,23, 0311 ,0317, formerly 8541

 

For anyone following this...THAT is USMC Binary output!

 

 

 

[meter 145]

Respects to both of you for serving so that soft asses like me can sit around in college all day programming bots for you

 

@Winston: No need to upload the bot again, I was wondering what node Ubot stopped at when you tried executing my bot. A screenshot would do

[meter 145]

@Winston:

 

You probably need to edit this node to a valid filepath in order to get my script working on your computer:

 

http://i49.tinypic.com/2pspq83.jpg

[Anewbist 18]

ok but first let address max question.........

 

Max Andrews it best to have many people as possible to look at you script.

that's why I pointed you in the direction of the support forum.

 

then come back here and up load it so the advance members that understand programing can look at it and to see if they can answer your questions.

 

then we examine the other bot and compare the script to see if it have the same type of coding. and if it does, and it is better.... use it. We are not trying to reinvent the wheel in this case study. but use the best methods that work to give us the results that we want. After we get the results then we torque it a little to what you want....

 

Remember it is a case study that is based on Aaron s bot. I know that he is swamped in coding and projects... so we are just trying to help out and learn as we go along. and the finish product will be given back to him for his review and to do with it as he please. because He gave first of his time and of the bot. not to mention the other bots that are out dated out their..

[Anewbist 18]

ok meter

 

so edit the node to work on my system..

 

ok but first I am going to run the system again

 

it went through every thing but would not save to the file.

 

and meter your welcome..... the programming friends that i had taught me quite a bit about the difference in people when i got out. and friendship

[meter 145]

Hey there Winston. You should be editting the node before you run the script. Just make it point to any text file

[Anewbist 18]

ok will give a shot ..... got to step away for a minute

[Anewbist 18]

ok made the changes ..... seems to be running ok

[Anewbist 18]

Just want to take the time out to the readers out their viewing this post

 

the comment that i made about friendship. I spent some time down in gainesville where the university of Florida is and now santa fe college, it used to be santa fe community college.

 

but the story goes some thing like this.

 

I was out of touch when I got back from overseas and I did not have much or any friends around me. I meet some programmers when i was at Georgia segal hall that goes into downtown and right by the colleges.... By the way many veteran are out of touch with mainstream if they spent a year or more in a concentrated situation.long story short.... they helped me to adjust to be a little bit more human. for some of the vet's that are out their I know that you can relate.

 

[Anewbist 18]

ok meter

 

 

I made the adjustments and it worked like a charm.

 

now we have to take a look at what you did... If you can can you walk us through the scripts that you used and in what format that you put them in. the reason and logic behind it.

 

 

once we get the insight on that and understanding the programming it is time to torque it to do what we want it to do... and get the same results of the original

 

so we need to add a few things

UI's for search topic and geo location

UI for visual counter

 

and we need it just to extract the name,address,city,state,zip,phone,email,website so we can import it to a excel spreadsheet or a csv file.

 

we already got the outline all we have to do now explain the process for every one to understand the logic and the Flow that Seth talked about in the videos on the Ubotstudios site in the training section

[meter 145]

Hey Winston,

 

I'm finishing the GUI and CSV support as we speak. I'll add it and comments later, maybe this weekend. I have a statistics midterm this Friday that I am totally fucked for that I need to study for

 

Cheers,

 

-meter

[meter 145]

Weee sample CSV Will be posting more when I get the chance.

 

http://www.filesend.net/download.php?f=131aaff28f33b7cf7b9db588de670819