V4 White Pages Scrape Grabbing Too Much

[Grey Hat 4]

Hi,

 

So I've upgraded to Ubot 4.0.66, I'm doing a scrape on the white pages. I want to grab the business name, mailing address, and phone number. It's all there for me, BUT, when I do the scrape I'm having trouble surpressing what Ubot grabs. It's grabbing everything because the only way I've been able to grab is "innertext" and then wildcard everything that the initial "<>" grabbed in the search box.

 

An example of my scraped list is attached. You'll see it repeats business name 3 times on each entry and that's not good. Then toward the end of the list it starts spitting out U.S. cities en mass and then paragraphs of garbage.

 

I thought of doing an If/Then string but the problem is how do you do an If/Then string if the data grab is out of a catch-all wildcard. I looked at UBot V3 and got lost in translation.

 

I've included my bot as well. It's been all day. I've tried %list, &Tables, ugggghhhh. I took a break and watched my football team lose. Bad day all around. Can someone please help?

 

Thanks!

 

Category criteria in the UI is :"Psychics" City: "Sedona" State: "AZ"

 

WhitePageScrape.txt

WP_Scrapper.ubot

[LoWrIdErTJ - BotGuru 904]

your scrape attribute is basically telling it to scrape everything.

 

$scrape attribute(<innertext=w"*">, "innertext")

 

that tells it so long as inner text is anything scrape it.

[JohnB 255]

This works, give it a try and see what you think.

 

clear table(&businesses)

ui text box("Category for search", #Search Category)

ui text box("City", #city)

ui text box("State", #State)

clear cookies

navigate("http://www.whitepages.com/", "Wait")

type text(<name="key">, #Search Category, "Standard")

type text(<id="bus_location">, "{#city}, {#State}", "Standard")

wait(2)

click(<id="business_submit">, "Left Click", "No")

wait(7)

clear list(%businesses)

add list to list(%businesses, $scrape attribute(<class="result_name">, "outertext"), "Delete", "Global")

clear list(%names)

add list to list(%names, $scrape attribute(<class="address_container">, "outertext"), "Delete", "Global")

add list to table as column(&businesses, 0, 0, %businesses)

add list to table as column(&businesses, 0, 1, %names)

save to file("{$special folder("Desktop")}/businesses.csv", &businesses)

 

 

John