smb1970

Hi, What's http get? Best regards Steve

Thanks, I don't have scrape box.

Hi, I have a list of around 20,000 domain names and I'm looking to get email addresses for them, I would imagine by reading the website and looking for a contact page. It doesn't have to be perfect a 20% hit rate would be fine. Anyone got anything? I'm prepared to swap for a working alpha state Yell scraper. Best regards Steve

Hi Jeff, That's a big ask. There are people out there marketing Yellow Pages scrapers and while mine is not robust enough to release commercially in it's current state, it is a solid and reliable bot that I've already used to obtain thousands of pages from Yell. There's really nothing to stop you taking what I've done, giving it a quick polish and releasing it yourself tomorrow. I'm happy to help out with advice if you have specific issues with your own development though. Feel free to PM me. Best regards Steve

Hi, I seem to be having a problem with the keyboard event command not always working. I'm giving focus to the application in question but issuing the keyboard event command seems to do nothing. Pressing the required key manually is working fine. Anyone else ever experience this problem? Best regards Steve

Thanks for the tip. I'll give it a go. Best regards Steve edit: I added in these lines. I also added in a clear cookies command at the end of each cycle, an extra wait for 2 seconds before each page scrape and switched it so the proxy change only happens at the end of each location scrape. Seems to work a lot better now, unfortunately it takes a little longer and I still need to bomb proof the HMA proxy switch but other than that I appear to have a sweet little Yell scraper now.

HI, I've been working on a Yellow Pages scraper the last few days. It pretty much works but the browser seems to lock out every now and then. So the click event on the Element to Click for the next page won't work for some reason. Stopping the script and trying to manually click on the web page doesn't work either, if I switch to another web site manually in the internal browser everything is fine again. It's almost as though Yell can tell they are being scraped and are somehow stopping me. Is this possible or is is potentially an issue with my code. I'm using Hide My Ass Pro and switching t

I managed to achieve this with default commands in Ubot standard. It's simply a case of using the windows commands. Here's how.

Hi, I'd be interested in knowing how you can change the IP address of Hide My Ass from within ubot please? Best regards Steve

This is a really useful video, thanks for sharing it. Are screen positions relative to the top of the program not at the mercy of the individual users settings? I think for some applications it might be better to use keystrokes rather than try and interpret where the mouse might need to be. Best regards Steve

Hi, I have come across a problem with a Yellow Pages scraper that seems only to occur when I'm using Run but does not occur when I'm using Step. Here's what I have ( attached ). So, I'm running the loop until it can't see the "Next" link for the next page, then after scraping the page and populating my lists ( via custom commands ) I click the next link, wait for the page to load and go back to the top to scan the next page. I then repeat the same process after the loop to retrieve the last page ( not shown ). Main list ( the one I'm clearing ) is the list that contains the scrape for

Thanks to everyone that provided assistance on this thread. I've subsequently managed to write my first Ubot - a Yellow Pages scraper which works nicely....until you get to page 10 and realise they are limiting results returning. Looks like I'll have to learn how to read a file and narrow down the search areas in order to scrape more data. Best regards Steve

Worked perfectly this, thanks Best regards Steve

Excellent, much appreciated it would be useful if that was offered as an alternative to the video tutorials ( for example at the top of the video tutorial page ). Best regards Steve

Thanks for the tips. Yes I can program and yes I did just jump straight in, I learn better that way, which I agree can be frustrating for the people I ask for support. My apologies if it's annoying. I have watched some of the videos but I don't consume video tuition particularly well, especially not when I would describe it as verbose. If there was a ubot book I would happily sit down and read through it in an hour or two. However watching a 20 minute video to glean information that can be conveyed in a couple of diagrams and a few lines of explanatory text isn't my idea of time well spent