[Scraper Program]

Hi, I bought Ubot standard edition, back in 2014 I think. Even though Ubot is supposed to be for the non-programmer, it always seemed too complicated to me, so I have never really used it. But I would like to try again. I need to replace a scraper program. I still have the setup files and the license key, but since the dev has gone out of business, it can't phone home to verify the license. And the computers I have it on have died. 

 

I have been watching some of the how-to videos on youtube, but I still don't have a clue what to do. I'm hoping that someone can sort of give me an outline of which functions (is that the right word?) I need to work with. Here is what it needs to do:

 

First, I have a list of urls, in a text file. One per line. In this case, they are redirects, so I need to save the original url, and get the url that it redirects to, and also save that. 

 

Next, I need to get things off the page source. The old scraper program (Happy Harvester) would get the text between x and y. Each thing you wanted to save was added as a "rule". For example, the text between <title> and </title>. Or the text between "<a href=" and ">Contact</a>" (which would give you the url to their contact page - if it existed). 

 

The program would save all this info in a csv file. 

 

I've seen the Ubot page scraping functions, but they seem to work on the live side and not the source side. 

 

I'm not asking for a total detailed how-to, but hoping someone can tell me to use "this" for my list of urls, and "this" to save the 2 url infos, and "this" to get to the page source, and "this" to save the various texts between x and y. Just sort of an outline. And then I can hopefully watch the videos and read the tutorials to figure out the rest. 

 

Really appreciate any help! 

[[Sell] Yelp Leads Scraper Bot]

Does your program get the businesses email address?

[Where do I start?]

#1 how are you installing these sites #2 are you installing the basic wp or modifying it list goes on and on?

 

 

Break it down into parts.

An easy one to start with to load a wp login/pass and login into wordpress and verify your logged in.

 

I did that! (went to the page and logged in - yay!). Actually a bit more than that.

 

The sites are already installed and a theme uploaded and activated. I want to be able to choose one site from a dropdown list and have the bot go to the log in page and login. I've read all the commands and some of the functions. Although I'm quite confused, I think I can see that there are several ways to do it.

 

I do want to log in to the first site in the list and then go through a list of things to set that site up, and then go to the next one, login, etc... But I first just want to login, and then run the set up bot (a separate UI button?), but be able to use the logins and NOT run the setup parts, maybe just do other things on the site.

 

I'll get there

[Where do I start?]

Thanks for the advice. I did print out the help files (had to go buy a new ink cartridge!) and will read them whenever I get a chance. I would rather read than watch videos, cuz I can go at my own speed - I get impatient watching videos! And thanks for the ideas, that is what I was looking for, rather than a step-by-step answer. I really want and need to master this software, but at the same time, don't have a lot of extra time to give to it. Catch 22?

[Where do I start?]

I thought maybe someone could point me to which video would cover this. Is there a place online to print out the commands and parameters/functions all at once, rather than page-by-page through the help guide?

[Where do I start?]

Hi, I am just getting started. I want to set up some WP sites, all pretty much the same way, so I'm thinking maybe there could be a dropdown box for the urls to navigate to, and for the usernames and passwords connected to them.

 

Where would I start with this? Is there a certain tutorial vid that addresses it?

 

Thanks!  

[Saving lines to new text file]

Thanks VaultBoss, but I don't think that would work for me. The file is over 10g in size, so I wouldn't want to overload any of my servers by working with it, and I don't have php/mysql/etc set up on any of my computers. Wouldn't there be a way to loop thru the file and create a new file with the lines that have the "abc" at the certain positions?

[Saving lines to new text file]

I have this huge stock file, it has about 1 1/2 million lines, which is more than my database can handle (MS Access), so I have to pull only the lines I really need.

 

What I need is something like: if position 283 thru 286 =  abc, then save the line to a new text file.

 

How would I go about this?

 

Thanks!