ed08724

I have noticed when I right click in the browser I can view source or view generated source. the generated source has the info I need. Is there a way to page scrape the generated source?

pash: the links don't exist in the source code to scrape, they only exist in the browser window. they are ajax.

I have a member matrix that shows 5 referrals info and there is a link to see their referrals and if their referrals have referrals then they have a link, etc. the problem is until you click every link you can't get all the info. when you click the referral link it updates the current webpage with the referrals info via ajax. I was going to scrape all the urls and process separately but nothing actually exists in the source. even the first 5 are only filled in via ajax onload. I can probably figure out a way to process the html and open every link with a new navigate but it would be much easie

pash I don't have any code yet as I have to figure out how to do it first.

HelloInsomnia: Thanks. There is a named div I can scrape. Do I just put all the html in the email list then go through and clean up the list or do I not even use lists and just iterate through each record and put each rows data directly in a table after processing it?

Just started and been through the videos. Here is my problem. I am trying to scrape data from a page with multiple records on it but not in html tables. It is member data but when they did not provide the info the complete element is missing. for example for a member that dis not provide an email address there is no mailto: to scrape between. this entire element is missing <a href="mailto:johndoe@gmail.com"> so if I scrape name, userid, email into 3 lists to make a table the lists won't match up because the name list might have 10 records but the email list only 6 records so who knows wh