Jump to content
UBot Underground

How To Save Scrape Of Several Elements?


Recommended Posts

Hi!

I have scraped 50 member urls to a list.

 

I want to loop through each one of them and on every single url save name, email and phone number that is displayed on each member page.
 

My end goal is to be able to open them in a neat excel table.
 

I have no problem achieving the loop part but I'm not sure how to save the scrape of the name, email and phone number.

 

Should I save each element (name, email etc) to unique variables and then save them to a list? Should I create one list per data type and then combine them after the scrape? Should I somehow scrape them and put them in a table straight away?

I'm using the xpath plugin for the scrape of the elements.

 

Thanks in advance :)

Link to post
Share on other sites

use xpath to scrape the block of data that has all the info

loop thru each block and parse out the data you need and the save each each variable

and repeat until done.

Link to post
Share on other sites

use xpath to scrape the block of data that has all the info

loop thru each block and parse out the data you need and the save each each variable

and repeat until done.

 

don't think I really understand what you mean tbh...

  • Like 1
Link to post
Share on other sites
  • 2 weeks later...

Normalize the data :)

define $phone {
    set(#F_pjhone,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", $document text, "//li", "InnerText", ""),"Global")
    if($comparison(#F_pjhone,"= Equals",$nothing)) {
        then {
            return("NA")
        }
        else {
            comment("returns user name if not blank")
            return(#F_phone)
        }
    }
}
define $user name {
    set(#F_user_name,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", $document text, "//li", "InnerText", ""),"Global")
    if($is blank(#F_user_name)) {
        then {
            return("NA")
        }
        else {
            comment("returns user name if not blank")
            return(#F_user_name)
        }
    }
}
add item to list(%data,"{$user name()},{$phone()}","Don\'t Delete","Global")

$is blank is an bot bank function, sub needed

use $comparison instead if needed

Regards,

Nick

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...