Jump to content
UBot Underground

How Do I Add A List To A Table With Missing Values?


Recommended Posts

I am attempting to scrape data from multiple website pages into a list, then add that list to a table.  The problem I am having is that some items are missing from some pages, the bot then goes to the next page and continues scraping.  Then when the list is posted to the table those missing values causes the data not to line up in the correct row as the missing values are not skipped in the table.  So the question is, how do I put in a placeholder for the spots when I scrape something that is not actually there.  If you can't tell, this is completely new to me.  Thanks for your help.

 

define Scrape Data {
    navigate("http:........)
    add list to list(%url,$scrape attribute(<class="a-link-normal s-access-detail-page  a-text-normal">,"fullhref"),"Delete","Global")
    add list to table as column(&data,0,0,$scrape attribute(<class="a-link-normal s-access-detail-page  a-text-normal">,"fullhref"))
    loop($list total(%url)) {
        navigate($next list item(%url),"Wait")
        add list to list(%title,$scrape attribute(<id="productTitle">,"innertext"),"Delete","Global")
        add list to list(%model,$scrape attribute(<innertext=w"Model Number: *">,"innertext"),"Delete","Global")
        add list to list(%price,$scrape attribute(<id="priceblock_ourprice">,"innertext"),"Delete","Global")
        add list to list(%rating,$scrape attribute(<class="reviewCountTextLinkedHistogram noUnderline">,"innertext"),"Delete","Global")
        add list to list(%ratingtotal,$scrape attribute($element offset(<id="acrCustomerReviewText">,0),"innertext"),"Delete","Global")
    }
    add list to table as column(&data,0,1,%title)
    add list to table as column(&data,0,2,%model)
    add list to table as column(&data,0,3,%price)
    add list to table as column(&data,0,4,%rating)
    add list to table as column(&data,0,5,%ratingtotal)
}
Scrape Data()

Link to post
Share on other sites

Hello.

 

I would first change Delete to Don't delete. Otherwise the list command will remove all duplicates. That's also the case for the empty fields.

I'm not 100% sure if the scrape attribute command will return empty elements as well. 

It could be, that the command is not able to do that. But that's something I would have to test. Don't know that out of my head at the moment.

 

But first give that Don't delete option a try. Maybe that already is the solution for you.

 

PS: If you have the HTTP Plugin, you can also take a look at my Free Scrape Framework. Link is i my signature.

 

Dan

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...