Jump to content
UBot Underground

Data Only Pulling For A Few Lines But Script Continues To Run?


Recommended Posts

Help,

I am trying to scrape specific data from multiple pages from the same website.  I am able to get this to work, but on multiple different sites it only pulls data for the first 18-20 url's then does not pull any data but continues to cycle through all of the rest of the pages.  Also I am running into an issue where I need the blank lines either kept as nothing or a placeholder put in place, otherwise when the data is written to the table it will not line up with existing data in the table. I am really new to this and any help would be appreciated.

 

clear table(&data)
clear list(%newprice)
clear table(&url)
clear list(%url)
plugin command("Advanced Data Text File.dll", "create table from excel", "\\\\vinc\\main share\\website Pricing Data\\URL\\PricingUrllist.xlsx", &url, "Website Name", "Sheet Name", "False")
add list to list(%url,$plugin function("TableCommands.dll", "$list from table", &url, "Column", 3),"Delete","Global")
define Scrape Data {
    loop($list total(%url)) {
        navigate($next list item(%url),"Wait")
        wait(15)
        add item to list(%newprice,$find regular expression($document text,"(?<=<script id=\"cxtPixel\" type=\"text/javascript\">\\n*.*\\n*.*\\n*.*\\n*.*\\n*.*ice: \")\\d*\\.\\d*"),"Don\'t Delete","Global")
    }
    add list to table as column(&data,0,4,%newprice)
}
Scrape Data()
plugin command("TableCommands.dll", "add table to table", &data, &url)
plugin command("Advanced Data Text File.dll", "save to excel file", &data, "\\\\vinc\\main share\\website Pricing Data\\Data\\WebsitenameData.xlsx")

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...