Abeeck 0 Posted June 10, 2016 Report Share Posted June 10, 2016 Help,I am trying to scrape specific data from multiple pages from the same website. I am able to get this to work, but on multiple different sites it only pulls data for the first 18-20 url's then does not pull any data but continues to cycle through all of the rest of the pages. Also I am running into an issue where I need the blank lines either kept as nothing or a placeholder put in place, otherwise when the data is written to the table it will not line up with existing data in the table. I am really new to this and any help would be appreciated. clear table(&data)clear list(%newprice)clear table(&url)clear list(%url)plugin command("Advanced Data Text File.dll", "create table from excel", "\\\\vinc\\main share\\website Pricing Data\\URL\\PricingUrllist.xlsx", &url, "Website Name", "Sheet Name", "False")add list to list(%url,$plugin function("TableCommands.dll", "$list from table", &url, "Column", 3),"Delete","Global")define Scrape Data { loop($list total(%url)) { navigate($next list item(%url),"Wait") wait(15) add item to list(%newprice,$find regular expression($document text,"(?<=<script id=\"cxtPixel\" type=\"text/javascript\">\\n*.*\\n*.*\\n*.*\\n*.*\\n*.*ice: \")\\d*\\.\\d*"),"Don\'t Delete","Global") } add list to table as column(&data,0,4,%newprice)}Scrape Data()plugin command("TableCommands.dll", "add table to table", &data, &url)plugin command("Advanced Data Text File.dll", "save to excel file", &data, "\\\\vinc\\main share\\website Pricing Data\\Data\\WebsitenameData.xlsx") Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.