MicB 1 Posted May 1, 2016 Report Share Posted May 1, 2016 I'm tired of my "new browsers" crashing and I only use them to load html from variables for scraping. Is it possible to scrape the variables without load html? I'm going to test using headless, but maybe someone here has knows how to just skip loading html altogether. Thanks! Quote Link to post Share on other sites
cүвεя_נυηкιε 68 Posted May 1, 2016 Report Share Posted May 1, 2016 I'm tired of my "new browsers" crashing and I only use them to load html from variables for scraping. Is it possible to scrape the variables without load html? I'm going to test using headless, but maybe someone here has knows how to just skip loading html altogether. Thanks! Depending what you want to scrape you can use regex to extract your data You can extract the data before you assign to a variable or you can do it by transferring to another variable, probably a few more ways you can achieve it, just think outside the box CheersCJ Quote Link to post Share on other sites
MicB 1 Posted May 1, 2016 Author Report Share Posted May 1, 2016 Thanks CJ, I'm going to look into Regex. Good idea! Quote Link to post Share on other sites
cүвεя_נυηкιε 68 Posted May 1, 2016 Report Share Posted May 1, 2016 Thanks CJ, I'm going to look into Regex. Good idea! Your welcome make sure to check out Helloinsomnia's "Regex Builder" and Brutals "Regex Cheater" (both here in the forum)both incredibly useful Have fun CheersCJ Quote Link to post Share on other sites
deliter 203 Posted May 1, 2016 Report Share Posted May 1, 2016 check out my plugin this is what it is made for,video tutorial their too, much faster and easier than writing Regex expressionshttp://network.ubotstudio.com/forum/index.php/topic/19108-offer-get-my-new-css-selector-plugin-for-free/ ayman alßo has a http post plugin that includes an xpath parser that would work too Quote Link to post Share on other sites
MicB 1 Posted May 2, 2016 Author Report Share Posted May 2, 2016 Thanks deliter! I actually have aymen's plugin I'm going to try the xpath parser, if that doesn't work I'll check out yours. Thanks a lot. Honestly, I never really understood what xpath is so I always overlook it. Quote Link to post Share on other sites
MicB 1 Posted May 2, 2016 Author Report Share Posted May 2, 2016 Is it possible to use wildcards with xpath? I'm used to scraping items like this: id=item_* where the * can be any character. Quote Link to post Share on other sites
MicB 1 Posted May 2, 2016 Author Report Share Posted May 2, 2016 Just figured it out in case someone else reads this one day. To do what I was looking for above I used: //*[starts-with(@id,'item_')] More info on xpath here: http://edutechwiki.unige.ch/en/XPath_tutorial_-_basics 1 Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.