daveconor 1 Posted April 9, 2014 Report Share Posted April 9, 2014 Hi fellas, I'm trying to scrape a list of websites for a specific word, say "test drive" for example. Is it possible for ubot to extract a particular's site home page html code without having to go there and then scrape the <body> tag which takes a relatively long time. thank in advance Quote Link to post Share on other sites
Kreatus (Ubot Ninja) 422 Posted April 9, 2014 Report Share Posted April 9, 2014 Yes, you gonna need http post plugin for that. Quote Link to post Share on other sites
Ptrick125 45 Posted April 9, 2014 Report Share Posted April 9, 2014 Aymen's http post plugin $set then you put $http get Quote Link to post Share on other sites
Pete 121 Posted April 9, 2014 Report Share Posted April 9, 2014 OR can use read from file just pop the site url inside the read form file 2 Quote Link to post Share on other sites
Kreatus (Ubot Ninja) 422 Posted April 9, 2014 Report Share Posted April 9, 2014 OR can use read from file just pop the site url inside the read form fileNice! i forgot that's even possible. Quote Link to post Share on other sites
UBotDev 276 Posted April 9, 2014 Report Share Posted April 9, 2014 Yep, as Zap said, you can you "read file" command if you only need to send GET requests. The code would look like this: set(#TEXT, $find regular expression($read file("http://www.ubotstudio.com/forum/index.php?/topic/16237-is-it-possible-to-extract-a-pages-html-without-having-to-go-it-physically/"), "test drive"), "Global") Quote Link to post Share on other sites
a2mateit 395 Posted April 10, 2014 Report Share Posted April 10, 2014 OR can use read from file just pop the site url inside the read form file Nice one zap. Learn something new everyday Quote Link to post Share on other sites
UBotDev 276 Posted April 10, 2014 Report Share Posted April 10, 2014 You should also know that this command actually uses UBot's "browser.exe" browser to get the content. On one side this is is a bad thing because of all problems related to UBot browser... ...but on the other side this allows you to easily get the content for which you have to log in, since you just do the log in to specific site inside UBot browser as you normally would (navigate->change attribute->click...) and then you can just use the read command to get the content for which authentication/authorization is required. I was actually using this as alternative to "http post" plugin to make "hybrid" bots with UBot native commands, which are running much faster. Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.