Aymen 385 Posted June 14, 2013 Author Report Share Posted June 14, 2013 Ok, this method works fine for small messages. Tested it several times and is blazing fast. The only downside is the character limit. When the message is large the GET truncates it to 1024 characters. In my case (utf-8 and lot of html markup) it truncates to around 40 words... I will get rid some of my html markup code, but the ideal solution would be to work like this with POST request. Thanks again Aymen for your help. UPD: I actually tried to declare the same variable with the "http post" function and it works just fine! Pretty happy right now, and I wouldn't find the solution without Aymen's help. +rep +1 cool , i was about to tell you to change the $_GET to $_POST to send a POST requestglad you figured it out ! Quote Link to post Share on other sites
Aymen 385 Posted June 16, 2013 Author Report Share Posted June 16, 2013 Hello guysIts been a while since i released an update 2.0 Update:-added new function scrape list that will replace list scraper command with new regex and few fixes !-added new function soap request to send xml soap requests to the servers-added http container command to be used for multi threading-fixed multithreading issue when multithreading the same website for $http post and $http get functions-fixed output encoding issue for foreign websites-added alexa rank checker function-added ability to save cookies to Disk and Loading them Back For Use-fixed some issues with multiple redirections errorsVideo Demo 1 : http://screencast.com/t/c683SzQpf9VeVideo Demo 2 : http://screencast.com/t/h3R4G7FgPS : sorry about the poor voice Check your emails for download links (if you didn't get it pm me an i'll send it to you ) 4 Quote Link to post Share on other sites
Code Docta (Nick C.) 638 Posted June 16, 2013 Report Share Posted June 16, 2013 You da man Aymen!!! Thank You! Awesomeness.... Quote Link to post Share on other sites
Aymen 385 Posted June 16, 2013 Author Report Share Posted June 16, 2013 Forgot to mention there is 2 new source codes added : -Bulk PR+alexa checker-tinyurl api example Quote Link to post Share on other sites
nuc63 27 Posted June 16, 2013 Report Share Posted June 16, 2013 Great Support.Amazing product.Constantly updated! Quote Link to post Share on other sites
seopolis 0 Posted June 16, 2013 Report Share Posted June 16, 2013 Great job Aymen! Thanks! Quote Link to post Share on other sites
VaultBoss 310 Posted June 16, 2013 Report Share Posted June 16, 2013 Aymen, I must say (bromance, lol) I LOVE you(r work) man...You are truly one of the best coders I've met on this arena. Kudos, mate! 2 Quote Link to post Share on other sites
Aymen 385 Posted June 16, 2013 Author Report Share Posted June 16, 2013 Thx guys for the kind words , i really appreciate it Quote Link to post Share on other sites
Aymen 385 Posted June 16, 2013 Author Report Share Posted June 16, 2013 Hi Aymen. Thanks for the update. The output encoding issue for foreign websites was one of the most important updates that I was expecting since the release of the plugin but for some reason it still doesn't work well for me, I'm still seeing weird characters when scraping words with accented characters on this website: http://www.clasificadosonline.com/UDRealEstateDetail.asp?ID=2572375 Maybe you can have a look and see whats happening. ok i will check it out ! Quote Link to post Share on other sites
presbyran 2 Posted June 16, 2013 Report Share Posted June 16, 2013 thanks Aymen! Quote Link to post Share on other sites
zdot 14 Posted June 16, 2013 Report Share Posted June 16, 2013 great stuff Aymen. update looks good. Going to play with it tomorrow. You rock bud Quote Link to post Share on other sites
illmill 87 Posted June 17, 2013 Report Share Posted June 17, 2013 Awesome update, man. Just a quick question since I'm having trouble hearing the audio on the new videos regarding the new cookies containers. I understand that we drag the http requests all into the http container and that's how it stores the cookies for that specific thread, but what number should we use for the new cookies containers if we are running multithreading with TJ's multithreading tutorial technique? Would it just be the number of open threads, or would we include that on the loop and just enter "1" so it opens a new cookie container every time it opens a new thread? Or what???? Thanks again Quote Link to post Share on other sites
Aymen 385 Posted June 17, 2013 Author Report Share Posted June 17, 2013 Awesome update, man. Just a quick question since I'm having trouble hearing the audio on the new videos regarding the new cookies containers. I understand that we drag the http requests all into the http container and that's how it stores the cookies for that specific thread, but what number should we use for the new cookies containers if we are running multithreading with TJ's multithreading tutorial technique? Would it just be the number of open threads, or would we include that on the loop and just enter "1" so it opens a new cookie container every time it opens a new thread? Or what???? Thanks again you can insert the same #num threads variable to the new cookies containers command Quote Link to post Share on other sites
Aymen 385 Posted June 17, 2013 Author Report Share Posted June 17, 2013 i just recorded a new video for the new functions in the plugin : http://screencast.com/t/FvKyHO2okE 1 Quote Link to post Share on other sites
iDollarsteam 13 Posted June 17, 2013 Report Share Posted June 17, 2013 Can you add the option to insert our own regex code in the $scrape list function?? this will give us freedom to use this function with almost any document.Thx! Quote Link to post Share on other sites
Aymen 385 Posted June 18, 2013 Author Report Share Posted June 18, 2013 Can you add the option to insert our own regex code in the $scrape list function?? this will give us freedom to use this function with almost any document.Thx! You mean instead of the dropdown , i add a text field so you can add your own regex code ?if thats what you mean that can be managed Quote Link to post Share on other sites
orbital 15 Posted June 18, 2013 Report Share Posted June 18, 2013 I am getting this error now after the update: http://www.screencast-o-matic.com/screenshots/u/I90/1371521082858-52811.png Quote Link to post Share on other sites
illmill 87 Posted June 18, 2013 Report Share Posted June 18, 2013 You mean instead of the dropdown , i add a text field so you can add your own regex code ?if thats what you mean that can be managedOr just add "regex" as one of the drop down options so we can use our own regex if we want. I like that you already have some regex programmed in for emails, phone numbers, etc., but it would be nice to have the flexibility to do our own regex in addition to this. EDIT: Quick question and rather than make a new post, thought I would just add it here: how do I set the proxy credentials for the http post command? It keeps giving me an error that proxy authentication is required. Here's what I have: EDIT2: Nevermind on the proxy user and pw issue. I decided to read through all the pages on this thread and found the answer on something like page 7. 1 Quote Link to post Share on other sites
Code Docta (Nick C.) 638 Posted June 18, 2013 Report Share Posted June 18, 2013 I know you can add/drag a variable in that drop down. Maybe you can add regex in a variable and see if it works. Aymen will that work?You mean instead of the dropdown , i add a text field so you can add your own regex code ?if thats what you mean that can be managed Quote Link to post Share on other sites
iDollarsteam 13 Posted June 18, 2013 Report Share Posted June 18, 2013 You mean instead of the dropdown , i add a text field so you can add your own regex code ?if thats what you mean that can be managedFor me this is the best option but may be you can add in the dropdown the option "custom" and in the body of the function a textfield, when the user select a predefined option the textbox will be readonly and when the user select "custom" the textbox will be editable and the user can insert the regex code that he want Quote Link to post Share on other sites
Aymen 385 Posted June 18, 2013 Author Report Share Posted June 18, 2013 I know you can add/drag a variable in that drop down. Maybe you can add regex in a variable and see if it works. Aymen will that work? i can add another function called $regex to list I am getting this error now after the update: http://www.screencast-o-matic.com/screenshots/u/I90/1371521082858-52811.png when did you get this error ? Quote Link to post Share on other sites
Aymen 385 Posted June 19, 2013 Author Report Share Posted June 19, 2013 Affiliate Program is open : http://ubot-plugins.com/affiliate-program/ Quote Link to post Share on other sites
orbital 15 Posted June 19, 2013 Report Share Posted June 19, 2013 when did you get this error ? I have two $http get functions: set(#data, $plugin function("HTTP post.dll", "$http get", "http://**************/forbot/tobechecked.php", "", "", "", ""), "Global") plugin command("DatabaseCommands.dll", "connect to database", "server=xx.xx.xx.xx;uid=xxxxxxxxx; pwd=xxxxxxxx; database=xxxxxxx; port=3306; pooling=false") { loop($list total(%asin)) { progress() set(#delay, $rand(2, 5), "Global") set(#current asin, $next list item(%asin), "Global") set(#amazon data, $plugin function("HTTP post.dll", "$http get", "http://www.amazon.com/dp/{#current asin}", "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:21.0) Gecko/20100101 Firefox/21.0", "", "", ""), "Global") in 1.8 it works fine. I don't need it store cookies or anything but then when I load up 2.0 when it hits the set(#amazon data, $plugin function(.....) it breaks with that error, so I just loaded up 1.8 and shipped the bot. I would like to know though what I need to do to make this work. Quote Link to post Share on other sites
Aymen 385 Posted June 20, 2013 Author Report Share Posted June 20, 2013 I have two $http get functions: set(#data, $plugin function("HTTP post.dll", "$http get", "http://**************/forbot/tobechecked.php", "", "", "", ""), "Global") plugin command("DatabaseCommands.dll", "connect to database", "server=xx.xx.xx.xx;uid=xxxxxxxxx; pwd=xxxxxxxx; database=xxxxxxx; port=3306; pooling=false") { loop($list total(%asin)) { progress() set(#delay, $rand(2, 5), "Global") set(#current asin, $next list item(%asin), "Global") set(#amazon data, $plugin function("HTTP post.dll", "$http get", "http://www.amazon.com/dp/{#current asin}", "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:21.0) Gecko/20100101 Firefox/21.0", "", "", ""), "Global") in 1.8 it works fine. I don't need it store cookies or anything but then when I load up 2.0 when it hits the set(#amazon data, $plugin function(.....) it breaks with that error, so I just loaded up 1.8 and shipped the bot. I would like to know though what I need to do to make this work. ok i can see the issue it will be fixed soon ! a solution would be to put all the http requests inside a http container command!then create a command before calling the http container plugin command("HTTP post.dll", "new cookies containers", 1) if you are not multi threading ! This will work too without causing an error Quote Link to post Share on other sites
Aymen 385 Posted June 23, 2013 Author Report Share Posted June 23, 2013 Update : 2.1fixed error of enable to cast objectfixed some multi threading issues with http container PS: some sites like twitter will log you in to the same session if they detect that you are using the same user agent and the same ip , so make sure to use different user agents and anonymous proxies ! PPS : more work will be done on the http container to make sure there is no other issues ! 1 Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.