[SELL] HTTP POST plugin + Crazy Bonuses Inside

[zharfan 0]

i check in the debugger when scrape google web page appear that massage

[Aymen 385]

IndexOutOfRangeException 

It's not happening all the time. Mostly when I stop and start the bot multiple times. 

I was not able to fully isolate the issue yet. But I'm not doing anything fancy within the cookies container. 

 

Only using http get and xpath parser index

Within a http container. 

 

No other commands are used from your plugin.

 

 

 

Dan

 

If you could reproduce the error or narrow it down that would be helpful

 

Hey bro if there error massage when i use $http get "The remote server returned an error: (503) Server Unavailable." what the soluton?

can you post the exact code here or pm it to me ?

[Bot-Factory 602]

Aymen,

 

what is http cookie response doing? Is that the cookie string that is also used for serializing?

 

Thanks

Dan

[Aymen 385]

Aymen,

 

what is http cookie response doing? Is that the cookie string that is also used for serializing?

 

Thanks

Dan

 

that's the cookies string from the latest http response

 

Regards

[Bot-Factory 602]

that's the cookies string from the latest http response

 

Regards

 

So within a http container this should be exactly what the plugin should use for it's next http request right? 

 

Dan

[Aymen 385]

yea , the plugin handles cookies automatically for you , in some cases some sites don't send all the cookies required because of excessive redirections , to bypass that , disable redirections and send cookies manually!

[Bot-Factory 602]

Aymen,

 

the error I get with my scraper when using http container is:

"object reference could not set to an instance"

 

Kev123 has a similar issue. But we are both not 100% sure where this comes from. 

If it's an issue in the http container, or some strange plugin stuff when we use it in combination with other plugins. 

 

Would be great if you could look into that when you work on the other issues. 

 

Thanks in advance for your help
Dan
 

[kev123 132]

Aymen,

 

the error I get with my scraper when using http container is:

"object reference could not set to an instance"

 

Kev123 has a similar issue. But we are both not 100% sure where this comes from. 

If it's an issue in the http container, or some strange plugin stuff when we use it in combination with other plugins. 

 

Would be great if you could look into that when you work on the other issues. 

 

Thanks in advance for your help

Dan

 

aymen the issue is with http get it looks like it returns nothing hence the error on the set command i'll create some isolated code when I get chance.

[Aymen 385]

aymen the issue is with http get it looks like it returns nothing hence the error on the set command i'll create some isolated code when I get chance.

 

Can you poste a code snippet ? or steps to reproduce the error ?

[kev123 132]

Can you poste a code snippet ? or steps to reproduce the error ?

. Yeah I'll make something up tomorrow

[Bot-Factory 602]

Not sure if it's possible at all, but one killer feature would be to share sessions / cookies between browser and http plugin.

Maybe with session IDs, so that we could switch between different sessions. And load / save session and share them between plugin and browser.

 

Just dreaming... :-)

 

Dan

[jamesfar 15]

Not sure if it's possible at all, but one killer feature would be to share sessions / cookies between browser and http plugin.

Maybe with session IDs, so that we could switch between different sessions. And load / save session and share them between plugin and browser.

 

Just dreaming... :-)

 

Dan

 

++ 1 for this

 

it would be awesome if we can switch between http and browser

[Bot-Factory 602]

Cookie Handling:

 

The whole cookie handling still confuses me.  Aymen, it would be great if you could explain that in detail.

 

• Are cookies sometimes shared with the ubot browser? Or are they always stand alone?
• Why are cookies downloaded to the cookies folder? When are they used?
• Are cookies within a http container unique? 
• Why are files saved to the cookies folder when I use http container? How are they separated from each other? (seems to only happen with v5)

 

 

 

I have done some additional tests. And there is definitely something wrong with UBot V5 and the plugin.

 

Even in V4 I sometimes get an error like:

the given key was not in the dictionary

 

when I use:

 alert($plugin function("HTTP post.dll", "$http cookie response"))

 

 

But in v4 the login always works. In v5 it doesn't work. Sometimes I'm logged in, sometimes not. 

Sometimes the cookie response is empty, sometimes I get that dictionary error and sometimes it contains a cookie string.

 

 

I will send you a test script via PM so that you can reproduce the issue.

 

Dan

[Bot-Factory 602]

Also strange:

 

in v5 it always writes files to c:\cookies

 

in v4 it doesn't do that at all?

 

Dan

[Marton 48]

• Why are cookies downloaded to the cookies folder? When are they used?
• Are cookies within a http container unique? 
• Why are files saved to the cookies folder when I use http container? How are they separated from each other?

 

I would like to learn about these too. Thanks.

[Marton 48]

Hi guys,

 

There seems to be demand for a working DeathByCaptcha source code (the one in the bonus package is not a full solution).

 

DeathByCaptcha doesn't automatically return the answer to the CAPTCHA. As explained in the API, you need to "poll" for the uploaded CAPTCHA status manually (it's not as simple as with Decaptcher for example), so you need to make sure you give enough time for them to actually solve your CAPTCHA before the final GET request. You can make a loop and check every 5 seconds (wouldn't use a much lower value, because you could get banned for too many checks).

 

I quickly updated the "DeathByCaptcha API.ubot" script to make it work (source code attached).

Example image to test it with: http://content.screencast.com/users/urban.marton/folders/Camtasia%20Studio/media/bfd5ee9f-81a4-4b6f-acd6-2d15a9032764/example_captcha.jpg

 

I added a 10+5 second wait before the first check, because in my experience it takes at least 15 seconds for DeathByCaptcha to give an answer. After that the script checks for an answer every 5 seconds, up to a maximum of 20 tries (I added this just so there is a timeout).

 

I also added a "Check Balance" button for you guys.

 

Hope this helps,

Marton

p.s.: Aymen feel free to include this in your bonus package if you like it

DeathByCaptcha_byMarton.ubot

[Aymen 385]

Cookie Handling:

 

The whole cookie handling still confuses me.  Aymen, it would be great if you could explain that in detail.

 

• Are cookies sometimes shared with the ubot browser? Or are they always stand alone?
• Why are cookies downloaded to the cookies folder? When are they used?
• Are cookies within a http container unique? 
• Why are files saved to the cookies folder when I use http container? How are they separated from each other? (seems to only happen with v5)

 

 

 

I have done some additional tests. And there is definitely something wrong with UBot V5 and the plugin.

 

Even in V4 I sometimes get an error like:

the given key was not in the dictionary

 

when I use:

 alert($plugin function("HTTP post.dll", "$http cookie response"))

 

 

But in v4 the login always works. In v5 it doesn't work. Sometimes I'm logged in, sometimes not. 

Sometimes the cookie response is empty, sometimes I get that dictionary error and sometimes it contains a cookie string.

 

 

I will send you a test script via PM so that you can reproduce the issue.

 

Dan

 

I just want to state that the http post plugin is built for the v4 not for the v5 , i've never tested it with the v5 nor used it with the v5 , my initial tests with the v5 showed that the threading system in both v4 and v5 are built differently as some of the http post functions (http last response , http cookies response etc...) requires the thread ID to keep data unique to each thread , looks like the plugin is not capturing the thread id with the V5!

 

about cookies , cookies are unique per session , if you open multiple sessions in different threads , you will see different cookies

And yes cookies are supposed to be unique per http container and thread (if you are opening different threads and using different data of course)

 

And finally , there is an option to save cookies files locally (exactly like CURL does) instead of in memory , if you specify a folder location in the save cookies to disk command then the cookies will be saved to disk instead of in memory! the folder shouldn't show up if you choose not to save to folder

 

Keep in mind that what i just said is related to V4 not V5!

the time when V5 becomes stable i'll make a patch for it!

[Bot-Factory 602]

I just want to state that the http post plugin is built for the v4 not for the v5 , i've never tested it with the v5 nor used it with the v5 , my initial tests with the v5 showed that the threading system in both v4 and v5 are built differently as some of the http post functions (http last response , http cookies response etc...) requires the thread ID to keep data unique to each thread , looks like the plugin is not capturing the thread id with the V5!

 

about cookies , cookies are unique per session , if you open multiple sessions in different threads , you will see different cookies

And yes cookies are supposed to be unique per http container and thread (if you are opening different threads and using different data of course)

 

And finally , there is an option to save cookies files locally (exactly like CURL does) instead of in memory , if you specify a folder location in the save cookies to disk command then the cookies will be saved to disk instead of in memory! the folder shouldn't show up if you choose not to save to folder

 

Keep in mind that what i just said is related to V4 not V5!

the time when V5 becomes stable i'll make a patch for it!

 

Thanks a lot Aymen. Appreciate that. I think this was very important. 

Maybe some people try to build bots with v5 and the plugin. And are wondering about strange errors. So that's the reason why. 

 

Stay with v4. It's still much better in a lot of scenarios.

Dan

[Aymen 385]

Hi guys,

 

There seems to be demand for a working DeathByCaptcha source code (the one in the bonus package is not a full solution).

 

DeathByCaptcha doesn't automatically return the answer to the CAPTCHA. As explained in the API, you need to "poll" for the uploaded CAPTCHA status manually (it's not as simple as with Decaptcher for example), so you need to make sure you give enough time for them to actually solve your CAPTCHA before the final GET request. You can make a loop and check every 5 seconds (wouldn't use a much lower value, because you could get banned for too many checks).

 

I quickly updated the "DeathByCaptcha API.ubot" script to make it work (source code attached).

Example image to test it with: http://www.google.co...jquiAo_H3ywspkU

 

I added a 10+5 second wait before the first check, because in my experience it takes at least 15 seconds for DeathByCaptcha to give an answer. After that the script checks for an answer every 5 seconds, up to a maximum of 20 tries (I added this just so there is a timeout).

 

I also added a "Check Balance" button for you guys.

 

Hope this helps,

Marton

 

p.s.: Aymen feel free to include this in your bonus package if you like it

 

Thanks Marton , if you can open a new thread with this source i'm sure a lot of people will see it and will appreciate it !

Thank you for the work

 

Regards,

[Marton 48]

Thanks Marton , if you can open a new thread with this source i'm sure a lot of people will see it and will appreciate it !

Thank you for the work

 

Regards,

 

Great thanks, I'll do that right now!

[Cyrax 2]

can someone please advise me what basic concepts i need to know before trying to use this plugin, as i am beginner in ubot and don't know why i need to make use of http post. and where http post and get fit in with ubot.

[Bot-Factory 602]

can someone please advise me what basic concepts i need to know before trying to use this plugin, as i am beginner in ubot and don't know why i need to make use of http post. and where http post and get fit in with ubot.

Ubot itself is using the built in browser. So you can navigate to sites and scrape / modify stuff on that page.

The http plugin allows you to create http requests. A Http request is the stuff your browser will send to a webserver when you do something on a website (not for everything).

So it's the action behind some of the things you do within a browser. 

 

When you click a button your browser will generate a http request (depends on the button) and send it to a webserver. If the button only executes local stuff (javascript), then there will be no http request.

 

Advantages:

Much faster than working with the browser

 

Disadvantages:

More complex. 

You need to understand http request.

You have to know how to sniff and "deconstruct" http request with tools like fiddler

You can't do everything with it (javascript limitations)

 

One is not better than the other. It depends on what you want to do. 

 

My recommendation:

Start with browser first. And then look at performance optimizations with http plugin later.

 

Dan

[Marton 48]

Disadvantages:

You can't do everything with it (javascript limitations)

Hi,

 

Do you have some examples for stuff that is not possible with HTTP Requests? Maybe not with Aymen's plugin, but in general as far as I know we are supposed to be able to do everything with them if used correctly. Even AJAX stuff.

 

Any insight on this?

 

Thanks,

Marton

[Bot-Factory 602]

Hi,

 

Do you have some examples for stuff that is not possible with HTTP Requests? Maybe not with Aymen's plugin, but in general as far as I know we are supposed to be able to do everything with them if used correctly. Even AJAX stuff.

 

Any insight on this?

 

Thanks,

Marton

Posted an example in another thread:

http://www.ubotstudio.com/forum/index.php?/topic/16427-http-plugin-value-hidden-behind-javascript/

 

Not possible = I haven't found a way to do it :-)

 

But if something is hidden within javascript, you need a way to execute that script to get the data.

 

Dan

[Marton 48]

Thanks Marton , if you can open a new thread with this source i'm sure a lot of people will see it and will appreciate it !

Thank you for the work

 

Regards,

 

Actually Aymen, could you spare a moment to look at the code I posted (it's really short, just a few commands)? I just found something, I think this could be interesting. Try it in this order:

 

1. Run "Check Balance" (works as expected, I get: "Status OK! Balance: 2.83$")

2. Run the script (works as expected, I get the answer to the CAPTCHA)

3. Run "Check Balance" again (takes a long time, I get blank page in browser, then I get: "Status OK! Balance: $")

 

According to my script, "Status OK!" means that it was actually able to get the balance, but then for some reason it doesn't display the right value (not a regex error though).

 

Could this be an anomaly with the plugin?

 

I actually had the same thing happen in one of my complex scripts, but I thought it was something in my code. But everything seems fine here. I even tried adding "http containers" to multiple places, doesn't seem to solve the problem. Anyone know what might be causing this?