Overwhelmed - Help, Please...

[jeepinjeff 4]

I've been using Ubot for a while now and have come to see that there are a lot of, what appear to be, very useful plugins available. This brings about a number of questions that I'm hoping you guys can help me out with.

 

1) What is the advantage of using the ExBrowser VS the standard one? Is it faster?  More efficient? Something else?

 

2) What's the benefit of using the HTTP plugin? Again - faster, more efficient, better in some other way?

 

3) And X-Paths - what's the deal with those?

 

As you can obliviously tell, I have 3 plugins in mind that seem to be fairly popular but I just don't fully understand the benefits of them, so your help would be greatly appreciated.

 

Thanks!

Jeff

[HelloInsomnia 1103]

1. ExBrowser uses your Chrome or Firefox browsers as a remote control (or PhantomJS which comes with it). This is advantageous as it is not using the Ubot Browser which could be detectable. It is used in cases where you want to look the most human like. Let's say you made a Facebook related bot, or some other site that is going to be actively trying to block bots then ExBrowser is going to be the best way to go.

 

2. HTTP Post allows you to get and post to web pages using sockets (no browser) making it much faster. You can run more threads which are running at a faster pace. This is going to be the thing to use when working with API's or scraping (basic websites, without login). Nothing is better for those two things then this. It can be used for other things as well such as logging into an account to post something - depending on the site but then you are getting back into Ubot or ExBrowser territory.

 

3. Xpath is like the search engine of the building blocks of a website. Pretend each kind of block has a different color and the same colored blocks have different numbers. Each block may have an attribute (or more) as well sort of like tags. You can use xpath to pull out the exact block you want and interact with it. 

 

You will need to use Xpath with both the HTTP Post and ExBrowser Plugins.

 

By the way those are two important and "must have" plugins I would highly recommend anyone get.

[jeepinjeff 4]

Thanks HelloInsomnia!  Now, where do proxies fit into all of this? Can you use proxies (or do you need to) when using sockets? And with regards to the ExBrowser, what's the difference between THAT and changing the User Agent?

 

Thanks again!

[HelloInsomnia 1103]

Thanks HelloInsomnia!  Now, where do proxies fit into all of this? Can you use proxies (or do you need to) when using sockets? And with regards to the ExBrowser, what's the difference between THAT and changing the User Agent?

 

Thanks again!

 

Proxies can be used in either plugin. And should be yes you will still need them in most cases (unless you're scraping). Changing the user agent just changes that one parameter. Using ExBrowser you can get around some of the issues that do not work in Ubot browser (forgot to mention that part) because you are using a normal browser. Also, you can load plugins, change window size do things like that which make you blend in even more.

[Code Docta (Nick C.) 638]

Would like to add that changing the "user agent" also reduces your footprint.

Same with window size, proxies, time between entering data into fields and what plugins are installed etc.

 

 Proxies protect your IP from being banned too.

 

You can use socks4/5 proxies with ExBrowser.

 

Xpath is faster at parsing HTML or XML then

Regex and it is easier to remember and more precise.

 

"Faster" is debatable, however in Ubot it is not noticeable.

You can use the respective browser's inspector to get the xpath or css selector.

Just right click on the element and copy xpath, copy css path or even the short version(if available) of xpath which is the best.

 

I use "firebug" add on for FF and is available for Chrome I believe.

 

CD