Who’s Building The Future of UBot? Pt 1

You might have noticed a new release of UBot Studio today – 5.5.13, which introduces some cool new concepts that make testing UBot Studio code a snap (more on those in a blog post later this week). I thought this would be a great time to thank the newest member of the UBot Studio team for working on this, and to introduce him to the UBot community: Say hello to Adam, our new lead developer.


Adam has been programming since he stopped wearing diapers. Forced to type the code of the games that he wanted to play into his Apple2E from magazines because hard drives weren’t mainstream, he naturally started fiddling to make the games do other things. After choosing to go to computer camp in 3rd grade instead of doing team sports, Adam’s nerd cred was thoroughly established.


Before joining the development team taking over development at UBot Studio, Adam was a Senior Engineer at a game development company that built some of the most popular games on the web. You know that facebook game that you get all those super annoying invites from? Adam worked on that. After a brief stint at an online education startup, he decided to go freelance and travel the world a bit. After completing several freelance projects with the UBot Studio team, he is now working closely with me to lead development of future updates to UBot Studio and our future projects. Adam’s enthusiasm, entrepreneurial spirit, and strong insight into programming theories make him a perfect fit for the team.


Why Your Bot Is Showing Up As A Virus (And What To Do About It)

Over the years, we’ve changed our mind about antivirus software.

Everyone used to need it. Back when the internet was filled with trojan horses, logic bombs, viruses, and malware, virus scanners used to be necessary. Back when you downloaded software willy-nilly and Google didn’t pre-label sites as potentially dangerous, it seemed like within no time at all you could pick up a virus, just by browsing the web.

But since then the need for virus protection has changed. In the Wall Street Journal, Symantec’s own Senior VP for Information Security has said anti-virus software “is dead.” Virus scanning software has become nearly obsolete in the business world, and companies like Symantec, who invented commercial antivirus with the ubiquitous Norton antivirus protection suite, now defend against cyber threats via automated threat forensics (checking network traffic to detect threats early on, for example). Journalists are writing headlines like “Antivirus Is Ailing” and “The Antivirus Era Is Over.” In the last decade, security experts have switched their focus from detecting software on your PC via antivirus products to stopping viruses before they ever get to your computer, and minimizing lost data once the viruses do get there. (In fact, Symantec only has about a 70% success detection rate – so it’s no surprise these companies are moving away from endpoint protection.)

Displeased Face Male

When your AV software tries to remove Skype because it’s “dangerous”.

Many don’t realize how anti-virus scanning works. For example, are more scanners better than one? Virus Total, a virus scanning aggregator, now lists over 50 virus scanners, and many often use this product to test files across many different scanners, but Virus Total specifically recommends AGAINST this: “In VirusTotal desktop-oriented solutions coexist with perimeter-oriented solutions; heuristics in this latter group may be more aggressive and paranoid, since the impact of false positives is less visible in the perimeter. It is simply not fair to compare both groups….Very often antivirus solutions and URL scanners will produce false positives, i.e. detect as malicious innocuous files and URLs. These erroneous detections may severely hinder the business activity/popularity of third party products (e.g. refrain access to a given site, dissuade users from downloading and installing a given application, etc.).”

Virus scanners, even according to the makers of them, often find out about actual problems too late, and often what they detect is a false positive. Some even detect chat clients because they’re productivity killers, and torrent clients because they might be used to download viruses accidentally.

Virus scanners, frankly, are often crap.

So if you ever notice that something you’ve compiled in UBot Studio is being detected as malware or as containing a virus, first, relax, and remember that this is almost certainly a false positive report, and it’s being sent from a dinosaur in a dying industry. Then, follow these steps to fix the problem.


Mostly Harmless

Click the “Angel” icon to vote for a file as harmless on Virus Total

      1. If the scanner is on your computer, be sure to add the file to the virus scanners exceptions list. This will let you continue using the file.
      2. Don’t upload your file to VirusTotal. Search for it there using its md5 hash to determine if it’s been uploaded already. This is quicker, and industry experts have told us that uploading your file is one reason that some products are falsely detected. (According to Virus Total, they help “antivirus labs by forwarding them the malware they fail to detect.”) The md5 hash is just a string that helps verify a file quickly. To check the hash, use the command line in Windows or OSX.) If your file is on Virus Total, make sure to vote that it is “Harmless” in the top right.
      3. Read your virus scanner results carefully. Make sure to check any “Additional Information” sections. Take note of any specifically important and ambiguous wording, such as: “While not necessarily malicious, the scanned file presents certain characteristics which depending on the user policies and environment may or may not represent a threat.” (Actual language from Virus Total.)
      4. Let the company or companies reporting it as malicious know that it’s a false positive. There are several ways to do this, but the best is to submit a false positive report. This site has a list of where to submit false positives for each company. A short message is all you need to give them in most false positive reports, along the lines of “Dear company, the executable I’ve attached was compiled using UBot Studio. Your virus scanner reported it as containing {virus x}. It does not contain this and this was a false positive. Please refrain from reporting this UBot Studio compiled executable as containing x.”
      5. No, really, tell the virus scanning company that it’s a false positive. Tweet to them. Email them. Post on their forum. Get your friends together and anyone else having this problem, and explain that you compiled the software yourself in UBot Studio, and it is definitely a false positive. If you do this as soon as you notice the issue, you are more likely to get a speedier resolution. Make no mistake – these AntiVirus companies are wrongly accusing you of a crime, and in doing so they may be harming your business unjustly. It is crucial that you do whatever you can to make these companies aware that you won’t be bullied.
      6. Let us know at http://tracker.ubotstudio.com. Be sure to link us to the Virus Total report and let us know the following:
        • Were you including the installer in your compilation?
        • What plugins were included?
        • What virus scanners are detecting your product as a false positive?
        • Include the script, if possible.
      7. Send anyone who might be using your compiled executable an email letting them know what’s going on. Here’s a quick message you can use or modify:

I just got word that the software I made is being detected as a virus by X antivirus program. First of all, let me assure you that it does not contain any malicious script and is entirely safe to run.

Second, I’ve contacted X, the company listing it falsely, and you can follow the progress of this false positive submission here: [Link to your sales page or blog or forum post explaining where the ticket or email is and how they can get more info.]

As you know, virus scanners work by analyzing software for specific actions and potential threats. My software checks with a server when it’s loaded to see if it has to download any additional support files, and this is probably why it’s being detected as malicious. This is known as a “generic downloader” signature and is a common problem for producing false positives. [Here include any ambiguous wording that could help explain that the virus scanner isn’t sure that it’s actually a virus, and maybe even send this link to show that even Info Security professionals have this same problem.]

Sometimes virus scanners help, but often times, they tell you there’s something dangerous on your computer when there really isn’t. In this case, you are totally safe to run the file and I’m working on getting this issue corrected. If you have any questions please let me know. To run the file, simply add it to the ‘exceptions list’ in your AV scanner. Contact me if you need help doing this.


Concerned Software Creator


Over the years, we’ve worked to make UBot Studio friendlier to AV software, by changing how compiling works and how the calls-to-home work. We’ve done everything we can to keep AV companies out of our business, and out of your business. As a software creator, it is now up to you to take that flag, carry it with pride, and, if necessary, beat a few AV company executives over the head with it.






Thank You For Another Successful Year

Here at UBot HQ, we work around the clock to make UBot Studio the most powerful, time-saving software in your marketing toolbox.  Our top-notch community exemplifies the ingenuity that makes the internet business industry the indispensable force that it is today. Over the last three years we have been compiling information from a survey customers like  you have completed. The feedback you provide  is crucial to our continuing improvement! As a way to say thank you, we want to share the results with you. The_UBot_Survey (1) The_UBot_Survey2 The_UBot_Survey3 The_UBot_Survey4   We are proud to be able to serve you and to be surrounded by such a talented community. Here’s to a prosperous, autonomous 2015! Seth Turin P.S. – We have a new survey that we would love to have you complete. It will only take you a few moments to fill out this quick 10 question survey, and it would mean a lot to us. Take the new survey!

Python and UBot: Give Your Bots a Voice… and Much More




Ask a programmer how a beginner should automate web tasks and chances are good they’ll say Python.

Python is a programming language that can help you automate everything from email sends to web scraping to robotics, and while it’s significantly more complicated than starting off with UBot Studio, it’s a very robust language and fairly easy to understand for beginners. (For more info on getting started with Python, click here to read about it and here to get it set up on your computer.)

Because at times using Python will make your bots faster, smarter, and more awesome, we included Python integrations inside of UBot Studio 5.5. As a Standard or Professional user, you will be able to run IronPython scripts. As a Developer user, you’ll be able to write and edit Python scripts directly inside UBot.

Seth and the Team had a great conversation with the UBot Studio community about how you can get set up using Python, what you might want to do with it to start, and the way that Python integration also allows you to add .dll files to your bots. There will be more Python conversations in the future and we’ll be sure to tie all these chats together on the blog!

For more info, check out the Run_Python command on our wiki.


Community Chat: Getting Started with Python


Seth Turin:

Python is a great language for beginning programmers and expert programmers alike. I chose Python because it’s widely regarded as the easiest language to learn, while still being a fully expressive, object oriented language.


JasonIconJason Kelley:

Yeah – I’m barely a programmer and even I’ve used it a little to do some really cool stuff, after getting some scripts from some other programmers and editing them.



To get started into Python with UBot, you’ll need to have IronPython installed on your system. We’re working on removing this as a dependency, but for now, you can download and install IronPython at http://IronPython.codeplex.com/releases/view/90087



And IronPython is a type of Python? In the way that Mysql is a type of SQL?


Dave: UBotIcon

IronPython works with .net, I think.



Yes, IronPython is an implementation of the Python spec that is built on the .net framework. It’s a nice implementation, too, because you not only get all of Python’s yumminess, you also get the entire .net framework! Once you have IronPython installed, you should be able to use Python, its core modules, and the Python Standard Library.


Let’s do a few examples to demonstrate how to have fun with IronPython in UBot.

alert($run python with result("import datetime
mydate = datetime.date(1943,3, 13) #year, month, day

What I’m doing with this bit of code is making a Python function return a result back, that I can show in an alert, or of course, do anything I want with. UBot’s date functions are somewhat limited, so just having a few lines of Python handy can give you a lot of power. The %A in the last line indicates day of the week. If you want to see more tricks with the datetime module, goes here http://www.Pythonforbeginners.com/basics/Python-datetime-time-examples

The Python standard library has a lot of stuff! This opens up UBot’s capabilities in a huge way. A full reference on the standard library is here https://docs.Python.org/2/library/

So what if we wanted to take our date example earlier and make it a custom function?



alert($get that date())
define $get that date {
return($run python with result("import datetime
mydate = datetime.date(1943,3, 13) #year, month, day

If you consider the possibilities here, you could easily make an entire bot of just defines, like a library, and include it into all your other bots. This effectively gives you the same power as creating plugins, though plugins have a few differences.



Does using Python change the size of your bots etc. now that you have to use the IronPython framework?



Yes, but not significantly. About as much as a plugin would and maybe less.



So, there’s more that you’ll want to know about how IronPython works with .net because it gives you a lot of great new options. IronPython comes equipped with a few special commands for importing .net framework libraries. Let me show you an example:


import clr
 from System.Speech.Synthesis import SpeechSynthesizer
 speechSynthesizer = SpeechSynthesizer()
 speechSynthesizer.SpeakAsync("hello I am bot")

A few things happen here:
1) We import clr. That’s basically where the core .net stuff lives.
2) We add a reference to system.speech, which is part of the .net framework.
3) Using some Python syntax, we get our speech synthesizer fired up
4) We tell it to talk in awkward English.



Whoa, does this make my bot talk to me?



Yes. Your bot now has a voice and it just wants to be heard.

Another implication of the .net framework is that you can load .net dll files. So any .net open source project that you want to utilize can now be a part of your bot. Here’s an example:

import clr
 import sys
 import NAudio
 from System.IO import File
 waveOutDevice = NAudio.Wave.WaveOut()
 audioFileReader = NAudio.Wave.AudioFileReader("c:\sounds\sound.mp3")

I start by importing clr again, but then I use AddReferenceToFileAndPath to load the dll. In this script, I use a special function that UBot has loaded to help manage libraries for use in the bot bank. To load your own dll, it might look more like:

clr.AddReferenceToFileAndPath(“NAudio.dll”) or clr.AddReferenceToFileAndPath(“c:\lib\NAudio.dll”)
Note that path names have double backslashes. Once I load up the dll, I import the stuff I want from it, along with some core modules for loading files, and I have my wave player play a sound file. For clarity’s sake, I’ve just been pasting the Python code that you would paste into a run Python command. The full uscript for this example looks like this:

run python("import clr
import sys
import NAudio
from System.IO import File
waveOutDevice = NAudio.Wave.WaveOut()
audioFileReader = NAudio.Wave.AudioFileReader("c:\\sounds\\sound.mp3")



So I’m using Python to include .net stuff and now my bot is essentially capable of the same things that a .net script is (or at least a lot of those things)?



IronPython is .net, so everything, yes



So IronPython is like gateway for Python to run on .net?



That’s one way to look at it Jay.



Ok, so I think the only thing left that’s confusing to me is how my Python script is interacting with .net dll’s – because aren’t they written in .net. Like I’m using Python commands to reference .net commands which is a different language?



Good question, the important thing there is that .net itself isn’t a language. .Net is a framework, written probably in C or C++, that a few languages are built on, such as VB.Net, C#, and IronPython. .Net uses a construct called the clr which stands for common language runtime – you saw references in the scripts earlier. C#, VB.Net, and IronPython all compile down to the clr. Make sense? All this is to say, IronPython can read dlls compiled in any of the previously mentioned clr languages.



Whoa, does Python completely open UBot up to being able to use all those languages, in a manner of speaking? In the sense that you can write modules in C# or VB, and then import them using 1 or 2 lines of Python?



I hadn’t thought of that, but yes.

Some of you may want to know if  IronPython or C# is faster. That question did come up while we were working on this, because we’re always wondering how we can make UBot faster and more agile while remaining easy to use. There’s a great discussion of this on stackoverflow:


In programming, there’s always a tradeoff. Assembly is the fastest language, by far, but it takes forever to write, and is awful. C++ is still fast, but slightly easier to write, C# is less fast and very easy to write, IronPython is less fast and extremely easy to write. Uscript is easier and slower than all of the previously mentioned languages. The tradeoff is speed of processing vs speed of implementation. The reason that UBot is so valuable is that you can write a UBot bot in 2 hours that does the same thing as a C# bot that took 2 weeks to write, because of all the built-in features of UBot. The C# bot is faster, but if you’re two weeks ahead of your competition, you clearly have the edge. Now I’m getting all theoretical, but the point is we now have all these options.



Like any serious programmer should really now be able to build incredible bots for UBot far outside of the uscript framework? And any non-serious programmer can move in that direction if they want?




If you want to dive deeper into the wonderful world of Python, here are some great learning resources.

Have fun with it! I think I speak for the whole team when I say that I’m eager to see what you guys come up with now with the limitless possibilities that Python provides.


Clean the Clutter with Include

Speed of implementation is everything in this business. UBot Studio simplifies the process of web automation, reducing your build time from 2 weeks to 2 hours. But what happens when your bots start to get larger and larger? If you’re not careful, you’ll start getting consumed in your bot’s complexity, as each little task taking longer and longer to complete.

That’s why we’ve introduced the “include” function into UBot Studio 5.5.

Include lets your current script reference previously written scripts without rewriting that code again, which means you can create whole scripts that do a lot in just a few lines, keeping your bots small and complexity under control.

The UBot team recently explained the Include feature in a popular chat room to discuss this feature with the community. This Q&A session has been edited and reproduced below along with a demo video and a script. For more information on using Include, make sure to check out the wiki and the forum.


Community Chat: How to use the include command


Seth Turin:

I thought I’d take a minute to explain how the include command works, both to my team and to the community


JasonIconJason Kelley:

When did this feature come out?



5.5.6 – It was actually in UBot 3.5, but got cut from UBot 4. It’s now back and ready for action.



Awesome. My impression from previous conversations about include is that it’s a way to organize code and make it more reusable, which helps because people have some really long code in some of their bots. Am I close?



Yes, you are. I’m going to explain it through the art of extended analogy. Imagine that you’re the CEO of a new car manufacturing startup. We’ll call it… mistubotsi. At first, the entire company is run by you and your trusty robot friend, Hal. You have to teach Hal how to do everything in the company, from manufacturing the cars, to sales and marketing, to customer service, etc. At first, this is fine, because there are only a few orders at a time, and Hal’s job is pretty simple, but soon, business picks up, and Hal finds himself doing more and more different kinds of jobs. Soon, Hal’s bot brain is overloaded and frazzled.

Hal gets angry. One day he locks you in a crash test car. You’re all like “Open the car door Hal!” and he’s like “I’m afraid I can’t do that, Dave”. The moral of the story is, you want to keep your bots happy by keeping them small, focused, and organized. Include is a means of doing this.


So I have a bunch of bots that use a kind of complex series of change proxy commands. In that instance I’d build that as a separate bot basically and use the include?



You’re pressing on another benefit of include, which I’ll return to a little later. But the short answer is yes.

So to keep your bots happy, keep them small and focus on a single job (or a single website). Happy bots have the benefit of being easy to maintain, less error prone, more stable, easy to change, and easy to understand.



Unhappy bots are bots that are big and bloated, unorganized, and try to do too many things… they suffer from breaking often, being hard to debug, being hard to maintain, and possibly… becoming homicidal. When a bot starts to become too large, break it into separate, focused bots, and include those into one central, manager bot. By doing this, you greatly increase your capacity to make large, complex bot systems, that will be far more productive than one bloated unhappy bot.


Manny  Hernandez:

It is simple. Build your bot in modules. Like if it has account creation and posting. One bot for account creation, another for posting… Managed both in a central bot for the Ul.



Exactly! Granted, if all those are small enough, you can just separate them into scripts. If account creation became a big operation, it makes sense to put it in a separate bot.


JasonIconJason Kelley:

So the idea is like, if you want to build a big bot, you can do that – just use include to break it into a bunch of separate pieces that you know all work. Then if you see a bug you know where it is and you can fix it (like normal programmers do i’d imagine).


Dave: UBotIcon

But you are only including the defines from the bot right not the whole bot???



Yes. Dave, the bot is technically minimally loaded in full, with all its plugins and its own included bots, but you can only access it through defined commands and functions.



Is there a limit to how many you can include??? In the real world, I mean.



It’s only limited by your machine’s hardware. In computer science, this idea is called “encapsulation”. It keeps the different sections of a bot clean and clutter free.



How is it used? You call the include and automatically the defines are available?


Screen Shot 2014-12-08 at 1.10.28 PMMacster

Then include does pretty much the same as what the local bot bank does … but it actually loads the defines when requested from the UBot file!?… Isn’t the whole process the same in both features?



Macster, to answer your question, the private bot bank is a little more limited than included bots.

In order of customizability, it goes 1) plugins, 2) included bots, 3) private bot bank, 4) custom commands

Jason, I want to address your point earlier… you said you had a complex proxy switch process that you use in a bunch of bots. You probably copy pasta from one into the next. The problem with copy pasta is that if, for instance, the server where you get your proxies from changes, you now have to go update every single bot separately



Yes, or if I decide to change how it works in one bot and not in a bunch of others



Instead, you could make a separate bot that just focuses on proxy switching, and include it into all the bots like you mentioned. If the server changes, you just change the one bot and the change is applied to all the bots that include it



In a compiled bot that uses the include, do we need to have the .ubot in the file folder for commercial bots? Or it works like the plugins do?



At the moment, you need to have it in the same folder, or specify that path to it


How do we pass inputs (variables, parameters) to the included bots? How do we pass user input data to the included bot from the main UI?



As a parameter, so the define in the included bot will take a parameter. Then in the master bot you just send in the variable as the parameter.



How is it used? You call the include and automatically the defines are available?


Once in the toolbox, you can use defined commands like any other command. Included bots can use plugins, and they can include their own bots, so you can potentially create highly expressive, large scale bots, while keeping everything small and manageable!



Looks good, does include reduce ram memory consumption?



Jay, it will considerably reduce resources at dev time. UBot Studio does a lot of stuff to make coding visual and simple, and that also means that huge scripts will get laggy. By using include, you reduce the amount of code UBot Studio is processing at any given time

It does, however, slightly raise the resource usage in the compiled bots. This increase is not significant, and the tradeoff is well worth it.



The include command will definitely be worth its weight in gold, if I understand you correctly. For instance I use a very special bot recipe for all my bots, and it works on almost any site I make bots for. At current, I’m using the same recipe for all my bots functions, and just making new tabs for the sets of define commands for each of those functions. So with the include command I will be able to take the code out of those tabs and make them into their own .ubot file and call them from one central bot instead of having a bot with 20 tabs, I can have one bot with one tab for the UI and 20 .ubot files being called when the user select to run a certain function of the bot. Correct?



Yes, you are exactly correct



That is going to make Ubot very very powerful. With limitless possibilities now.


A quick example of how to use include [code attached].



So what we’re doing is making a bot that will search the big 3 search engines. This particular bot is small enough that it could all be done with one bot, but in this case we’re going use it to demonstrate how include works. So we’re going to separate the search functions into separate bots, and then consume all 3 bots in our manager bot.

Making one of those bots looks something like this:

Unable to display content. Adobe Flash is required.

 Once we have all three, we put them together like this:

Unable to display content. Adobe Flash is required.

Includes should always be the first thing in a script. You can see that by putting include into your script, it creates a new category under commands or parameters (in this case parameters) with all the defines from the other bot.

Bot 1: Google Scraper

define $search google(#search term) {
    type text(<name="q">,#search term,"Standard")
    click(<name="btnK">,"Left Click","No")
    add list to list(%results,$scrape attribute(<tagname="a">,"href"),"Delete","Global")


Bot 2: Bing Scraper:

define $bingbot(#bingbot) {
    type text(<name="q">,#bingbot,"Standard")
    click(<name="go">,"Left Click","No")
    add list to list(%bingresults,$scrape attribute(<tagname="a">,"fullhref"),"Delete","G    lobal")


Bot 3: Yahoo Scraper:

define $search yahoo(#search term) {
    type text(<id="p_13838465-p">,#search term,"Standard")
    click(<id="search-submit">,"Left Click","No")
    add list to list(%results,$scrape attribute(<id=w"link-*">,"href"),"Delete","Global")


Bot 4: Include in Action – substitute your directory for  [LocationOfScripts] 

set(#search term,"hal the happy bot","Global")
include("C:\Users\[LocationOfScripts]\google scraper.ubot")
include("C:\Users\[LocationOfScripts]\search yahoo.ubot")
load html("<pre>{$run function("google scraper","$search google")}
{$run function("bingbot","$bingbot")}
{$run function("search yahoo","$search yahoo")}</pre>")


For more basic details of how to use include, see the wiki, and thanks to all who attended our chat!