Jump to content



Photo

[Solved] Threading question

threading multi threading

  • Please log in to reply
6 replies to this topic

#1 jomark3

jomark3

    Advanced Member

  • Fellow UBotter
  • PipPipPip
  • 183 posts
  • Locationnj
  • OS:Windows Vista
  • Total Memory:3Gb
  • Framework:v4.0
  • License:Developer Edition

Posted 25 January 2013 - 06:06 AM

I am building a bot that navigates to 3 different sites, logs in (with 3 different logins & passwords), scrapes data, and posts the information. The bot works well, but I would like to speed things up by threading and doing all 3 processes at the same time.

 

I am using defines for each site as follows:

 

Define site 1

     navigate to site

     enter login data

     scrape page

     save scraped data to list

Define site 2

     navigate to site

     enter login data

     scrape page

     save scraped data to list

Define site 3

     navigate to site

     enter login data

     scrape page

     save scraped data to list

 

For each define, I tried:

 

Define site 1

Thread

     in new browser

          navigate to site

          enter login data

          scrape page

          save scraped data to list

 

And although this technique performs each define in a new browser, the process takes place 1 define at a time and is not any faster. How do I use threading to perform these 3 defines simultaneously?

 


Signature-Image.jpg


#2 LazyBotter

LazyBotter

    Advanced Member

  • Fellow UBotter
  • PipPipPip
  • 744 posts
  • LocationManchester
  • OS:Windows 8
  • Total Memory:More Than 9Gb
  • Framework:v3.5 & v4.0
  • License:Developer Edition

Posted 25 January 2013 - 07:10 AM

Hi there are may different ways to do multithreading, this is a simple way to code it, I did this quick but is probably what you need.

 

Start()
define Start {
    thread {
        in new browser {
            Define One()
        }
    }
    thread {
        in new browser {
            Define Two()
        }
    }
    thread {
        in new browser {
            Define Three()
        }
    }
}
define Define One {
    navigate("http://google.com", "Wait")
    wait(5)
}
define Define Two {
    navigate("http://google.com", "Wait")
    wait(5)
}
define Define Three {
    navigate("http://google.com", "Wait")
    wait(5)
}

Screenshot-2018-08-08-11.18.50-e15337237


#3 HelloInsomnia

HelloInsomnia

    Advanced Member

  • Moderators
  • 2977 posts
  • OS:Windows 10
  • Total Memory:More Than 9Gb
  • Framework:v4.5+, unsure
  • License:Developer Edition

Posted 25 January 2013 - 07:11 AM

Yes that should be correct. Here is a threading example that works maybe it will help you:

 

startnow()
define startnow {
    google()
    yahoo()
}
define google {
    thread {
        in new browser {
            navigate("http://google.com", "Wait")
            wait(2)
            type text(<name="q">, "modern marvels", "Standard")
            wait for browser event("Everything Loaded", "")
            wait(10)
        }
    }
}
define yahoo {
    thread {
        in new browser {
            navigate("http://www.yahoo.com/", "Wait")
            wait(2)
            type text(<class="input-query input-long med-large  ">, "top gear", "Standard")
            wait(.5)
            click(<id="search-submit">, "Left Click", "No")
            wait for browser event("Everything Loaded", "")
            wait(10)
        }
    }
}


#4 jomark3

jomark3

    Advanced Member

  • Fellow UBotter
  • PipPipPip
  • 183 posts
  • Locationnj
  • OS:Windows Vista
  • Total Memory:3Gb
  • Framework:v4.0
  • License:Developer Edition

Posted 25 January 2013 - 07:23 AM

Thanks guys, worked like a charm!!!


Signature-Image.jpg


#5 bestmacros

bestmacros

    Advanced Member

  • Fellow UBotter
  • PipPipPip
  • 396 posts
  • LocationIsrael
  • OS:Windows 10
  • Total Memory:More Than 9Gb
  • Framework:v4.5+, unsure
  • License:Developer Edition

Posted 28 January 2013 - 02:19 AM

it is good post and I've encountered same problem with thread command, in my opinion current implementation of thread command is only useful for ddos some site or things like this, because if I want to perform some task in let's say 20 threads I would need to create 20 define's for each tread - it is insane!

And what if I want 100 threads :o ?  


bestmacros.jpg


#6 VaultBoss

VaultBoss

    I have no fear, for fear is the little death that kills me

  • Fellow UBotter
  • PipPipPip
  • 891 posts
  • OS:Windows 8
  • Total Memory:4Gb
  • Framework:v3.5 & v4.0
  • License:Developer Edition

Posted 28 January 2013 - 09:00 AM

it is good post and I've encountered same problem with thread command, in my opinion current implementation of thread command is only useful for ddos some site or things like this, because if I want to perform some task in let's say 20 threads I would need to create 20 define's for each tread - it is insane!

And what if I want 100 threads :o ?  

 

I'm not really sure that is the issue that you have, but if you're stating that you need to re-DEFINE your custom commands/functions for each thread, then I'm afraid YOU are doing something wrong there. 
(no offense meant, just trying to help you)

 

My guess is you are using GLOBAL variables in your code, all over, including the DEFINEs .. which, if it were true, might indeed mix up values and render you bots useless, in my opinion.

 

On the other hand, if your variables are set as LOCAL in each DEFINE, you can repeatedly call the same function/command in each thread and it should keep the values confined to that thread, thus, w/o mixing them with other threads running concurrently, there is no room left for errors anymore
(at least from THIS issue << which I observed to be a common mistake in people's coding)

 

Hope this helps...


If you liked my post/contribution, hit the LIKE THIS button ↑↑↑ above...
GET CaptchaSniper with a 20% off discount below:
8903615856_1a01c0e40c_z.jpg

Also GET my "Sexy eMail Marketing" Book ($27 real value) as a free bonus when you buy from my link ↑↑↑ above...


#7 bestmacros

bestmacros

    Advanced Member

  • Fellow UBotter
  • PipPipPip
  • 396 posts
  • LocationIsrael
  • OS:Windows 10
  • Total Memory:More Than 9Gb
  • Framework:v4.5+, unsure
  • License:Developer Edition

Posted 28 January 2013 - 01:27 PM

You right ! I'm using Global all other the place, will try to switch to local for each define. Thanks for the help!

I'm not really sure that is the issue that you have, but if you're stating that you need to re-DEFINE your custom commands/functions for each thread, then I'm afraid YOU are doing something wrong there. 
(no offense meant, just trying to help you)

 

My guess is you are using GLOBAL variables in your code, all over, including the DEFINEs .. which, if it were true, might indeed mix up values and render you bots useless, in my opinion.

 

On the other hand, if your variables are set as LOCAL in each DEFINE, you can repeatedly call the same function/command in each thread and it should keep the values confined to that thread, thus, w/o mixing them with other threads running concurrently, there is no room left for errors anymore
(at least from THIS issue << which I observed to be a common mistake in people's coding)

 

Hope this helps...


bestmacros.jpg






Also tagged with one or more of these keywords: threading, multi threading

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users