Jump to content
UBot Underground

wtf

Fellow UBotter
  • Content Count

    25
  • Joined

  • Last visited

  • Days Won

    1

Posts posted by wtf

  1. Whenever you read or write you to or from a global variable,list or table you have to contain it inside a lock container to keep the data integrity in tact.

     

    Yeah understand that about global but in this instance I am asking about local. The only global entity would be the list to write the value of the local var In the code.

     

    #counter would be local so is it local to the browser instance? i.e. can #counter have its own unique values within the browser its running in?

     

     

  2. 2. Exactly, "lock" the code that writes or read to and from the global list by placing it inside a lock command. The time delay be minimalistic at what you lose from locking a protected piece of code. Let's say adding to a list takes 2-3ms then all threads that are waiting have to wait 2-3ms but it assures data integrity. But most of the time they don't even have to wait it's just for those cases where it hit's the code simultaneously.

    Thats what I hoped, cool, thanks for that.

     

    1. The data integrity will not be protected by merely using the CBrowser commands. However using the lock commands that are provided with it will preserve the integrity of the data.

    So any #var assigments would also have to be locked for the entire required duration of the existance of that var or it will be overwritten by a different browser instance assigning a different value to that same named #var?

     

    Ok just to 100% sure.....

     

    What would happen if the value of 1 was assigned to #counter in browser 1, then 5 seconds later the value of 2 was assigned to #counter in browser 2 (assuming the assign code wasnt in a lock) then 3 seconds after that, the code in browser 1 added the value of #counter to a global list? (add to list code would be locked of course)

     

    What would the value written by browser 1 be? 1 or 2?

  3. Hey guys, looks like this PI could solve some issues Im having but would just like to ask a couple of things to see if I could adapt a current project to make use of this. Excuse the drawn out explanation but I want to be sure this will work before buying and starting to rewrite what Ive done already........

    I have a multithreaded bot using thread > in new browser to spawn say a max of 10 instances of browser which all run the same code running through a series of commands. Some instances will finish quickly 1-3 secs as the initial check will show theres nothing to be done while some will take up to a few mins as each code sequence runs stuff, loops waiting for the scraped html then finishes or continues on depending on the results scraped and so on.

    As the code runs it it updates a global list outside the thread with certain key info on whats happend and the final result before the thread closes so when say 100k instances are done I can go back and trace exactly whats happened in each thread any if any didnt work, why.

    Long story short with the current setup..... Im getting data integrity issues between threads, threads will go off on the wrong course because UB seems not to preserving data intergirty within each thread and trying to debug is a nightmare as I cant even be 100% sure about the debugging info as Im writing to files as this appears to be mixed up too.

    So I want to eliminate the thread > in new browser and replace this with a loop where the loop spawns 10 browsers each running through the same code with each instance able to update a global list for the key info and final result for that instance once completed.

    I dont want to use the client server architecture with a server spawning 10 bots each with a single browser instance because although this seems perfect for data integrity, the time overhead loading a new client bot 100k times will be add up too much especially when some instances currently close as fast as 1-3 secs as theres nothing to be done.

    So, it sounds like this would be something this PI can do even just by using the ability for one bot to spawn multiple independent browser instances with no need for the UB "thread > in new browser" with each browser instance updating a global list for the key info and final result of each of the 10 concurrent browser instances until all 100k instances have completed.

    Now for the questions......

    1. In the above scenario, if I have 10 browser instances each running code assiging values to the same named var, is data integrity preserved in each occurance of that same named var simply by merit of the fact each is in a truly isolated/seperate browser session?

    2. If so, I assume all I would need to do then would be to "lock" the section of code that writes to the global list and that would ensure the 10 browser instances would still in the main run concurently and the only delay would be if/when the adding of the data list item to the global list was taking place if 2 or more  instances tried to add to list at the same time?

    If you can confim/correct this I'd be very grateful :)

×
×
  • Create New...