Jump to content



Denethor

Member Since 01 Aug 2013
Offline Last Active May 23 2019 02:18 PM
-----

Topics I've Started

Verify Emails Give Error "length Cannot Be Less Than Zero. Parameter Name: Length"

23 May 2019 - 02:02 PM

Using the built in email verifier the script crashes randomly with this error.

simple code, so not much to debug on my end.

 

connect to mail server("IMAP With SSL","userremoved@gmail.com","passwordremoved","imap.gmail.com",993) {

        verify emails("Love .*","https://.*","Processed Verification Emails")
    }
 
So it logs in successfully to gmail, downloads all the email headers (hundreds match the query above) and looks for any with a subject that starts with "Love " and any thing after it (the .* wildcard). If it finds a subject that matches it clicks the first link it finds (the only one in the email) that starts "https://"
Then it deletes the email once it has been clicked.
 
Sometimes it processes one email and craps out with the error, sometimes it does 50 or more before it throws the error.
"length cannot be less than zero. Parameter name: length"
The function is pretty useless if it doesn't work and it can't recover from an error.
 
Any thoughts on what "length" its looking for?
 

Any Way To Detect If All Threads Are Finished Before Proceeding?

25 July 2018 - 07:48 AM

Is there anyway to detect that all threads have finished before the main script continues?

I have a script that spawns 21 threads, two at a time, with a minute pause between each set of two (just so everything isn't launching at once).

loop while($table cell(&NEW,#a,0) != "") {
     loop(2) {
         MyLogin($table cell(&NEW,#a,0), $table cell(&NEW,#a,1), $table cell(&NEW,#a,2), $table cell(&NEW,#a,3), $table cell(&NEW,#a,4), $table cell(&NEW,#a,7))
         increment(#a)
     }
wait(60)
}

The problem I have is some threads take longer to complete than others and during testing I've seen the script start to move on to the next step in the process before all the threads had finished. Which is an issue because the next step needs data that is being retrieved from the threaded part.


Save Downloaded File Very Slow When Threaded

05 June 2018 - 09:03 AM

Here is a basic script I tried to thread to speed up the time to process all records. It's not resource issue as I can limit it to one thread at a time and it still doesn't work in a timely fashion. The same code, not in a thread, saves instantly. Thread it and it takes 10 to 14 sec for the file to appear in the folder after the file is actually downloaded.

loop while($table cell(&NEW,#a,0) != "") {
    loop(2) {
        MyLogin($table cell(&NEW,#a,0), $table cell(&NEW,#a,1), $table cell(&NEW,#a,2), $table cell(&NEW,#a,3), $table cell(&NEW,#a,4))
        increment(#a)
    }
    wait(240)
}
define MyLogin(#id, #pass, #City, #CityURL, #Expected) {
    thread {
        set(#CSVfilename,$add($add("C:\\temp",#City),".csv"),"Local")
        set(#ResultFile,$add($add("C:\\Result",#City),".jpg"),"Local")
        in new browser {
            navigate(#CityURL,"Wait")
            wait for element(<login link>,"","Appear")
            wait for element($element offset(<name="remember_me">,1),"","Appear")
            change checkbox($element offset(<name="remember_me">,1),"Unchecked")
            type text($element offset(<email field>,1),#id,"Standard")
            wait(2)
            type text($element offset(<password field>,1),#pass,"Standard")
            wait(2)
            click($element offset(<login button>,1),"Left Click","No")
            wait for element(<class="analysis-body">,"","Appear")
            navigate(#CityURL,"Wait")
            wait(5)
            set(#FileSave,0,"Local")
            set(#TryCount,0,"Local")
            set(#MoreTime,0,"Local")
            loop while(#FileSave = 0) {
                set(#ErrorSeen,$scrape attribute($element offset(<tagname="h1">,0),"innertext"),"Local")
                if(#ErrorSeen = "Log in") {
                    then {
                        set(#FileSave,1,"Local")
                    }
                    else {
                    }
                }
                set(#ErrorSeen,$url,"Local")
                if(#ErrorSeen = "https://mysite.com/about") {
                    then {
                        set(#FileSave,1,"Local")
                    }
                    else {
                    }
                }
                if(#FileSave = 0) {
                    then {
                        wait for element(<class="Icon Icon--featherDownload">,"","Appear")
                        click(<class="Icon Icon--caretDown controls-caret">,"Left Click","No")
                        click(<innertext="Last 7 Days">,"Left Click","No")
                        wait(5)
                        click(<class="btn btn-default ladda-button">,"Left Click","No")
                        wait(2)
                        save downloaded file(#CSVfilename)
                        set(#MoreTime,0,"Local")
                        loop while(#MoreTime = 0) {
                            set(#ButtonState,$scrape attribute(<id="export">,"outerhtml"),"Local")
                            if($find index(#ButtonState,"data-style=\"expand-right\" disabled=\"\" data-loading=\"\">") < 0) {
                                then {
                                    comment("Button State changed back and file is supposedly Saved")
                                    if($file exists(#CSVfilename)) {
                                        then {
                                            alert("File exists keep verifing")
                                            set(#FileSize,$plugin function("Advanced Shell.dll", "$shell batch hidden", "for %I in ({#CSVfilename}) do @echo.%~zI"),"Local")
                                            alert(#FileSize)
                                            if(#FileSize = 0) {
                                                then {
                                                    delete file(#CSVfilename)
                                                    set(#MoreTime,1,"Local")
                                                    alert("Delete File, set MoreTime 1")
                                                }
                                                else {
                                                    clear table(&CSVfilename)
                                                    create table from file(#CSVfilename,&CSVfilename)
                                                    set(#Actual,$table cell(&CSVfilename,1,1),"Local")
                                                    set(#IndexFound,$find index(#Actual,#Expected),"Local")
                                                    comment("compare to empty here")
                                                    if($comparison(&CSVfilename,"= Equals",&EmptyFile)) {
                                                        then {
                                                            comment("File was saved and contains nothing but headers, no data")
                                                            set table cell(&Login,$plugin function("TableCommands.dll", "$table search", &Login, #CityURL, "Row Index"),5,"true")
                                                            set table cell(&Login,$plugin function("TableCommands.dll", "$table search", &Login, #CityURL, "Row Index"),6,$now)
                                                            set(#FileSave,1,"Local")
                                                            set(#MoreTime,1,"Local")
                                                        }
                                                        else {
                                                            Comment("File saved contians data continue chekcing")
                                                            if($find index(#Actual,#Expected) = 0) {
                                                                then {
                                                                    comment("File contains data expected, exit filesave loop, exit moretime loop")
                                                                    set table cell(&Login,$plugin function("TableCommands.dll", "$table search", &Login, #CityURL, "Row Index"),5,"true")
                                                                    set table cell(&Login,$plugin function("TableCommands.dll", "$table search", &Login, #CityURL, "Row Index"),6,$now)
                                                                    set(#FileSave,1,"Local")
                                                                    set(#MoreTime,1,"Local")
                                                                }
                                                                else {
                                                                    comment("File contains bad data, delete, end loop, try again")
                                                                    set table cell(&Login,$plugin function("TableCommands.dll", "$table search", &Login, #CityURL, "Row Index"),5,"false")
                                                                    set table cell(&Login,$plugin function("TableCommands.dll", "$table search", &Login, #CityURL, "Row Index"),6,$now)
                                                                    delete file(#CSVfilename)
                                                                    set(#MoreTime,1,"Local")
                                                                    alert("File contains bad data,FS0 MT1, delete file")
                                                                }
                                                            }
                                                        }
                                                        save to file("C:\\SpecLog.txt",&Login)
                                                        alert("Save Login table")
                                                    }
                                                }
                                            }
                                        }
                                        else {
                                            set(#MoreTime,1,"Local")
                                            alert("File Doesn't exist, MT1")
                                        }
                                    }
                                }
                                else {
                                    comment("Data Still Loading")
                                    wait(4)
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

I added all the alerts to see where the thread was failing since you can't see what's going on inside of a thread as far as I know.

It's very basic script, login to site, select time frame, wait for download button to have be in the ready state, click download button, check status of button (when the file is being compiled on the server side the button state changes, so I use that to know when to look for the local file), I pause 4 sec between each check just to not eat up too many resources checking constantly. If the button changes back to ready that signals the file has been sent to the local browser, I detect that and look for the file from the Save Downloaded File command. Threaded like above it always fails to find the file it supposedly saved and tries to download it again, which then fails because the file then already exists. I have left an file explorer window open to see when the file actually gets written out and when not threaded it happens instantly. When threaded, I have to add a 10 or more wait to ensure the file has been saved. 

 

Am I missing something here or is this a bug that needs to be reported?


Tables And Lists In A Thread Local Or Global?

24 May 2018 - 06:56 AM

If I create a table or list in a thread, is that local to that thread or global?

I have a multi threaded bot that scrapes a page and stores stuff in a temp list to compare it to a known value I'm looking for.

The issue is I think the temp table is being overwritten by the other threads so the compare fails because its now being compared to some other threads expected outcome.

IF its global and not local, how can I get around that?

I don't see how I can generate a random table/list name and know what to use later?