Ui Dropdown Auto Generated From A Scrape Of A Dropdown

[christojuan 5]

Hi,

I'm trying to populate a ui drop down from a scrape of a dropdown and would appreciate any guidance / help.

 

Here is what I have so far... 

 

navigate("http://www.bazoogle3.com/testscrape/","Wait")

wait(5)

set(#var02,$scrape attribute(<id=w"reportMonthSelector_chzn_o_*">,"innertext"),"Global")

ui drop down("Select Start date",#var02,#var02)

 

I'd like the list of dates to populate the ui dropdown, but as you as see it ends up being one drop down option that includes the list of all of the dates in one row.

 

Any help/input/guidance on a fix would be appreciated.   Please note that the next step in my process will be to have my scraper select each of the previous 12 months, one-by-one,  and then scrape some specific data from each result (in case that affects your guidance/help).

 

Thanks in advance for any help offered.

Chris

Edited June 8, 2017 by christojuan

[Code Docta (Nick C.) 638]

Hi,

 

Here are two options using the file management plugin

 

Just put your scrape attribute instead of my fake data.

comment("most likely the data will look like a list
on seperate lines")
set(#scraped data,"one
two
three","Global")
comment("$text from list 
just joins the list with a coma")
set(#list box,$plugin function("File Management.dll", "$listbox dialog", "Choose an item!", "Choose", $text from list($list from text(#scraped data,$new line),",")),"Global")
set(#drop down,$plugin function("File Management.dll", "$dropdown dialog", "Choose item...", "Choose:", $text from list($list from text(#scraped data,$new line),",")),"Global")

Regards,
Nick

file management dd an list box - example.ubot

[christojuan 5]

Thanks very much Nick. I'll dig in this weekend and try make this work.

Edited June 9, 2017 by christojuan

[christojuan 5]

Hey - Thanks very much for your help Nick. While I was not able to download the file management plugin (the site is down and amazon permissions are not allowing downloads). I used an idea from something I saw in your code to do the following and it worked great!

 

navigate("http://bazoogle3.com/testscrape/","Wait")
set(#var1,$scrape attribute(<id=w"reportMonthSelector_chzn_o_*">,"innertext"),"Global")
add list to list(%list1,$list from text(#var1,"
"),"Delete","Global")
ui drop down("Select",$text from list(%list1,","),#var2)

 

Thanks again!

[christojuan 5]

Quick follow up to that.  When I run the script the first time, the drop down does not populate, but when I run it a second time it does.

Does anyone have any insights on why that happens and how I might be able to fix?

Thanks!
Chris

[HelloInsomnia 1103]

Quick follow up to that.  When I run the script the first time, the drop down does not populate, but when I run it a second time it does.

Does anyone have any insights on why that happens and how I might be able to fix?

Thanks!

Chris

 

UI refresh problem I think, if you click to code view and back after the first time does it populate? Or add a new tab and come back. I don't know if there's a solution to that I think somebody has a plugin but maybe there is some other way to do it. I use HTML or XAML GUI to get around this kind of thing.

[Code Docta (Nick C.) 638]

Sorry, you can get it from the store.

 

http://network.ubotstudio.com/store/product-category/plugins/

[christojuan 5]

That was it regarding he refresh!   And Aymen sent me a direct link to the free plugins so I have those now (still my solution works ok for now).

Thanks Nick and NIck!

Edited June 10, 2017 by christojuan

[christojuan 5]

Hey,

Quick follow-up to Nick's code sample. When I finally got Aymen's plugin, I realized how cool this is but... I'm struggling with how to execute the code.

 

I'm probably missing something very obvious, but how can I have the selected dropdown item become the first drop-down selection where those dates previous to the selected dropdown would not be executed.  In other words the first scraped page should be the result of the selected drop down (e.g. 03/2016) and all subsequent should be 02/2016 then 01/2016 and then 12/2015 to the end excluding all dates prior to the 03/2016)

 

(Note that this code will generate 92 over and over. That's OK. on the live intranet site, it will page through the dates properly.)

 

navigate("http://bazoogle3.com/testscrape/","Wait")
clear list(%dates)
add list to list(%dates,$list from text($scrape attribute(<id=w"reportMonthSelector_chzn_o_*">,"innertext"),$new line),"Delete","Global")
set(#drop down,$plugin function("File Management.dll", "$dropdown dialog", "Choose item...", "Choose:", $text from list(%dates,",")),"Global")
loop($list total(%dates)) {
    set(#date NLI,$next list item(%dates),"Global")
    wait(3)
    change dropdown(<value=w"*">,#date NLI)
    set(#var3,$scrape attribute($element offset(<class="stat">,0),"innertext"),"Global")
    set(#var4,$replace(#var3," CONTACTS",$nothing),"Global")
    add item to list(%list2,#var4,"Don\'t Delete","Global")
}

 

Any additional help would be appreciated.

Thanks!
Chris

[HelloInsomnia 1103]

You will have to test this of course but this should check for the date selected then start adding that date and those after to a new list which then gets used instead:

navigate("http://bazoogle3.com/testscrape/","Wait")
clear list(%dates)
add list to list(%dates,$list from text($scrape attribute(<id=w"reportMonthSelector_chzn_o_*">,"innertext"),$new line),"Delete","Global")
set(#drop down,$plugin function("File Management.dll", "$dropdown dialog", "Choose item...", "Choose:", $text from list(%dates,",")),"Global")
clear list(%useableDates)
set(#dateFound,"false","Global")
loop($list total(%dates)) {
    set(#curDate,$next list item(%useableDates),"Global")
    if($comparison(#drop down,"= Equals",#curDate)) {
        then {
            set(#dateFound,"true","Global")
        }
        else {
        }
    }
    if($comparison(#dateFound,"=","true")) {
        then {
            add item to list(%useableDates,#curDate,"Don\'t Delete","Global")
        }
        else {
        }
    }
}
loop($list total(%useableDates)) {
    set(#date NLI,$next list item(%useableDates),"Global")
    wait(3)
    change dropdown(<value=w"*">,#date NLI)
    set(#var3,$scrape attribute($element offset(<class="stat">,0),"innertext"),"Global")
    set(#var4,$replace(#var3," CONTACTS",$nothing),"Global")
    add item to list(%list2,#var4,"Don\'t Delete","Global")
}

[christojuan 5]

Hey Nick - Thanks very much for your response and solution.

 

https://www.screencast.com/t/9554pJKoN3

 

For some reason, I'm getting a script error.

 

I'm not experienced enough to understand how to debug in this scenario.

 

Can you please help?

 

Chris

[HelloInsomnia 1103]

Whoops accidentally switched something around, try this:

navigate("http://bazoogle3.com/testscrape/","Wait")
clear list(%dates)
add list to list(%dates,$list from text($scrape attribute(<id=w"reportMonthSelector_chzn_o_*">,"innertext"),$new line),"Delete","Global")
set(#drop down,$plugin function("File Management.dll", "$dropdown dialog", "Choose item...", "Choose:", $text from list(%dates,",")),"Global")
clear list(%useableDates)
set(#dateFound,"false","Global")
loop($list total(%dates)) {
    set(#curDate,$next list item(%dates),"Global")
    if($comparison(#drop down,"= Equals",#curDate)) {
        then {
            set(#dateFound,"true","Global")
        }
        else {
        }
    }
    if($comparison(#dateFound,"=","true")) {
        then {
            add item to list(%useableDates,#curDate,"Don\'t Delete","Global")
        }
        else {
        }
    }
}
loop($list total(%useableDates)) {
    set(#date NLI,$next list item(%useableDates),"Global")
    wait(3)
    change dropdown(<value=w"*">,#date NLI)
    set(#var3,$scrape attribute($element offset(<class="stat">,0),"innertext"),"Global")
    set(#var4,$replace(#var3," CONTACTS",$nothing),"Global")
    add item to list(%list2,#var4,"Don\'t Delete","Global")
}

[christojuan 5]

Absolutely perfect!  I'll tell you what, it's the selfless help from people like you, and Nick (Code Docta), Dan (bot-factory), Buddy S., Aymen, Deliter, and Valo (and that's just in the few weeks that I have been posting) that has made this work. This is why ubot will work for me and I will continue to subscribe. I love the learning and very much appreciate the help.