Jump to content
UBot Underground

Variable In Scrape Attribute


Recommended Posts

I am trying to just dynamically populate the id/class to scrape based on the variable input

 

- It looks correct in Node view, but when I run it does not worl

- I switch to code view and have to always fix it, but then when I immediately run from there it does work.

 

Any ideas?

 

Thanks!

 

post-14885-0-91834000-1509388545_thumb.pngpost-14885-0-17435500-1509388546_thumb.pngpost-14885-0-33942600-1509388546_thumb.png

Link to post
Share on other sites

 

In node view try making it:

<id=#elementScrape>

I think the variable part stuck ans is working, for some reason it sometimes works the way you posted sometime I need to go in code view and use "{#var}"

 

Now I am just trying to get this to work in a define function. If I run the node manually inside the function it works fine but when I just call the function it does not work (neither populate the set scrape var, nor the return variable)

Link to post
Share on other sites

I think the variable part stuck ans is working, for some reason it sometimes works the way you posted sometime I need to go in code view and use "{#var}"

 

Now I am just trying to get this to work in a define function. If I run the node manually inside the function it works fine but when I just call the function it does not work (neither populate the set scrape var, nor the return variable)

 

Can you post the code of that define?

Link to post
Share on other sites

Sure thing. I am not sure why it is not working with the variable, as I see the var has the right value in debug. When I use that exact value directly passed it does work, but obviously I want to keep it dynamic

 

So tried to just return #content , but as scrape does not work value is empty as well. When I manually run the nodes for scraping it works just fine


define $Simple scrape of content(#URL, #elementScrape) {
    comment("Set passed css selector to variable")
    set(#elementScrape,#elementScrape,"Global")
    comment("Set passed URL to variable")
    set(#URL,#URL,"Global")
    comment("We are grabbing first character of element")
    set(#kindOfElement,$eval("var element = \"{#elementScrape}\";
var element1 = element.charAt(0);
element1;"),"Global")
    comment("Now navigate to the URL")
    navigate(#URL,"Wait")
    wait for browser event("Everything Loaded","")
    if($comparison(#kindOfElement,"= Equals",".")) {
        then {
            set(#elementScrape,$eval("var element = \"{#elementScrape}\";
var element1 = element.substr(1).trim();
element1;"),"Global")
            wait(2)
            set(#content,$scrape attribute(<class=#elementScrape>,"innertext"),"Global")
            wait(1)
            alert(#content)
            return($scrape attribute(<class=#elementScrape>,"innertext"))
        }
        else {
        }
    }
}

Thanks !

Link to post
Share on other sites

I tried the same with http get and xpath, but once I get the variable in the xpath I get the same issue (in addition to noticing that http get does not work properly within the function)

 

Thanks again for the help

Link to post
Share on other sites

Yes thats what I wanted to do. For some reason that just did not work all day yesterday IN THE DEFINE. It always worked just as a regular command, but in an effort to get everything more organized I wanted to start using more defines. For now it seems today the define actually works , using http get and xpath parsing (does not render page and gets hard coded xpath in variable)

define $scrape content {
    set(#page,$plugin function("HTTP post.dll", "$http get", "https://www.fpl.com/smart-meters/you.html", "", "", "", ""),"Global")
    set(#xpath,"//*[contains(concat( \" \", @class, \" \" ), concat( \" \", \"main-content\", \" \" ))]","Global")
    set(#content,$plugin function("HeopasCustom.dll", "$Heopas Xpath Parser", #page, #xpath, "InnerText", ""),"Global")
    return(#content)
}

Link to post
Share on other sites

@consultingad

 

One suggestion that I will offer here is to test the code thoroughly that you want to put inside of a Define.  I realize that you are passing data to it but debugging a Define is near impossible because you cannot follow the execution of the code (the purple box that moves from command to command).

 

When ever I ask noobs if they have done this I get the answer no.  Regardless of their answer if they are wanting me to debug it then I always pull the code out and run it in the environment.  I cannot tell you how many times I have found errors like that.

 

As I said, these are just suggestions.

 

Buddy

Link to post
Share on other sites

Hi Buddy,

 

Yes, that is indeed good to remember. However, in my case the single command in the define worked when I clicked "run node" as well as in loop they just would not want to run when I called my custom define. 

For some reason it seems to be working now, the other day even some restarts did not solve that.

 

On a side note, is there any way to even use the private bot bank? (I'd otherwise just do all defines in dedicated tabs, then use that bot as an include as needed.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...