Click Link (Text) But Avoid All Paid-Advertising Link (Same Text)?

[bhat 0]

The text is the same, there are some slight differences like text that says "sponsored" right above the link and the URL is different as well.  Is there a way to make ubot avoid clicking all sponsored-paid-advertising links with the same link text?  Please let me know, thanks in advance.

 

Bhat

 

[HelloInsomnia 1103]

In general you may want to scrape the outer container of the link and see if that says sponsored somewhere.

 

But if you post the site somebody can probably give you the code to check it.

[bhat 0]

The ads are very similar to Amazon ads when you do a search on their search engine. Search "blue spatula" as an example search results: They will have ads on the main search engine results and smaller ads on the side boxes all with the same text with the word "sponsored" above it.  Would you please show an example there of a code that clicks a link but avoids all ads on the search results and side boxes of the same link text name?

Edited May 19, 2017 by bhat

[HelloInsomnia 1103]

Sure thing, there's a lot of ways to do it but without using any plugins or anything then regex is your best friend:

navigate("https://www.amazon.com/s/?url=search-alias%3Daps&field-keywords=blue+spatula","Wait")
wait(2)
clear list(%resultContainers)
add list to list(%resultContainers,$scrape attribute(<id=w"result_*">,"outerhtml"),"Delete","Global")
clear list(%results)
loop($list total(%resultContainers)) {
    set(#resultContainer,$next list item(%resultContainers),"Global")
    if($contains(#resultContainer,"Sponsored")) {
        then {
            comment("Sponsored item")
        }
        else {
            add item to list(%results,$find regular expression(#resultContainer,"(?<=href=\\\")[^\"]*?(?=\\\"><h2)"),"Don\'t Delete","Global")
        }
    }
}

[bhat 0]

Ah I'm even more confused as I don't quite understand that code.  So my ubot code is set up right now to navigate to the page, then search a specified keyword, then wait for page to load so the search results are displayed.  Then I have a "loop while" code  to keep clicking "next" until my link text is found and then to click on the link text.  So, in order to avoid clicking on sponsored paid ads that may appear that have the same exact link text, do I just copy and paste the code you gave before my "loop while" code?  Or where do I put this code?

 

clear list(%resultContainers)
add list to list(%resultContainers,$scrape attribute(<id=w"result_*">,"outerhtml"),"Delete","Global")
clear list(%results)
loop($list total(%resultContainers)) {
set(#resultContainer,$next list item(%resultContainers),"Global")
if($contains(#resultContainer,"Sponsored")) {
then {
comment("Sponsored item")
}
else {
add item to list(%results,$find regular expression(#resultContainer,"(?<=href=\\\")[^\"]*?(?=\\\"><h2)"),"Don\'t Delete","Global")
}
}
}

 

I am not too familiar with this coding, do I need to change anything in the code? Can you help me understand the code a little better?  Is your code recognizing the "sponsored" wording on the top of the link text and then letting ubot ignore the link text directly below "sponsored"?  If so, then what about the side boxes where there are several "sponsored" ads displayed but only one "sponsored" text on the way top, will that be a problem that there is only one "sponsored" text above them all?

 

I am just starting out to learn uBot myself, so I'm sorry for all the questions.  I really appreciate your help.  Thanks again

 

Bhat

Edited May 20, 2017 by bhat

[HelloInsomnia 1103]

That code will only work for Amazon search results so it won't work on any other website.

 

Basically what it is doing is scraping the result containers - so like Amazon makes a big wide box for each result with the picture and title and stuff like that, this gets that big container. Each result in the case of Amazon has its own container - and its like this for most websites which display results. But the containers on other websites will be different.

 

To try and visualize this you can open the search results above in Chrome (or Firefox but I like Chrome) and then right click on the picture of a result and then click "inspect" then move the mouse up the HTML tree and you will see the items on the page highlight, once you move up far enough and get to a line like this one: <li id="result_0" data-asin="B00VSSX9FG" class="s-result-item celwidget ">

 

You will see the entire result container hightlighted, and if you close that tree item you should see the rest of the result containers stacked and you can highlight over the closed tree items to see each result container on the page highlighted.

 

So once we get the outer containers we scraper the outerhtml of each and put it into a list, this way we have all the code inside of each one.

 

Then we look inside of each one for a word like "Sponsored" and ignore those items.

 

Finally, we get the desired link in this case using a regular expression but otherwise it would be good to use something like xpath here (but that would required using a plugin - Dan has a free one here: http://www.bot-factory.com/free-ubotstudio-xpath-plugin/)

 

So this post is getting kind of long and I will continue in the next one about how to specifically help your situation.

[HelloInsomnia 1103]

So in order to find your link - but not the sponsored one we need to slightly modify the code. We can look for anything but in the case below we look for a title (or just a part of it if you want) and then only add those links to the result list.

Note: there is a way to then go on and click it here but I don't want this to get too confusing so if you need help with that too first try to understand this then let me know.

 

So we have to modify the code but we can keep most of it, we want to keep the part that ignores the sponsored listings so they don't get used and then in the non sponsored listings we do one more check for the title (or whatever you want to find in that container just keep in mind it's HTML so best to look inside of a single element like the title):

 

and the modified code:

navigate("https://www.amazon.com/s/?url=search-alias%3Daps&field-keywords=blue+spatula","Wait")
wait(2)
clear list(%resultContainers)
add list to list(%resultContainers,$scrape attribute(<id=w"result_*">,"outerhtml"),"Delete","Global")
clear list(%results)
loop($list total(%resultContainers)) {
    set(#resultContainer,$next list item(%resultContainers),"Global")
    if($contains(#resultContainer,"Sponsored")) {
        then {
            comment("Sponsored item")
        }
        else {
            if($contains(#resultContainer,"VAPSINT")) {
                then {
                    add item to list(%results,$find regular expression(#resultContainer,"(?<=href=\\\")[^\"]*?(?=\\\"><h2)"),"Don\'t Delete","Global")
                }
                else {
                }
            }
        }
    }
}

I suggest trying it a few times to see how it works in action. In this case it looks for the word "VAPSINT" because that is one of the results I see it starts with that brand name and when I run it that is the only result that comes back because of it.

[bhat 0]

Hi, this is the code I need but I am not sure exactly where to put it in my script. Would you please PM me your skype id?  I would like to show you my script, and if you can help show me where to put this code in my script and also look if my script can be improved on then that would be much appreciated, and I can pay you for your help as well.  Thank you

Edited May 29, 2017 by bhat

[HelloInsomnia 1103]

Hi, this is the code I need but I am not sure exactly where to put it in my script. Would you please PM me your skype id?  I would like to show you my script, and if you can help show me where to put this code in my script and also look if my script can be improved on then that would be much appreciated, and I can pay you for your help as well.  Thank you

 

I don't Skype but you can PM it to me and I can have a look.

[LoWrIdErTJ - BotGuru 904]

In this situation I would usually use the change attribute command, and remove the paid links from the page then capture all the links that im wanting to possibly click on then randomize the position based on 0 - (list total -1)