Jump to content
UBot Underground

Finding And Outputting Search Page Number (Google)


Recommended Posts

As I create my bot to scrape for specific data on the search engines, when the data is found, I want to output the page number that it is found on. First to an "alert", and then to file (once I understand how to do an alert with this output, I can easily figure how to output it to file).

 

I am having trouble figuring out how to make the bot find out which page it is on when it finds the item it is programmed for, then output it to an alert. I am guessing that the output part will be fairly easy, once I understand the logic needed to actually seek out and confirm the page number. Is this even possible?

Link to post
Share on other sites

So I figured it out on my own, here for any others having similar issues:

set(#pagenum,$scrape attribute(<innertext=w"Page * ">,"innertext"),"Global")
                    alert("Url is found at: {#pagenum}")
                    stop script

However, for some odd reason, it  shows up twice in the alert and I want to get rid of all of the info except the number.

 

I have a similar issue with bing:

            if($search page(#surl)) {
                then {
                    set(#pagenum,$scrape attribute(<class="sb_pagS">,"outerhtml"),"Global")
                    alert("Url is found! {#pagenum}")
                    stop script

It shows "sb_pagS">9</a> (the number 9 being the page), I cannot figure a better way to scrape bing for the page number, or display without the added text. Any ideas on what I am doing wrong?

Link to post
Share on other sites

Try this code 


​ui stat monitor("Page Counter:",#pagecounter)
navigate("http://www.google.com/search?q=how+to+make+money+online","Wait")
wait for browser event("Everything Loaded","")
set(#pagecounter,1,"Global")
loop while($not($exists(<innertext="How to Make Money Online - Huffington Post">))) {
    increment(#pagecounter)
    click(<innertext="Next">,"Left Click","No")
    wait for browser event("Everything Loaded","")
    wait($rand(2,3))
}

  • Like 1
Link to post
Share on other sites

Page Counter will show u the current page u are on. while rest is a simple loop while statement to find a specific element on the page and increase the page counter.

Here i am using innertext, u can edit the example as per ur requirements. 

 

Hope it helps :)

  • Like 1
Link to post
Share on other sites

Page Counter will show u the current page u are on. while rest is a simple loop while statement to find a specific element on the page and increase the page counter.

 

Here i am using innertext, u can edit the example as per ur requirements. 

 

Hope it helps :)

You know what, I actually didn't even think about stat counter. I have been outputting info with alerts, and do want to stick to them, but I think that stat counter may be a good fit as well! Ahh the choices, lol! Thank you!

Link to post
Share on other sites

I am allowing the end user to select a specific page count to loop the bot to run. How could I output a message to them when that count is reached? Along with the counter of course, I just also want there to be a check and then an alert, before the script stops.

Link to post
Share on other sites

You know what, I actually didn't even think about stat counter. I have been outputting info with alerts, and do want to stick to them, but I think that stat counter may be a good fit as well! Ahh the choices, lol! Thank you!

 

Typically you only want to use an alert when the user clicks on something and expects a result instantly or at the end of the program to notify the user the program is finished or at the very beginning if the user input isn't correct. The reason being - an alert prevents the current script from running any further until the alert box is closed. So for any information that is going to be changing several times throughout the script (like what page number the scraper is on) it's best to use a stat monitor instead.

 

I am allowing the end user to select a specific page count to loop the bot to run. How could I output a message to them when that count is reached? Along with the counter of course, I just also want there to be a check and then an alert, before the script stops.

 

At the end of your script is where it's a good idea to use an alert so you could say:

alert("Finished! Scraped {#pagecounter} pages.")

Or anything you like in there really, but you can use variables in the alert to notify the user of the page that has been on in this case I used the variable from Abbas' code earlier.

  • Like 1
Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...