Jump to content
UBot Underground

Recommended Posts

I have a bot that needs to scrape the seconds (#s), minutes (#m) or hours (#h) that is listed on the first 15 search results.  It just won't save the times properly at all-I can't get them separated into separate rows or do a replace to add in commas or anything.  I'm very stuck-what am I doing wrong?

 

Here is my code:

clear list(%search term)
clear list(%times)
navigate("https://twitter.com/search-home", "Wait")
type text(<id="search-home-input">, "puppy", "Standard")
click(<class="button btn primary-btn submit selected search-btn">, "Left Click", "No")
wait for element(<title="Find the most recent tweets">, "", "Appear")
click(<title="Find the most recent tweets">, "Left Click", "No")
wait for element(<title="Find the most recent tweets">, "", "Appear")
set(#time, $scrape attribute(<span,class="_timestamp js-short-timestamp js-relative-timestamp">, "innerhtml"), "Global")
set(#seconds, $replace(%times, "s", "s, {$new line}"), "Global")
set(#minutes, $replace(#time, "m", "m, {$new line}"), "Global")
add item to list(%times, #time, "Don\'t Delete", "Global")
save to file("C:\\Users\\Katelynn\\Desktop\\times.csv", %times)
set(#term, $scrape attribute(<class="search-query">, "innertext"), "Global")
loop(15) {
    add item to list(%search term, #term, "Don\'t Delete", "Global")
}
save to file("C:\\Users\\Katelynn\\Desktop\\search.txt", %search term)

Edited by katelynnm
  • Like 1
Link to post
Share on other sites

Not exactly sure if I understand what you are trying to accomplish, but if you are trying to get all of the times into their own row of a list, I made a few changes to your code:

clear list(%search term)
clear list(%times)
navigate("https://twitter.com/search-home", "Wait")
type text(<id="search-home-input">, "puppy", "Standard")
click(<class="button btn primary-btn submit selected search-btn">, "Left Click", "No")
wait for element(<title="Find the most recent tweets">, "", "Appear")
click(<title="Find the most recent tweets">, "Left Click", "No")
wait for element(<title="Find the most recent tweets">, "", "Appear")
add list to list(%times, $scrape attribute(<data-time=w"*">, "innertext"), "Delete", "Global")
save to file("{$special folder("Desktop")}\\times.csv", %times)
set(#term, $scrape attribute(<class="search-query">, "innertext"), "Global")
loop(15) {
    add item to list(%search term, #term, "Don\'t Delete", "Global")
}
save to file("{$special folder("Desktop")}\\search.txt", %search term)

  • Like 1
Link to post
Share on other sites

Hi Steve-that works great!  Is there any way that I can now set a limit so that it only scrapes the first 15 times?

 

It looked like by default it was already doing that? It seemed like it was only loading 15 results unless you scroll down on the page after it loads? 

If you need to make sure its only the first 15, the code will need to be modified, and it will prob. require the use offsets and adding each item to the list individually.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...