Jon12345 0 Posted July 29, 2011 Report Share Posted July 29, 2011 When I go to Twitter, I see 20 odd tweets ready to scrape. But if I scroll to the bottom of the page, it keeps adding more Tweets. Before, I am sure there was a button that you used to click for more tweets, right? So what can you do now? If I want to scrape I sizeable page of tweets, I figure I will need to load the page and then scroll that page a few times, then do the scrape. How do I get ubot to scroll so more tweets are added? Thanks, Jon Quote Link to post Share on other sites
turbolapp 5 Posted August 6, 2011 Report Share Posted August 6, 2011 run javascript location.reload() Assuming you're searching through popular results, like trends or something, you need only to refresh the page for new results. Quote Link to post Share on other sites
JohnB 255 Posted August 6, 2011 Report Share Posted August 6, 2011 I just logged in and it looks like it just keeps loading and loading...you may just have to put in a significant delay and wait for them (i.e. 60 seconds or more) John Quote Link to post Share on other sites
LoWrIdErTJ - BotGuru 904 Posted August 6, 2011 Report Share Posted August 6, 2011 run javascriptwindow.scrollTo(0,document.body.scrollHeight);delay or wait finishrun it again window.scrollTo(0,document.body.scrollHeight);delay or wait finish repeat process as many times as you want till enough are showing. Have fun. TJ 2 Quote Link to post Share on other sites
amul 1 Posted August 28, 2011 Report Share Posted August 28, 2011 run javascriptwindow.scrollTo(0,document.body.scrollHeight);delay or wait finishrun it again window.scrollTo(0,document.body.scrollHeight);delay or wait finish repeat process as many times as you want till enough are showing. Have fun. TJ I was trying to figure out how to do get javascript to load this..Now I've to do some regex for friend ids.. to message. thanks TJ!! Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.