Jump to content
UBot Underground

Document Text Doesn't Refresh After Page Scroll


Recommended Posts

Hi there, I have found a bug in the Ubot software where the browser function $document text doesn't refresh after a page scroll. I'm trying to scroll down and scrape a page but I found out that it always scrapes only the first 22 results even though I see the page being scrolled down and see all the other results. Ubot, in the loop scrapes just the first 22 results because it scrapes from $document text which doesn't refresh itself after the scroll. When I manually check the source code, it shows the unrefreshed content - the same one as $document text and when I check the view generated source, it correctly shows the next loaded page with new data.


 


So the question is: how can I scrape from the generated source code which is being refreshed as opposed to the classic source code ($document text)?


 


Thanks so much in advance for any help!


Link to post
Share on other sites

Yes, I have actually started doing it differently and now I just don't use the $document text function but scrape the current source code and then scrape the necessary text from there.

  • Like 1
Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...