Jump to content



Photo

Scraping Instagram Insights Via Business.facebook/creator Studio


  • Please log in to reply
1 reply to this topic

#1 ugakebet

ugakebet

    Advanced Member

  • Fellow UBotter
  • PipPipPip
  • 235 posts
  • OS:Windows 8
  • Total Memory:4Gb
  • Framework:v3.5
  • License:Professional Edition

Posted 07 February 2020 - 12:48 PM

Hi guys,

 

I'm not sure if anyone has been able to do this.. But I have been trying to scrape Insta insights using the Content Studio option provided by FB..

 

I've taken a screenshot of this (though you might have tolog in via your own account to see the HTML code and stuff)

 

Attached File  123.png   84.75KB   0 downloads

 

The problem I'm having with this is as follows:

1. Lazy load - When first loaded, uBOT only picks up the posts that are visible on the screen due to lazy loading. But if I scroll down either manually (or via run 

javascript("window.scrollTo(0,document.body.scrollHeight);"))

, what happens is that the top half of posts now disappear and gets hidden out of view from the HTML so again uBOT's $scrape attribute doesnt pick it up - it only sees ~5- visible posts at a time (ideally I'd want to scrape all 30-50 posts that I have in one shot)...

 

2. Also,I'm using the $scrape attribute feature for the scrapping. Since everything appears visually like a table, in reality the HTML/css is only <div's> rather than an HTML table so uBot doesnt recognize it with $scrape_table. Hence I have to scrape separately the post headings in one list, dates in another list, then the likes + comments in 3rd list and then due to the lazy load from the earlier point I have to manually see if there are any duplicated entries and take them out...

 

Is there a fast way to scrape: Name of post, datetime,likes,comments as column headings & then the data for each of them?



#2 NicheAmbition

NicheAmbition

    Advanced Member

  • Fellow UBotter
  • PipPipPip
  • 33 posts
  • OS:Windows 10
  • Total Memory:More Than 9Gb
  • Framework:v4.5+, unsure
  • License:Developer Edition

Posted 08 February 2020 - 03:04 AM

You may want to try messing with the css height/width settings for the container the data is being displayed in to see if you can forcefully make it larger, possibly bringing more data into view. If more data is not displayed then there is a coded display limit and you may have to continue scrolling to fetch your data.

 

Also, check to see if there is an iframe on the page, because that may actually be where all of the data is. iframes are ancient technology but still great at screwing up the scrape attribute function. 

 

It can be good practice to ctrl+f the pages source code with some of the data you want to scrape, to see if it is being pulled from a script that's loaded in the site's footer. Sometimes the data is actually sitting in a footer script helplessly waiting for you to parse it.

 

And when all else fails you can try the "is it just me" test by seeing if a small tool like listly can scrape it. If that chrome extension can't do it then you know it's a tough scrape.






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users