steelersfan 38 Posted September 2, 2017 Report Share Posted September 2, 2017 I have a bot that scrapes youtube comments, and the way I have it set up currently, it uses ubot to get the data, not http get. I want to use http get, or the YouTube API, because I don't want a video to load on the end user when they run the bot (a very unprofessional outcome!). My first choice is to use http get, but when I get the video url data, no comments come up at all. Is there a way around this, without having to use the API? This would be my preferred method, because the API limits the comments to 100, and I want to get all comments for any given video (sometimes in the thousands!). Now the next choice would be to use http get, to pull the data with the following argument URL: https://www.googleapis.com/youtube/v3/commentThreads?key=ENTER_KEY_HERE&textFormat=plainText&part=snippet&videoId=kffacxfA7G4&maxResults=100 But that would result in only 100 results, with no clear way to fetch more. Now, I searched for a solution to this and found the following answer:https://stackoverflow.com/questions/36585824/how-to-get-all-comments-more-than-100-of-a-video-using-youtube-data-api-v3 citing: https://developers.google.com/youtube/v3/docs/comments/list Where the guy says that the API page "clearly indicates" to utilize "nextPageToken" and "pageInfo", which sounds great, but I am totally lost as to how to implement this! The "answer" was a very vague one with no context as to how to implement the function or use it. So my question here is, how would this be achieved? Obviously, this could apply to other API functions like getting videos from a channel, etc (hopefully). But I just want to understand this first. Anyone know? Quote Link to post Share on other sites
HelloInsomnia 1103 Posted September 3, 2017 Report Share Posted September 3, 2017 It sounds like you need to scrape the nextPageToken string and then it looks like you need to use pageToken as a parameter, so if nextPageToken had the value of XYZ, then your call would become: https://www.googleapis.com/youtube/v3/commentThreads?key=ENTER_KEY_HERE&textFormat=plainText&part=snippet&videoId=kffacxfA7G4&pageToken=XYZ&maxResults=100 You will have to test to see for yourself but this is what is looks like they were saying. Quote Link to post Share on other sites
steelersfan 38 Posted September 3, 2017 Author Report Share Posted September 3, 2017 It sounds like you need to scrape the nextPageToken string and then it looks like you need to use pageToken as a parameter, so if nextPageToken had the value of XYZ, then your call would become: https://www.googleapis.com/youtube/v3/commentThreads?key=ENTER_KEY_HERE&textFormat=plainText&part=snippet&videoId=kffacxfA7G4&pageToken=XYZ&maxResults=100 You will have to test to see for yourself but this is what is looks like they were saying.Hmm, so that string would scrape the first page then go to the next page and scrape it as well? I don't see how it would work to keep going page after page, or would I have to loop that string in ubot, and make it happen over and over until no more results? Quote Link to post Share on other sites
HelloInsomnia 1103 Posted September 3, 2017 Report Share Posted September 3, 2017 You would need to scrape the nextPageToken string. If you have some example output I can take a look and show you how. 1 Quote Link to post Share on other sites
steelersfan 38 Posted September 3, 2017 Author Report Share Posted September 3, 2017 How would I scrape the next page token? I think that is where I am stuck at Quote Link to post Share on other sites
Kreatus (Ubot Ninja) 422 Posted September 4, 2017 Report Share Posted September 4, 2017 Pash have a youtube plugin.You might want to check it http://network.ubotstudio.com/forum/index.php/topic/17384-sell-plugin-advanced-google-youtube-v3/ Quote Link to post Share on other sites
steelersfan 38 Posted September 4, 2017 Author Report Share Posted September 4, 2017 Pash have a youtube plugin.You might want to check it http://network.ubotstudio.com/forum/index.php/topic/17384-sell-plugin-advanced-google-youtube-v3/I have that plugin already. Honestly it is too much trouble to do this with it. It uses oauth as opposed to direct key calls, it has no documentation, and it is limited to the same maximum as this method. It's also sadly a PITA to use, and this method is a lot easier, as it just uses the public key to run API functions in one HTTP GET request. If that plugin had some actual documentation and good training how to set up properly and use, I would recommend it. However, it does not, so I wouldn't recommend it at all. Quote Link to post Share on other sites
HelloInsomnia 1103 Posted September 4, 2017 Report Share Posted September 4, 2017 How would I scrape the next page token? I think that is where I am stuck at You would use the Json Path Parser plugin most likely (it's free): http://network.ubotstudio.com/forum/index.php/topic/16166-free-plugin-jsonpath-parser-plugin/ You can also use regex as well. Like I said if you post some example output like from the get request I can take a look and show you how to scrape it. 1 Quote Link to post Share on other sites
steelersfan 38 Posted September 7, 2017 Author Report Share Posted September 7, 2017 When I run the API in http get. Here is what I get: { "kind": "youtube#commentThreadListResponse", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/F1bK0Wo4VY6v25XGUBJOlK7iXd0\"", "nextPageToken": "QURTSl9pMy04NTVUSE1IOEtVWS0yQlFxUlpPdFFUY2RtLWZ3c0Z5cUkyalNESG93R1ZSM21kTmM2T1p4S3puUG5Dd1FPR3lIQ1pJSWR1VEJCQWItWjJnelowdExZWTBJemlVMm5YOW1TeEJhTWNDQTZWTnFiSzBSYjFQQ3RaNm9TaEI5", "pageInfo": { "totalResults": 10, "resultsPerPage": 10 }, "items": [ { "kind": "youtube#commentThread", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/WDLSlWYZUCyHIGbMzZC8iFqyQJY\"", "id": "z23njbupps3dhrqclacdp4334f01aexntr424c3zcwhw03c010c", "snippet": { "videoId": "2oaRXlpCLxg", "topLevelComment": { "kind": "youtube#comment", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/eHDvqXf-Vx_zGns0T1SWB9U2mSY\"", "id": "z23njbupps3dhrqclacdp4334f01aexntr424c3zcwhw03c010c", "snippet": { "authorDisplayName": "Iris Zizka", "authorProfileImageUrl": "https://yt3.ggpht.com/-HAT-VqFq77s/AAAAAAAAAAI/AAAAAAAAAAA/rbMFz_rdWYc/s28-c-k-no-mo-rj-c0xffffff/photo.jpg", "authorChannelUrl": "http://www.youtube.com/channel/UCATO8LCrlHiHJf-9BZdutwA", "authorChannelId": { "value": "UCATO8LCrlHiHJf-9BZdutwA" }, "videoId": "2oaRXlpCLxg", "textDisplay": "you guys are such faggots, always going on and and on about how the games you are looking at are rare and expensive. yes, we know, you have them, yes thank you for driving up the fucking price of tg16 games single fucking handedly. you assholes.", "textOriginal": "you guys are such faggots, always going on and and on about how the games you are looking at are rare and expensive. yes, we know, you have them, yes thank you for driving up the fucking price of tg16 games single fucking handedly. you assholes.", "canRate": true, "viewerRating": "none", "likeCount": 36, "publishedAt": "2017-09-04T04:22:44.000Z", "updatedAt": "2017-09-04T04:22:44.000Z" } }, "canReply": true, "totalReplyCount": 80, "isPublic": true } }, { "kind": "youtube#commentThread", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/Ts5V6MSu9Q_Z8adK62BbCfs0Gtw\"", "id": "z22rcjvhjvi4i3c3uacdp43b2eef0bac1ymeifkqsxpw03c010c", "snippet": { "videoId": "2oaRXlpCLxg", "topLevelComment": { "kind": "youtube#comment", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/Hdu_ZVz3ydh_K_xT0PcfQnQ0XQQ\"", "id": "z22rcjvhjvi4i3c3uacdp43b2eef0bac1ymeifkqsxpw03c010c", "snippet": { "authorDisplayName": "Overclocked User", "authorProfileImageUrl": "https://yt3.ggpht.com/-4QKqO74kAr0/AAAAAAAAAAI/AAAAAAAAAAA/ZC0oCmb8HMA/s28-c-k-no-mo-rj-c0xffffff/photo.jpg", "authorChannelUrl": "http://www.youtube.com/channel/UCHLIuWB-7d1EmiiOjBYuYsw", "authorChannelId": { "value": "UCHLIuWB-7d1EmiiOjBYuYsw" }, "videoId": "2oaRXlpCLxg", "textDisplay": "Underated console at best.\nBetween bad marketing hardly slim to none proper pr or marketing.\nAnd a slim game library this unappreciated console will always hold a soft spot in my life and heart!", "textOriginal": "Underated console at best.\nBetween bad marketing hardly slim to none proper pr or marketing.\nAnd a slim game library this unappreciated console will always hold a soft spot in my life and heart!", "canRate": true, "viewerRating": "none", "likeCount": 0, "publishedAt": "2017-09-07T01:31:05.000Z", "updatedAt": "2017-09-07T01:31:05.000Z" } }, "canReply": true, "totalReplyCount": 0, "isPublic": true } }, { "kind": "youtube#commentThread", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/1pGt_3UI5yVpSPNj7sg5IGEORx8\"", "id": "z22icjiqcnizi3fji04t1aokgkvmobga2kz3tdt2wovmrk0h00410", "snippet": { "videoId": "2oaRXlpCLxg", "topLevelComment": { "kind": "youtube#comment", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/WazSwSvznlf2s4gumP6qUDi-07A\"", "id": "z22icjiqcnizi3fji04t1aokgkvmobga2kz3tdt2wovmrk0h00410", "snippet": { "authorDisplayName": "Atomicskull", "authorProfileImageUrl": "https://yt3.ggpht.com/-86NESkzhCEs/AAAAAAAAAAI/AAAAAAAAAAA/mC8tsHx8hDA/s28-c-k-no-mo-rj-c0xffffff/photo.jpg", "authorChannelUrl": "http://www.youtube.com/channel/UCRtZGOQQ4Ir5wHTJ2g7nmZg", "authorChannelId": { "value": "UCRtZGOQQ4Ir5wHTJ2g7nmZg" }, "videoId": "2oaRXlpCLxg", "textDisplay": "The arcade version of Bloody Wolf ran on hardware that was basically an upgraded PC Engine. They used a new pallet chip (the PC Engine graphics hardware is actually two chips one that handles tiles and sprites and one that handles the color pallets and generates the video signal) and added a YM2203 FM sound chip and a MSM5205 speech synthesis chip as well as a second HuC6280 CPU (an improved 6502 with some extra instructions and additional features) and twice as much VRAM.", "textOriginal": "The arcade version of Bloody Wolf ran on hardware that was basically an upgraded PC Engine. They used a new pallet chip (the PC Engine graphics hardware is actually two chips one that handles tiles and sprites and one that handles the color pallets and generates the video signal) and added a YM2203 FM sound chip and a MSM5205 speech synthesis chip as well as a second HuC6280 CPU (an improved 6502 with some extra instructions and additional features) and twice as much VRAM.", "canRate": true, "viewerRating": "none", "likeCount": 0, "publishedAt": "2017-09-07T00:36:58.000Z", "updatedAt": "2017-09-07T00:38:41.000Z" } }, "canReply": true, "totalReplyCount": 0, "isPublic": true } }, { "kind": "youtube#commentThread", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/LpL2xEWSMyB7lZJfoG_hDX9fGd4\"", "id": "z22bytl55s3uvzidfacdp434e3gzob45o24fwx0kwxpw03c010c", "snippet": { "videoId": "2oaRXlpCLxg", "topLevelComment": { "kind": "youtube#comment", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/WKIS7eMxuezKSwjIbx1VOcBXUP8\"", "id": "z22bytl55s3uvzidfacdp434e3gzob45o24fwx0kwxpw03c010c", "snippet": { "authorDisplayName": "Percy Saucedo", "authorProfileImageUrl": "https://yt3.ggpht.com/-zqa3TE49wiQ/AAAAAAAAAAI/AAAAAAAAAAA/ht9jONVxlsc/s28-c-k-no-mo-rj-c0xffffff/photo.jpg", "authorChannelUrl": "http://www.youtube.com/channel/UCk1dMsP_fEMKTvmaWfC0OvQ", "authorChannelId": { "value": "UCk1dMsP_fEMKTvmaWfC0OvQ" }, "videoId": "2oaRXlpCLxg", "textDisplay": "BEST SYSTEM EVER!!!!! Back in the day I had the Turbo Duo system. It was far superior than its rivals but to bad it did not gain much popularity in the US. I still have my TurboGrafx 16 though and allot of HueCard games!", "textOriginal": "BEST SYSTEM EVER!!!!! Back in the day I had the Turbo Duo system. It was far superior than its rivals but to bad it did not gain much popularity in the US. I still have my TurboGrafx 16 though and allot of HueCard games!", "canRate": true, "viewerRating": "none", "likeCount": 0, "publishedAt": "2017-09-07T00:11:16.000Z", "updatedAt": "2017-09-07T00:11:16.000Z" } }, "canReply": true, "totalReplyCount": 0, "isPublic": true } }, { "kind": "youtube#commentThread", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/cRZsPPbiRVyZrl1dOuGbfXjCvNg\"", "id": "z23cfbuohsrjuhimracdp4322ziyu1oudsnahqgaoixw03c010c", "snippet": { "videoId": "2oaRXlpCLxg", "topLevelComment": { "kind": "youtube#comment", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/gUDPqOsQAqVHdBwknrPAfuNGy2Q\"", "id": "z23cfbuohsrjuhimracdp4322ziyu1oudsnahqgaoixw03c010c", "snippet": { "authorDisplayName": "miniwar monger", "authorProfileImageUrl": "https://yt3.ggpht.com/-dbvMsBEcKv0/AAAAAAAAAAI/AAAAAAAAAAA/3niAoNM5Z-c/s28-c-k-no-mo-rj-c0xffffff/photo.jpg", "authorChannelUrl": "http://www.youtube.com/channel/UCS-Fsb5EcD-UGtqSTK9ynNA", "authorChannelId": { "value": "UCS-Fsb5EcD-UGtqSTK9ynNA" }, "videoId": "2oaRXlpCLxg", "textDisplay": "As always love you guys.", "textOriginal": "As always love you guys.", "canRate": true, "viewerRating": "none", "likeCount": 0, "publishedAt": "2017-09-06T23:37:52.000Z", "updatedAt": "2017-09-06T23:37:52.000Z" } }, "canReply": true, "totalReplyCount": 0, "isPublic": true } }, { "kind": "youtube#commentThread", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/FTIJ0dnjPJFkGUXTyJGTgno298w\"", "id": "z22rgh24hvz5h5zzo04t1aokgwixnvzqewsdcbc1a1g0rk0h00410", "snippet": { "videoId": "2oaRXlpCLxg", "topLevelComment": { "kind": "youtube#comment", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/7tf7Jv3BG0xXdphhJUMutp2pvJ8\"", "id": "z22rgh24hvz5h5zzo04t1aokgwixnvzqewsdcbc1a1g0rk0h00410", "snippet": { "authorDisplayName": "DARK DRAKONIS", "authorProfileImageUrl": "https://yt3.ggpht.com/-KvB_sBzQ6OE/AAAAAAAAAAI/AAAAAAAAAAA/CKbOhe18c2k/s28-c-k-no-mo-rj-c0xffffff/photo.jpg", "authorChannelUrl": "http://www.youtube.com/channel/UCuGsrnRSeDtz1S6kec-68Mg", "authorChannelId": { "value": "UCuGsrnRSeDtz1S6kec-68Mg" }, "videoId": "2oaRXlpCLxg", "textDisplay": "I always found it funny to see SonSon in MVC2 :D", "textOriginal": "I always found it funny to see SonSon in MVC2 :D", "canRate": true, "viewerRating": "none", "likeCount": 0, "publishedAt": "2017-09-06T22:31:25.000Z", "updatedAt": "2017-09-06T22:31:25.000Z" } }, "canReply": true, "totalReplyCount": 0, "isPublic": true } }, { "kind": "youtube#commentThread", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/c1wkqEYcS6uomc7dkWjuLMe5puE\"", "id": "z23dih1jiwzsc50yc04t1aokglivexmoae0skmibptxork0h00410", "snippet": { "videoId": "2oaRXlpCLxg", "topLevelComment": { "kind": "youtube#comment", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/0uVg_ZV8GwWP45uCD0R0hvmUC58\"", "id": "z23dih1jiwzsc50yc04t1aokglivexmoae0skmibptxork0h00410", "snippet": { "authorDisplayName": "Wayne Winquist", "authorProfileImageUrl": "https://yt3.ggpht.com/-MnZj5WpH1uE/AAAAAAAAAAI/AAAAAAAAAAA/Iey8PAedHf4/s28-c-k-no-mo-rj-c0xffffff/photo.jpg", "authorChannelUrl": "http://www.youtube.com/channel/UCdbn8oKLw6BJUzXw6Ia1Wgg", "authorChannelId": { "value": "UCdbn8oKLw6BJUzXw6Ia1Wgg" }, "videoId": "2oaRXlpCLxg", "textDisplay": "Wait - did Dave just beat Joe with a foam banana? LOL", "textOriginal": "Wait - did Dave just beat Joe with a foam banana? LOL", "canRate": true, "viewerRating": "none", "likeCount": 0, "publishedAt": "2017-09-06T21:04:35.000Z", "updatedAt": "2017-09-06T21:04:35.000Z" } }, "canReply": true, "totalReplyCount": 0, "isPublic": true } }, { "kind": "youtube#commentThread", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/gu1VCayX1KjrN58lUFEGd72XKSQ\"", "id": "z23qijvbhmjufneil04t1aokgyul3xwphlc1qih5talork0h00410", "snippet": { "videoId": "2oaRXlpCLxg", "topLevelComment": { "kind": "youtube#comment", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/hBUx7O_clbhKoALf0XjCfSjsUtk\"", "id": "z23qijvbhmjufneil04t1aokgyul3xwphlc1qih5talork0h00410", "snippet": { "authorDisplayName": "Peter James Ledwith", "authorProfileImageUrl": "https://yt3.ggpht.com/-EESzKH6-9YE/AAAAAAAAAAI/AAAAAAAAAAA/h6qF8SRY8G0/s28-c-k-no-mo-rj-c0xffffff/photo.jpg", "authorChannelUrl": "http://www.youtube.com/channel/UCCe_oLT4aWDHSbXLg2bjQLQ", "authorChannelId": { "value": "UCCe_oLT4aWDHSbXLg2bjQLQ" }, "videoId": "2oaRXlpCLxg", "textDisplay": "great video. il never get to play most of these obscure games except through emulation. good to see them here.", "textOriginal": "great video. il never get to play most of these obscure games except through emulation. good to see them here.", "canRate": true, "viewerRating": "none", "likeCount": 0, "publishedAt": "2017-09-06T20:47:58.000Z", "updatedAt": "2017-09-06T20:47:58.000Z" } }, "canReply": true, "totalReplyCount": 0, "isPublic": true } }, { "kind": "youtube#commentThread", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/VHh4T8duquv9AIjwkK4rc-L5LMY\"", "id": "z23tx55jzrm1x5xfsacdp430jpmd14x4w034ookoqk1w03c010c", "snippet": { "videoId": "2oaRXlpCLxg", "topLevelComment": { "kind": "youtube#comment", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/yIcAoqRoWtuJYI_xzJv7BILERII\"", "id": "z23tx55jzrm1x5xfsacdp430jpmd14x4w034ookoqk1w03c010c", "snippet": { "authorDisplayName": "Oldtech81", "authorProfileImageUrl": "https://yt3.ggpht.com/--7M2uTAdM1o/AAAAAAAAAAI/AAAAAAAAAAA/I0bbdSnpvTQ/s28-c-k-no-mo-rj-c0xffffff/photo.jpg", "authorChannelUrl": "http://www.youtube.com/channel/UC-xa8NmzxdPINtuGWr2CvZQ", "authorChannelId": { "value": "UC-xa8NmzxdPINtuGWr2CvZQ" }, "videoId": "2oaRXlpCLxg", "textDisplay": "Awesome video as always!", "textOriginal": "Awesome video as always!", "canRate": true, "viewerRating": "none", "likeCount": 0, "publishedAt": "2017-09-06T17:47:56.000Z", "updatedAt": "2017-09-06T17:47:56.000Z" } }, "canReply": true, "totalReplyCount": 0, "isPublic": true } }, { "kind": "youtube#commentThread", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/lJvxOXA6sBvNLZIlHSARGJJsI6Y\"", "id": "z232yx5xsnaisboai04t1aokgx1pectc15esuqfsxagark0h00410", "snippet": { "videoId": "2oaRXlpCLxg", "topLevelComment": { "kind": "youtube#comment", "etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/I7bydK-D1G_fmqTVBLYladoQS7I\"", "id": "z232yx5xsnaisboai04t1aokgx1pectc15esuqfsxagark0h00410", "snippet": { "authorDisplayName": "Jd Combs", "authorProfileImageUrl": "https://yt3.ggpht.com/-ijh6HOPCYuc/AAAAAAAAAAI/AAAAAAAAAAA/-dpPhBHmi2Y/s28-c-k-no-mo-rj-c0xffffff/photo.jpg", "authorChannelUrl": "http://www.youtube.com/channel/UCmTBtCkkEgz4-wo035PEH2A", "authorChannelId": { "value": "UCmTBtCkkEgz4-wo035PEH2A" }, "videoId": "2oaRXlpCLxg", "textDisplay": "Thanks for the video! Love the TG16!", "textOriginal": "Thanks for the video! Love the TG16!", "canRate": true, "viewerRating": "none", "likeCount": 0, "publishedAt": "2017-09-06T17:11:25.000Z", "updatedAt": "2017-09-06T17:11:25.000Z" } }, "canReply": true, "totalReplyCount": 0, "isPublic": true } } ] } Quote Link to post Share on other sites
HelloInsomnia 1103 Posted September 7, 2017 Report Share Posted September 7, 2017 Okay try this, this will just pull the next page token out and create the next url, you will have to do the first get request then pull out the next page token (you can just copy/paste the nextPageToken var) and then the next get request is that url just add in the video id. comment("Do first get request here, I will simulate this by putting in a variable named #response below with the response you got back") set(#response,"\{ \"kind\": \"youtube#commentThreadListResponse\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/F1bK0Wo4VY6v25XGUBJOlK7iXd0\\\"\", \"nextPageToken\": \"QURTSl9pMy04NTVUSE1IOEtVWS0yQlFxUlpPdFFUY2RtLWZ3c0Z5cUkyalNESG93R1ZSM21kTmM2T1p4S3puUG5Dd1FPR3lIQ1pJSWR1VEJCQWItWjJnelowdExZWTBJemlVMm5YOW1TeEJhTWNDQTZWTnFiSzBSYjFQQ3RaNm9TaEI5\", \"pageInfo\": \{ \"totalResults\": 10, \"resultsPerPage\": 10 \}, \"items\": [ \{ \"kind\": \"youtube#commentThread\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/WDLSlWYZUCyHIGbMzZC8iFqyQJY\\\"\", \"id\": \"z23njbupps3dhrqclacdp4334f01aexntr424c3zcwhw03c010c\", \"snippet\": \{ \"videoId\": \"2oaRXlpCLxg\", \"topLevelComment\": \{ \"kind\": \"youtube#comment\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/eHDvqXf-Vx_zGns0T1SWB9U2mSY\\\"\", \"id\": \"z23njbupps3dhrqclacdp4334f01aexntr424c3zcwhw03c010c\", \"snippet\": \{ \"authorDisplayName\": \"Iris Zizka\", \"authorProfileImageUrl\": \"https://yt3.ggpht.com/-HAT-VqFq77s/AAAAAAAAAAI/AAAAAAAAAAA/rbMFz_rdWYc/s28-c-k-no-mo-rj-c0xffffff/photo.jpg\", \"authorChannelUrl\": \"http://www.youtube.com/channel/UCATO8LCrlHiHJf-9BZdutwA\", \"authorChannelId\": \{ \"value\": \"UCATO8LCrlHiHJf-9BZdutwA\" \}, \"videoId\": \"2oaRXlpCLxg\", \"textDisplay\": \"you guys are such faggots, always going on and and on about how the games you are looking at are rare and expensive. yes, we know, you have them, yes thank you for driving up the fucking price of tg16 games single fucking handedly. you assholes.\", \"textOriginal\": \"you guys are such faggots, always going on and and on about how the games you are looking at are rare and expensive. yes, we know, you have them, yes thank you for driving up the fucking price of tg16 games single fucking handedly. you assholes.\", \"canRate\": true, \"viewerRating\": \"none\", \"likeCount\": 36, \"publishedAt\": \"2017-09-04T04:22:44.000Z\", \"updatedAt\": \"2017-09-04T04:22:44.000Z\" \} \}, \"canReply\": true, \"totalReplyCount\": 80, \"isPublic\": true \} \}, \{ \"kind\": \"youtube#commentThread\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/Ts5V6MSu9Q_Z8adK62BbCfs0Gtw\\\"\", \"id\": \"z22rcjvhjvi4i3c3uacdp43b2eef0bac1ymeifkqsxpw03c010c\", \"snippet\": \{ \"videoId\": \"2oaRXlpCLxg\", \"topLevelComment\": \{ \"kind\": \"youtube#comment\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/Hdu_ZVz3ydh_K_xT0PcfQnQ0XQQ\\\"\", \"id\": \"z22rcjvhjvi4i3c3uacdp43b2eef0bac1ymeifkqsxpw03c010c\", \"snippet\": \{ \"authorDisplayName\": \"Overclocked User\", \"authorProfileImageUrl\": \"https://yt3.ggpht.com/-4QKqO74kAr0/AAAAAAAAAAI/AAAAAAAAAAA/ZC0oCmb8HMA/s28-c-k-no-mo-rj-c0xffffff/photo.jpg\", \"authorChannelUrl\": \"http://www.youtube.com/channel/UCHLIuWB-7d1EmiiOjBYuYsw\", \"authorChannelId\": \{ \"value\": \"UCHLIuWB-7d1EmiiOjBYuYsw\" \}, \"videoId\": \"2oaRXlpCLxg\", \"textDisplay\": \"Underated console at best.\\nBetween bad marketing hardly slim to none proper pr or marketing.\\nAnd a slim game library this unappreciated console will always hold a soft spot in my life and heart!\", \"textOriginal\": \"Underated console at best.\\nBetween bad marketing hardly slim to none proper pr or marketing.\\nAnd a slim game library this unappreciated console will always hold a soft spot in my life and heart!\", \"canRate\": true, \"viewerRating\": \"none\", \"likeCount\": 0, \"publishedAt\": \"2017-09-07T01:31:05.000Z\", \"updatedAt\": \"2017-09-07T01:31:05.000Z\" \} \}, \"canReply\": true, \"totalReplyCount\": 0, \"isPublic\": true \} \}, \{ \"kind\": \"youtube#commentThread\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/1pGt_3UI5yVpSPNj7sg5IGEORx8\\\"\", \"id\": \"z22icjiqcnizi3fji04t1aokgkvmobga2kz3tdt2wovmrk0h00410\", \"snippet\": \{ \"videoId\": \"2oaRXlpCLxg\", \"topLevelComment\": \{ \"kind\": \"youtube#comment\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/WazSwSvznlf2s4gumP6qUDi-07A\\\"\", \"id\": \"z22icjiqcnizi3fji04t1aokgkvmobga2kz3tdt2wovmrk0h00410\", \"snippet\": \{ \"authorDisplayName\": \"Atomicskull\", \"authorProfileImageUrl\": \"https://yt3.ggpht.com/-86NESkzhCEs/AAAAAAAAAAI/AAAAAAAAAAA/mC8tsHx8hDA/s28-c-k-no-mo-rj-c0xffffff/photo.jpg\", \"authorChannelUrl\": \"http://www.youtube.com/channel/UCRtZGOQQ4Ir5wHTJ2g7nmZg\", \"authorChannelId\": \{ \"value\": \"UCRtZGOQQ4Ir5wHTJ2g7nmZg\" \}, \"videoId\": \"2oaRXlpCLxg\", \"textDisplay\": \"The arcade version of Bloody Wolf ran on hardware that was basically an upgraded PC Engine. They used a new pallet chip (the PC Engine graphics hardware is actually two chips one that handles tiles and sprites and one that handles the color pallets and generates the video signal) and added a YM2203 FM sound chip and a MSM5205 speech synthesis chip as well as a second HuC6280 CPU (an improved 6502 with some extra instructions and additional features) and twice as much VRAM.\", \"textOriginal\": \"The arcade version of Bloody Wolf ran on hardware that was basically an upgraded PC Engine. They used a new pallet chip (the PC Engine graphics hardware is actually two chips one that handles tiles and sprites and one that handles the color pallets and generates the video signal) and added a YM2203 FM sound chip and a MSM5205 speech synthesis chip as well as a second HuC6280 CPU (an improved 6502 with some extra instructions and additional features) and twice as much VRAM.\", \"canRate\": true, \"viewerRating\": \"none\", \"likeCount\": 0, \"publishedAt\": \"2017-09-07T00:36:58.000Z\", \"updatedAt\": \"2017-09-07T00:38:41.000Z\" \} \}, \"canReply\": true, \"totalReplyCount\": 0, \"isPublic\": true \} \}, \{ \"kind\": \"youtube#commentThread\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/LpL2xEWSMyB7lZJfoG_hDX9fGd4\\\"\", \"id\": \"z22bytl55s3uvzidfacdp434e3gzob45o24fwx0kwxpw03c010c\", \"snippet\": \{ \"videoId\": \"2oaRXlpCLxg\", \"topLevelComment\": \{ \"kind\": \"youtube#comment\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/WKIS7eMxuezKSwjIbx1VOcBXUP8\\\"\", \"id\": \"z22bytl55s3uvzidfacdp434e3gzob45o24fwx0kwxpw03c010c\", \"snippet\": \{ \"authorDisplayName\": \"Percy Saucedo\", \"authorProfileImageUrl\": \"https://yt3.ggpht.com/-zqa3TE49wiQ/AAAAAAAAAAI/AAAAAAAAAAA/ht9jONVxlsc/s28-c-k-no-mo-rj-c0xffffff/photo.jpg\", \"authorChannelUrl\": \"http://www.youtube.com/channel/UCk1dMsP_fEMKTvmaWfC0OvQ\", \"authorChannelId\": \{ \"value\": \"UCk1dMsP_fEMKTvmaWfC0OvQ\" \}, \"videoId\": \"2oaRXlpCLxg\", \"textDisplay\": \"BEST SYSTEM EVER!!!!! Back in the day I had the Turbo Duo system. It was far superior than its rivals but to bad it did not gain much popularity in the US. I still have my TurboGrafx 16 though and allot of HueCard games!\", \"textOriginal\": \"BEST SYSTEM EVER!!!!! Back in the day I had the Turbo Duo system. It was far superior than its rivals but to bad it did not gain much popularity in the US. I still have my TurboGrafx 16 though and allot of HueCard games!\", \"canRate\": true, \"viewerRating\": \"none\", \"likeCount\": 0, \"publishedAt\": \"2017-09-07T00:11:16.000Z\", \"updatedAt\": \"2017-09-07T00:11:16.000Z\" \} \}, \"canReply\": true, \"totalReplyCount\": 0, \"isPublic\": true \} \}, \{ \"kind\": \"youtube#commentThread\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/cRZsPPbiRVyZrl1dOuGbfXjCvNg\\\"\", \"id\": \"z23cfbuohsrjuhimracdp4322ziyu1oudsnahqgaoixw03c010c\", \"snippet\": \{ \"videoId\": \"2oaRXlpCLxg\", \"topLevelComment\": \{ \"kind\": \"youtube#comment\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/gUDPqOsQAqVHdBwknrPAfuNGy2Q\\\"\", \"id\": \"z23cfbuohsrjuhimracdp4322ziyu1oudsnahqgaoixw03c010c\", \"snippet\": \{ \"authorDisplayName\": \"miniwar monger\", \"authorProfileImageUrl\": \"https://yt3.ggpht.com/-dbvMsBEcKv0/AAAAAAAAAAI/AAAAAAAAAAA/3niAoNM5Z-c/s28-c-k-no-mo-rj-c0xffffff/photo.jpg\", \"authorChannelUrl\": \"http://www.youtube.com/channel/UCS-Fsb5EcD-UGtqSTK9ynNA\", \"authorChannelId\": \{ \"value\": \"UCS-Fsb5EcD-UGtqSTK9ynNA\" \}, \"videoId\": \"2oaRXlpCLxg\", \"textDisplay\": \"As always love you guys.\", \"textOriginal\": \"As always love you guys.\", \"canRate\": true, \"viewerRating\": \"none\", \"likeCount\": 0, \"publishedAt\": \"2017-09-06T23:37:52.000Z\", \"updatedAt\": \"2017-09-06T23:37:52.000Z\" \} \}, \"canReply\": true, \"totalReplyCount\": 0, \"isPublic\": true \} \}, \{ \"kind\": \"youtube#commentThread\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/FTIJ0dnjPJFkGUXTyJGTgno298w\\\"\", \"id\": \"z22rgh24hvz5h5zzo04t1aokgwixnvzqewsdcbc1a1g0rk0h00410\", \"snippet\": \{ \"videoId\": \"2oaRXlpCLxg\", \"topLevelComment\": \{ \"kind\": \"youtube#comment\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/7tf7Jv3BG0xXdphhJUMutp2pvJ8\\\"\", \"id\": \"z22rgh24hvz5h5zzo04t1aokgwixnvzqewsdcbc1a1g0rk0h00410\", \"snippet\": \{ \"authorDisplayName\": \"DARK DRAKONIS\", \"authorProfileImageUrl\": \"https://yt3.ggpht.com/-KvB_sBzQ6OE/AAAAAAAAAAI/AAAAAAAAAAA/CKbOhe18c2k/s28-c-k-no-mo-rj-c0xffffff/photo.jpg\", \"authorChannelUrl\": \"http://www.youtube.com/channel/UCuGsrnRSeDtz1S6kec-68Mg\", \"authorChannelId\": \{ \"value\": \"UCuGsrnRSeDtz1S6kec-68Mg\" \}, \"videoId\": \"2oaRXlpCLxg\", \"textDisplay\": \"I always found it funny to see SonSon in MVC2 \", \"textOriginal\": \"I always found it funny to see SonSon in MVC2 \", \"canRate\": true, \"viewerRating\": \"none\", \"likeCount\": 0, \"publishedAt\": \"2017-09-06T22:31:25.000Z\", \"updatedAt\": \"2017-09-06T22:31:25.000Z\" \} \}, \"canReply\": true, \"totalReplyCount\": 0, \"isPublic\": true \} \}, \{ \"kind\": \"youtube#commentThread\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/c1wkqEYcS6uomc7dkWjuLMe5puE\\\"\", \"id\": \"z23dih1jiwzsc50yc04t1aokglivexmoae0skmibptxork0h00410\", \"snippet\": \{ \"videoId\": \"2oaRXlpCLxg\", \"topLevelComment\": \{ \"kind\": \"youtube#comment\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/0uVg_ZV8GwWP45uCD0R0hvmUC58\\\"\", \"id\": \"z23dih1jiwzsc50yc04t1aokglivexmoae0skmibptxork0h00410\", \"snippet\": \{ \"authorDisplayName\": \"Wayne Winquist\", \"authorProfileImageUrl\": \"https://yt3.ggpht.com/-MnZj5WpH1uE/AAAAAAAAAAI/AAAAAAAAAAA/Iey8PAedHf4/s28-c-k-no-mo-rj-c0xffffff/photo.jpg\", \"authorChannelUrl\": \"http://www.youtube.com/channel/UCdbn8oKLw6BJUzXw6Ia1Wgg\", \"authorChannelId\": \{ \"value\": \"UCdbn8oKLw6BJUzXw6Ia1Wgg\" \}, \"videoId\": \"2oaRXlpCLxg\", \"textDisplay\": \"Wait - did Dave just beat Joe with a foam banana? LOL\", \"textOriginal\": \"Wait - did Dave just beat Joe with a foam banana? LOL\", \"canRate\": true, \"viewerRating\": \"none\", \"likeCount\": 0, \"publishedAt\": \"2017-09-06T21:04:35.000Z\", \"updatedAt\": \"2017-09-06T21:04:35.000Z\" \} \}, \"canReply\": true, \"totalReplyCount\": 0, \"isPublic\": true \} \}, \{ \"kind\": \"youtube#commentThread\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/gu1VCayX1KjrN58lUFEGd72XKSQ\\\"\", \"id\": \"z23qijvbhmjufneil04t1aokgyul3xwphlc1qih5talork0h00410\", \"snippet\": \{ \"videoId\": \"2oaRXlpCLxg\", \"topLevelComment\": \{ \"kind\": \"youtube#comment\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/hBUx7O_clbhKoALf0XjCfSjsUtk\\\"\", \"id\": \"z23qijvbhmjufneil04t1aokgyul3xwphlc1qih5talork0h00410\", \"snippet\": \{ \"authorDisplayName\": \"Peter James Ledwith\", \"authorProfileImageUrl\": \"https://yt3.ggpht.com/-EESzKH6-9YE/AAAAAAAAAAI/AAAAAAAAAAA/h6qF8SRY8G0/s28-c-k-no-mo-rj-c0xffffff/photo.jpg\", \"authorChannelUrl\": \"http://www.youtube.com/channel/UCCe_oLT4aWDHSbXLg2bjQLQ\", \"authorChannelId\": \{ \"value\": \"UCCe_oLT4aWDHSbXLg2bjQLQ\" \}, \"videoId\": \"2oaRXlpCLxg\", \"textDisplay\": \"great video. il never get to play most of these obscure games except through emulation. good to see them here.\", \"textOriginal\": \"great video. il never get to play most of these obscure games except through emulation. good to see them here.\", \"canRate\": true, \"viewerRating\": \"none\", \"likeCount\": 0, \"publishedAt\": \"2017-09-06T20:47:58.000Z\", \"updatedAt\": \"2017-09-06T20:47:58.000Z\" \} \}, \"canReply\": true, \"totalReplyCount\": 0, \"isPublic\": true \} \}, \{ \"kind\": \"youtube#commentThread\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/VHh4T8duquv9AIjwkK4rc-L5LMY\\\"\", \"id\": \"z23tx55jzrm1x5xfsacdp430jpmd14x4w034ookoqk1w03c010c\", \"snippet\": \{ \"videoId\": \"2oaRXlpCLxg\", \"topLevelComment\": \{ \"kind\": \"youtube#comment\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/yIcAoqRoWtuJYI_xzJv7BILERII\\\"\", \"id\": \"z23tx55jzrm1x5xfsacdp430jpmd14x4w034ookoqk1w03c010c\", \"snippet\": \{ \"authorDisplayName\": \"Oldtech81\", \"authorProfileImageUrl\": \"https://yt3.ggpht.com/--7M2uTAdM1o/AAAAAAAAAAI/AAAAAAAAAAA/I0bbdSnpvTQ/s28-c-k-no-mo-rj-c0xffffff/photo.jpg\", \"authorChannelUrl\": \"http://www.youtube.com/channel/UC-xa8NmzxdPINtuGWr2CvZQ\", \"authorChannelId\": \{ \"value\": \"UC-xa8NmzxdPINtuGWr2CvZQ\" \}, \"videoId\": \"2oaRXlpCLxg\", \"textDisplay\": \"Awesome video as always!\", \"textOriginal\": \"Awesome video as always!\", \"canRate\": true, \"viewerRating\": \"none\", \"likeCount\": 0, \"publishedAt\": \"2017-09-06T17:47:56.000Z\", \"updatedAt\": \"2017-09-06T17:47:56.000Z\" \} \}, \"canReply\": true, \"totalReplyCount\": 0, \"isPublic\": true \} \}, \{ \"kind\": \"youtube#commentThread\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/lJvxOXA6sBvNLZIlHSARGJJsI6Y\\\"\", \"id\": \"z232yx5xsnaisboai04t1aokgx1pectc15esuqfsxagark0h00410\", \"snippet\": \{ \"videoId\": \"2oaRXlpCLxg\", \"topLevelComment\": \{ \"kind\": \"youtube#comment\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/I7bydK-D1G_fmqTVBLYladoQS7I\\\"\", \"id\": \"z232yx5xsnaisboai04t1aokgx1pectc15esuqfsxagark0h00410\", \"snippet\": \{ \"authorDisplayName\": \"Jd Combs\", \"authorProfileImageUrl\": \"https://yt3.ggpht.com/-ijh6HOPCYuc/AAAAAAAAAAI/AAAAAAAAAAA/-dpPhBHmi2Y/s28-c-k-no-mo-rj-c0xffffff/photo.jpg\", \"authorChannelUrl\": \"http://www.youtube.com/channel/UCmTBtCkkEgz4-wo035PEH2A\", \"authorChannelId\": \{ \"value\": \"UCmTBtCkkEgz4-wo035PEH2A\" \}, \"videoId\": \"2oaRXlpCLxg\", \"textDisplay\": \"Thanks for the video! Love the TG16!\", \"textOriginal\": \"Thanks for the video! Love the TG16!\", \"canRate\": true, \"viewerRating\": \"none\", \"likeCount\": 0, \"publishedAt\": \"2017-09-06T17:11:25.000Z\", \"updatedAt\": \"2017-09-06T17:11:25.000Z\" \} \}, \"canReply\": true, \"totalReplyCount\": 0, \"isPublic\": true \} \} ] \}","Global") set(#nextPageToken,$plugin function("JSONpath.dll", "$JSONpath parser", #response, "$.nextPageToken"),"Global") comment("This is the next get request url") set(#nextGetRequest,"https://www.googleapis.com/youtube/v3/commentThreads?key={#apiKey}&textFormat=plainText&part=snippet&videoId=kffacxfA7G4&pageToken={#nextPageToken}&maxResults=100","Global") 2 Quote Link to post Share on other sites
steelersfan 38 Posted September 8, 2017 Author Report Share Posted September 8, 2017 So, let me see if I understand the code:First I do a get request by setting a variable to the get request, and plugging in the API data string needed. It then spits out the data plus the next page token. (I would scrape the comment data somehow to preserve it at this point as well, correct?) Next I set that nextpagetoken to another variable via set command and the scraping with json parser. Then I use the API string again, but with the next page token in the API string, which will bring me the data needed to repeat the process until all comments were reached?I would just have to figure out how to properly loop this at this point then? Is my assessment mostly correct? Thank you for this help Helloinsomnia! The only problem I have though, is I can't get the plugin! I had subscribed to his list long ago and got a few plugins, but they are all gone now and he never replies to mails anymore. I think aymen is long gone. That was one of the few free plugins that I didn't grab when I had the chance, ignorantly thinking I would never need it! Anyone have access to him or a copy of it? Quote Link to post Share on other sites
Code Docta (Nick C.) 638 Posted September 8, 2017 Report Share Posted September 8, 2017 Here is a function to parse JSON objects. set(#json string,"\{ \"kind\": \"youtube#commentThreadListResponse\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/F1bK0Wo4VY6v25XGUBJOlK7iXd0\\\"\", \"nextPageToken\": \"QURTSl9pMy04NTVUSE1IOEtVWS0yQlFxUlpPdFFUY2RtLWZ3c0Z5cUkyalNESG93R1ZSM21kTmM2T1p4S3puUG5Dd1FPR3lIQ1pJSWR1VEJCQWItWjJnelowdExZWTBJemlVMm5YOW1TeEJhTWNDQTZWTnFiSzBSYjFQQ3RaNm9TaEI5\", \"pageInfo\": \{ \"totalResults\": 10, \"resultsPerPage\": 10 \}, \"items\": [ \{ \"kind\": \"youtube#commentThread\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/WDLSlWYZUCyHIGbMzZC8iFqyQJY\\\"\", \"id\": \"z23njbupps3dhrqclacdp4334f01aexntr424c3zcwhw03c010c\", \"snippet\": \{ \"videoId\": \"2oaRXlpCLxg\", \"topLevelComment\": \{ \"kind\": \"youtube#comment\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/eHDvqXf-Vx_zGns0T1SWB9U2mSY\\\"\", \"id\": \"z23njbupps3dhrqclacdp4334f01aexntr424c3zcwhw03c010c\", \"snippet\": \{ \"authorDisplayName\": \"Iris Zizka\", \"authorProfileImageUrl\": \"https://yt3.ggpht.com/-HAT-VqFq77s/AAAAAAAAAAI/AAAAAAAAAAA/rbMFz_rdWYc/s28-c-k-no-mo-rj-c0xffffff/photo.jpg\", \"authorChannelUrl\": \"http://www.youtube.com/channel/UCATO8LCrlHiHJf-9BZdutwA\", \"authorChannelId\": \{ \"value\": \"UCATO8LCrlHiHJf-9BZdutwA\" \}, \"videoId\": \"2oaRXlpCLxg\", \"textDisplay\": \"you guys are such faggots, always going on and and on about how the games you are looking at are rare and expensive. yes, we know, you have them, yes thank you for driving up the fucking price of tg16 games single fucking handedly. you assholes.\", \"textOriginal\": \"you guys are such faggots, always going on and and on about how the games you are looking at are rare and expensive. yes, we know, you have them, yes thank you for driving up the fucking price of tg16 games single fucking handedly. you assholes.\", \"canRate\": true, \"viewerRating\": \"none\", \"likeCount\": 36, \"publishedAt\": \"2017-09-04T04:22:44.000Z\", \"updatedAt\": \"2017-09-04T04:22:44.000Z\" \} \}, \"canReply\": true, \"totalReplyCount\": 80, \"isPublic\": true \} \}, \{ \"kind\": \"youtube#commentThread\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/Ts5V6MSu9Q_Z8adK62BbCfs0Gtw\\\"\", \"id\": \"z22rcjvhjvi4i3c3uacdp43b2eef0bac1ymeifkqsxpw03c010c\", \"snippet\": \{ \"videoId\": \"2oaRXlpCLxg\", \"topLevelComment\": \{ \"kind\": \"youtube#comment\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/Hdu_ZVz3ydh_K_xT0PcfQnQ0XQQ\\\"\", \"id\": \"z22rcjvhjvi4i3c3uacdp43b2eef0bac1ymeifkqsxpw03c010c\", \"snippet\": \{ \"authorDisplayName\": \"Overclocked User\", \"authorProfileImageUrl\": \"https://yt3.ggpht.com/-4QKqO74kAr0/AAAAAAAAAAI/AAAAAAAAAAA/ZC0oCmb8HMA/s28-c-k-no-mo-rj-c0xffffff/photo.jpg\", \"authorChannelUrl\": \"http://www.youtube.com/channel/UCHLIuWB-7d1EmiiOjBYuYsw\", \"authorChannelId\": \{ \"value\": \"UCHLIuWB-7d1EmiiOjBYuYsw\" \}, \"videoId\": \"2oaRXlpCLxg\", \"textDisplay\": \"Underated console at best.\\nBetween bad marketing hardly slim to none proper pr or marketing.\\nAnd a slim game library this unappreciated console will always hold a soft spot in my life and heart!\", \"textOriginal\": \"Underated console at best.\\nBetween bad marketing hardly slim to none proper pr or marketing.\\nAnd a slim game library this unappreciated console will always hold a soft spot in my life and heart!\", \"canRate\": true, \"viewerRating\": \"none\", \"likeCount\": 0, \"publishedAt\": \"2017-09-07T01:31:05.000Z\", \"updatedAt\": \"2017-09-07T01:31:05.000Z\" \} \}, \"canReply\": true, \"totalReplyCount\": 0, \"isPublic\": true \} \}, \{ \"kind\": \"youtube#commentThread\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/1pGt_3UI5yVpSPNj7sg5IGEORx8\\\"\", \"id\": \"z22icjiqcnizi3fji04t1aokgkvmobga2kz3tdt2wovmrk0h00410\", \"snippet\": \{ \"videoId\": \"2oaRXlpCLxg\", \"topLevelComment\": \{ \"kind\": \"youtube#comment\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/WazSwSvznlf2s4gumP6qUDi-07A\\\"\", \"id\": \"z22icjiqcnizi3fji04t1aokgkvmobga2kz3tdt2wovmrk0h00410\", \"snippet\": \{ \"authorDisplayName\": \"Atomicskull\", \"authorProfileImageUrl\": \"https://yt3.ggpht.com/-86NESkzhCEs/AAAAAAAAAAI/AAAAAAAAAAA/mC8tsHx8hDA/s28-c-k-no-mo-rj-c0xffffff/photo.jpg\", \"authorChannelUrl\": \"http://www.youtube.com/channel/UCRtZGOQQ4Ir5wHTJ2g7nmZg\", \"authorChannelId\": \{ \"value\": \"UCRtZGOQQ4Ir5wHTJ2g7nmZg\" \}, \"videoId\": \"2oaRXlpCLxg\", \"textDisplay\": \"The arcade version of Bloody Wolf ran on hardware that was basically an upgraded PC Engine. They used a new pallet chip (the PC Engine graphics hardware is actually two chips one that handles tiles and sprites and one that handles the color pallets and generates the video signal) and added a YM2203 FM sound chip and a MSM5205 speech synthesis chip as well as a second HuC6280 CPU (an improved 6502 with some extra instructions and additional features) and twice as much VRAM.\", \"textOriginal\": \"The arcade version of Bloody Wolf ran on hardware that was basically an upgraded PC Engine. They used a new pallet chip (the PC Engine graphics hardware is actually two chips one that handles tiles and sprites and one that handles the color pallets and generates the video signal) and added a YM2203 FM sound chip and a MSM5205 speech synthesis chip as well as a second HuC6280 CPU (an improved 6502 with some extra instructions and additional features) and twice as much VRAM.\", \"canRate\": true, \"viewerRating\": \"none\", \"likeCount\": 0, \"publishedAt\": \"2017-09-07T00:36:58.000Z\", \"updatedAt\": \"2017-09-07T00:38:41.000Z\" \} \}, \"canReply\": true, \"totalReplyCount\": 0, \"isPublic\": true \} \}, \{ \"kind\": \"youtube#commentThread\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/LpL2xEWSMyB7lZJfoG_hDX9fGd4\\\"\", \"id\": \"z22bytl55s3uvzidfacdp434e3gzob45o24fwx0kwxpw03c010c\", \"snippet\": \{ \"videoId\": \"2oaRXlpCLxg\", \"topLevelComment\": \{ \"kind\": \"youtube#comment\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/WKIS7eMxuezKSwjIbx1VOcBXUP8\\\"\", \"id\": \"z22bytl55s3uvzidfacdp434e3gzob45o24fwx0kwxpw03c010c\", \"snippet\": \{ \"authorDisplayName\": \"Percy Saucedo\", \"authorProfileImageUrl\": \"https://yt3.ggpht.com/-zqa3TE49wiQ/AAAAAAAAAAI/AAAAAAAAAAA/ht9jONVxlsc/s28-c-k-no-mo-rj-c0xffffff/photo.jpg\", \"authorChannelUrl\": \"http://www.youtube.com/channel/UCk1dMsP_fEMKTvmaWfC0OvQ\", \"authorChannelId\": \{ \"value\": \"UCk1dMsP_fEMKTvmaWfC0OvQ\" \}, \"videoId\": \"2oaRXlpCLxg\", \"textDisplay\": \"BEST SYSTEM EVER!!!!! Back in the day I had the Turbo Duo system. It was far superior than its rivals but to bad it did not gain much popularity in the US. I still have my TurboGrafx 16 though and allot of HueCard games!\", \"textOriginal\": \"BEST SYSTEM EVER!!!!! Back in the day I had the Turbo Duo system. It was far superior than its rivals but to bad it did not gain much popularity in the US. I still have my TurboGrafx 16 though and allot of HueCard games!\", \"canRate\": true, \"viewerRating\": \"none\", \"likeCount\": 0, \"publishedAt\": \"2017-09-07T00:11:16.000Z\", \"updatedAt\": \"2017-09-07T00:11:16.000Z\" \} \}, \"canReply\": true, \"totalReplyCount\": 0, \"isPublic\": true \} \}, \{ \"kind\": \"youtube#commentThread\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/cRZsPPbiRVyZrl1dOuGbfXjCvNg\\\"\", \"id\": \"z23cfbuohsrjuhimracdp4322ziyu1oudsnahqgaoixw03c010c\", \"snippet\": \{ \"videoId\": \"2oaRXlpCLxg\", \"topLevelComment\": \{ \"kind\": \"youtube#comment\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/gUDPqOsQAqVHdBwknrPAfuNGy2Q\\\"\", \"id\": \"z23cfbuohsrjuhimracdp4322ziyu1oudsnahqgaoixw03c010c\", \"snippet\": \{ \"authorDisplayName\": \"miniwar monger\", \"authorProfileImageUrl\": \"https://yt3.ggpht.com/-dbvMsBEcKv0/AAAAAAAAAAI/AAAAAAAAAAA/3niAoNM5Z-c/s28-c-k-no-mo-rj-c0xffffff/photo.jpg\", \"authorChannelUrl\": \"http://www.youtube.com/channel/UCS-Fsb5EcD-UGtqSTK9ynNA\", \"authorChannelId\": \{ \"value\": \"UCS-Fsb5EcD-UGtqSTK9ynNA\" \}, \"videoId\": \"2oaRXlpCLxg\", \"textDisplay\": \"As always love you guys.\", \"textOriginal\": \"As always love you guys.\", \"canRate\": true, \"viewerRating\": \"none\", \"likeCount\": 0, \"publishedAt\": \"2017-09-06T23:37:52.000Z\", \"updatedAt\": \"2017-09-06T23:37:52.000Z\" \} \}, \"canReply\": true, \"totalReplyCount\": 0, \"isPublic\": true \} \}, \{ \"kind\": \"youtube#commentThread\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/FTIJ0dnjPJFkGUXTyJGTgno298w\\\"\", \"id\": \"z22rgh24hvz5h5zzo04t1aokgwixnvzqewsdcbc1a1g0rk0h00410\", \"snippet\": \{ \"videoId\": \"2oaRXlpCLxg\", \"topLevelComment\": \{ \"kind\": \"youtube#comment\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/7tf7Jv3BG0xXdphhJUMutp2pvJ8\\\"\", \"id\": \"z22rgh24hvz5h5zzo04t1aokgwixnvzqewsdcbc1a1g0rk0h00410\", \"snippet\": \{ \"authorDisplayName\": \"DARK DRAKONIS\", \"authorProfileImageUrl\": \"https://yt3.ggpht.com/-KvB_sBzQ6OE/AAAAAAAAAAI/AAAAAAAAAAA/CKbOhe18c2k/s28-c-k-no-mo-rj-c0xffffff/photo.jpg\", \"authorChannelUrl\": \"http://www.youtube.com/channel/UCuGsrnRSeDtz1S6kec-68Mg\", \"authorChannelId\": \{ \"value\": \"UCuGsrnRSeDtz1S6kec-68Mg\" \}, \"videoId\": \"2oaRXlpCLxg\", \"textDisplay\": \"I always found it funny to see SonSon in MVC2 \", \"textOriginal\": \"I always found it funny to see SonSon in MVC2 \", \"canRate\": true, \"viewerRating\": \"none\", \"likeCount\": 0, \"publishedAt\": \"2017-09-06T22:31:25.000Z\", \"updatedAt\": \"2017-09-06T22:31:25.000Z\" \} \}, \"canReply\": true, \"totalReplyCount\": 0, \"isPublic\": true \} \}, \{ \"kind\": \"youtube#commentThread\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/c1wkqEYcS6uomc7dkWjuLMe5puE\\\"\", \"id\": \"z23dih1jiwzsc50yc04t1aokglivexmoae0skmibptxork0h00410\", \"snippet\": \{ \"videoId\": \"2oaRXlpCLxg\", \"topLevelComment\": \{ \"kind\": \"youtube#comment\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/0uVg_ZV8GwWP45uCD0R0hvmUC58\\\"\", \"id\": \"z23dih1jiwzsc50yc04t1aokglivexmoae0skmibptxork0h00410\", \"snippet\": \{ \"authorDisplayName\": \"Wayne Winquist\", \"authorProfileImageUrl\": \"https://yt3.ggpht.com/-MnZj5WpH1uE/AAAAAAAAAAI/AAAAAAAAAAA/Iey8PAedHf4/s28-c-k-no-mo-rj-c0xffffff/photo.jpg\", \"authorChannelUrl\": \"http://www.youtube.com/channel/UCdbn8oKLw6BJUzXw6Ia1Wgg\", \"authorChannelId\": \{ \"value\": \"UCdbn8oKLw6BJUzXw6Ia1Wgg\" \}, \"videoId\": \"2oaRXlpCLxg\", \"textDisplay\": \"Wait - did Dave just beat Joe with a foam banana? LOL\", \"textOriginal\": \"Wait - did Dave just beat Joe with a foam banana? LOL\", \"canRate\": true, \"viewerRating\": \"none\", \"likeCount\": 0, \"publishedAt\": \"2017-09-06T21:04:35.000Z\", \"updatedAt\": \"2017-09-06T21:04:35.000Z\" \} \}, \"canReply\": true, \"totalReplyCount\": 0, \"isPublic\": true \} \}, \{ \"kind\": \"youtube#commentThread\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/gu1VCayX1KjrN58lUFEGd72XKSQ\\\"\", \"id\": \"z23qijvbhmjufneil04t1aokgyul3xwphlc1qih5talork0h00410\", \"snippet\": \{ \"videoId\": \"2oaRXlpCLxg\", \"topLevelComment\": \{ \"kind\": \"youtube#comment\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/hBUx7O_clbhKoALf0XjCfSjsUtk\\\"\", \"id\": \"z23qijvbhmjufneil04t1aokgyul3xwphlc1qih5talork0h00410\", \"snippet\": \{ \"authorDisplayName\": \"Peter James Ledwith\", \"authorProfileImageUrl\": \"https://yt3.ggpht.com/-EESzKH6-9YE/AAAAAAAAAAI/AAAAAAAAAAA/h6qF8SRY8G0/s28-c-k-no-mo-rj-c0xffffff/photo.jpg\", \"authorChannelUrl\": \"http://www.youtube.com/channel/UCCe_oLT4aWDHSbXLg2bjQLQ\", \"authorChannelId\": \{ \"value\": \"UCCe_oLT4aWDHSbXLg2bjQLQ\" \}, \"videoId\": \"2oaRXlpCLxg\", \"textDisplay\": \"great video. il never get to play most of these obscure games except through emulation. good to see them here.\", \"textOriginal\": \"great video. il never get to play most of these obscure games except through emulation. good to see them here.\", \"canRate\": true, \"viewerRating\": \"none\", \"likeCount\": 0, \"publishedAt\": \"2017-09-06T20:47:58.000Z\", \"updatedAt\": \"2017-09-06T20:47:58.000Z\" \} \}, \"canReply\": true, \"totalReplyCount\": 0, \"isPublic\": true \} \}, \{ \"kind\": \"youtube#commentThread\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/VHh4T8duquv9AIjwkK4rc-L5LMY\\\"\", \"id\": \"z23tx55jzrm1x5xfsacdp430jpmd14x4w034ookoqk1w03c010c\", \"snippet\": \{ \"videoId\": \"2oaRXlpCLxg\", \"topLevelComment\": \{ \"kind\": \"youtube#comment\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/yIcAoqRoWtuJYI_xzJv7BILERII\\\"\", \"id\": \"z23tx55jzrm1x5xfsacdp430jpmd14x4w034ookoqk1w03c010c\", \"snippet\": \{ \"authorDisplayName\": \"Oldtech81\", \"authorProfileImageUrl\": \"https://yt3.ggpht.com/--7M2uTAdM1o/AAAAAAAAAAI/AAAAAAAAAAA/I0bbdSnpvTQ/s28-c-k-no-mo-rj-c0xffffff/photo.jpg\", \"authorChannelUrl\": \"http://www.youtube.com/channel/UC-xa8NmzxdPINtuGWr2CvZQ\", \"authorChannelId\": \{ \"value\": \"UC-xa8NmzxdPINtuGWr2CvZQ\" \}, \"videoId\": \"2oaRXlpCLxg\", \"textDisplay\": \"Awesome video as always!\", \"textOriginal\": \"Awesome video as always!\", \"canRate\": true, \"viewerRating\": \"none\", \"likeCount\": 0, \"publishedAt\": \"2017-09-06T17:47:56.000Z\", \"updatedAt\": \"2017-09-06T17:47:56.000Z\" \} \}, \"canReply\": true, \"totalReplyCount\": 0, \"isPublic\": true \} \}, \{ \"kind\": \"youtube#commentThread\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/lJvxOXA6sBvNLZIlHSARGJJsI6Y\\\"\", \"id\": \"z232yx5xsnaisboai04t1aokgx1pectc15esuqfsxagark0h00410\", \"snippet\": \{ \"videoId\": \"2oaRXlpCLxg\", \"topLevelComment\": \{ \"kind\": \"youtube#comment\", \"etag\": \"\\\"m2yskBQFythfE4irbTIeOgYYfBU/I7bydK-D1G_fmqTVBLYladoQS7I\\\"\", \"id\": \"z232yx5xsnaisboai04t1aokgx1pectc15esuqfsxagark0h00410\", \"snippet\": \{ \"authorDisplayName\": \"Jd Combs\", \"authorProfileImageUrl\": \"https://yt3.ggpht.com/-ijh6HOPCYuc/AAAAAAAAAAI/AAAAAAAAAAA/-dpPhBHmi2Y/s28-c-k-no-mo-rj-c0xffffff/photo.jpg\", \"authorChannelUrl\": \"http://www.youtube.com/channel/UCmTBtCkkEgz4-wo035PEH2A\", \"authorChannelId\": \{ \"value\": \"UCmTBtCkkEgz4-wo035PEH2A\" \}, \"videoId\": \"2oaRXlpCLxg\", \"textDisplay\": \"Thanks for the video! Love the TG16!\", \"textOriginal\": \"Thanks for the video! Love the TG16!\", \"canRate\": true, \"viewerRating\": \"none\", \"likeCount\": 0, \"publishedAt\": \"2017-09-06T17:11:25.000Z\", \"updatedAt\": \"2017-09-06T17:11:25.000Z\" \} \}, \"canReply\": true, \"totalReplyCount\": 0, \"isPublic\": true \} \} ] \}","Global") alert($find regular expression(#json string,"(?<= \"nextPageToken\": \").*(?=\",)")) comment("NEXT PAGE TOKEN") alert($JSON object parser(#json string, "nextPageToken")) comment("ORIGINAL COMMENT") alert($JSON object parser(#json string, "textOriginal")) define $JSON object parser(#JSON STRING, #OBJECT) { return($find regular expression(#json string,"(?<= \"{#OBJECT}\": \").*(?=\",)")) } If no page token then I assume there are no more pages.Regards,Nick 1 Quote Link to post Share on other sites
steelersfan 38 Posted September 14, 2017 Author Report Share Posted September 14, 2017 I'm confused as to how to implement this Nick. Is the long wall of code within the "#json string" variable the code that returns after the youtube API is called? Meaning, I can place that variable within the "#json string" variable, correct? Or is that actual code that I will need to write into the "#json string" variable? Quote Link to post Share on other sites
HelloInsomnia 1103 Posted September 14, 2017 Report Share Posted September 14, 2017 Your first request is the same and you get the JSON response. Then you parse out the nextPageToken and then you use the value of that to add the parameter &pageToken= to the next request. So the part that you add is: &pageToken={nextPageToken} In the example above it would look like this: &pageToken=QURTSl9pMy04NTVUSE1IOEtVWS0yQlFxUlpPdFFUY2RtLWZ3c0Z5cUkyalNESG93R1ZSM21kTmM2T1p4S3puUG5Dd1FPR3lIQ1pJSWR1VEJCQWItWjJnelowdExZWTBJemlVMm5YOW1TeEJhTWNDQTZWTnFiSzBSYjFQQ3RaNm9TaEI5 But of course you would use a variable to store it. You can parse it out with this regex: (?<=nextPageToken\":\s\").*?(?=\")From then on you would keep looking for the next page token and keep using the newest token in that parameter so on each request after the first you would look for the nextPageToken and use it in pageToken in the request. My guess is that the last page won't have a nextPageToken and so that it how you know it's the last page. Hope that clears it up. 1 Quote Link to post Share on other sites
steelersfan 38 Posted October 2, 2017 Author Report Share Posted October 2, 2017 So I have had a bit of trouble absorbing all of this, but then I got a hold of the json plugin and started again from scratch. This time around I think I have a firm understanding of the process. So here I will try to explain what I have found, and hopefully it will help me to finally get this done, as well as help others who struggle with this kind of thought process as well. ui text box("YouTube API Key:",#ytapikey) ui text box("Video Id:",#videoid) divider divider set(#getcomments,$plugin function("HTTP post.dll", "$http get", "https://www.googleapis.com/youtube/v3/commentThreads?key={#ytapikey}&textFormat=plainText&part=snippet&videoId={#videoid}&maxResults=100", $plugin function("HTTP post.dll", "$http useragent string", "Random"), "https://www.youtube.com", "", 30),"Global") set(#nextpagetoken,$plugin function("JSONpath.dll", "$JSONpath parser", #getcomments, "$.nextPageToken"),"Global") set(#nextgetrequest,"https://www.googleapis.com/youtube/v3/commentThreads?key={#ytapikey}&textFormat=plainText&part=snippet&videoId={#videoid}&pageToken={#nextpagetoken}&maxResults=100","Global") So, what I did was:First I tested the beginning process by simply using the API call on a video, and looked at the output. I looked for the next page token inside of the Ubot Debugger. Then, I followed Helloinsomnia's post #10 here in this thread, and used it to get the next page token. (I still don't know how JSON parser works, but I assume that the get API request outputs the page data and the heading of the actual next page token is a JSON syntax?) Meaning that I guess "nextPageToken": "some long string here" can be parsed by the JSON plugin, and it reads whatever is in that particular string - after "nextPageToken": "reads this string here". So writing "$.nextPageToken" in the JSONpath Expression section of the plugin command will return that data and set it to my variable. (okay simple enough now, I guess) Then setting the variable "nextgetrequest" to the new value of the google YouTube API with the next token variable that is found by the step 2 process will allow me to loop from this point on.Now I am just working out the looping logic in my head (does thinking programmatically ever get easier guys?!), which should make my head hurt a bit more, and then I will have it all together! So, I hope my assessment is correct, and that it helps others who are as slow on the uptake as I am! Thanks a lot guys!!! Much appreciated! 1 Quote Link to post Share on other sites
steelersfan 38 Posted October 2, 2017 Author Report Share Posted October 2, 2017 Hmm, I am having trouble figuring out how to loop this properly. I was thinking that I needed "loop while", and I thought to use "exists" that must scan a page, but am unsure how to make it work. How can I look into the file and validate the existence of the next get request, to set the loop while? Quote Link to post Share on other sites
steelersfan 38 Posted October 2, 2017 Author Report Share Posted October 2, 2017 So I figured it out! It feels kind of hack'y, but I think it is a fine way to loop the process... I just used "loop while" with "contains" from the comments got, using the word nextPageToken So what I did was to get the comments page renewed within the loop. All is working as intended now! The code is a lot cleaner than doing it the ubot way, a lot less complicated, and a lot less prone to mistakes or rewrites needed. (Since it totally uses the YouTube API) Quote Link to post Share on other sites
HelloInsomnia 1103 Posted October 2, 2017 Report Share Posted October 2, 2017 It sounds like you get it from your last couple of posts, using a loop while and looking for that nextPageToken also sounds right. 1 Quote Link to post Share on other sites
steelersfan 38 Posted October 6, 2017 Author Report Share Posted October 6, 2017 It sounds like you get it from your last couple of posts, using a loop while and looking for that nextPageToken also sounds right.Indeed, thanks to your help and code docta! Thanks guys! Oh, and I look forward to your tutorials launch! If you need any help with ideas or lesson plans, let me know. I would be glad to help, and you know I have a history of asking challenging and instructive questions! Questions that I'm sure everyone will benefit from at the intermediate level. 1 Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.