steelersfan

Hmm, so I think I understand now! Using the element editor to find only specific things is the best and proper use of it, and using regex to that end is the proper way. An improper or counterproductive way to use regex, is to use it to find multiple sources of the same type on the page, which is better served by using wildcards, or just scraping a specific area and then using further methods on that area you isolated. Is that a correct assessment of what I should be using the advanced element editor for?

Thanks a lot Code Doc! Indeed, I have found that xpath is a lot better and have Dan's tutorials, the plugin and xpath builder! They have been a great help for me in this regard thus far! The main thing I get from your words, which is very helpful indeed, is that I should grab the text content that I need from the website/s first, and then process it later. I have learned that the hard way with my last project involving YouTube comment scraping as well! The second most important lesson your words taught me (and correct me if I'm wrong), is that the element editor should not really be relied

So I did a mix of things here to achieve the best results. Here is a breakdown for those who are interested: Firstly, using the API is limited to 100 results. I don't want that, and my method returns well over 500 emails from specific targeted videos.Secondly, the YouTube API plugin from pash is far too complicated and annoying to set up for this anyway, while HTTP Post plugin is a lot easier. (though again, the API is limited anyway, so it is a waste of time to use!)I couldn't find a good Regex to grab only the comments that had the "@" mail sign in them, so I used the following xpath:

I am very confused with how to use most plugin functions. It seems that any function I wish to use needs me to log in, okay so I created an oauth 2.0 client ID. However, all that does is prompt me to log into the application I created through it's oauth consent screen. Once I log in there, manually, I can then run any functions from the plugin. But what about using a normal API key? How do I use the plugin functions with a regular unrestricted API key? Is that even possible? Also, the "youtube connect" function, how does that work? It asks for an "authorization code", where do I get that?

Oh I don't dispute the API is a better choice. I just don't understand the code you shared, or where to put it. (the large example you shared) I understand this code: https://www.googleapis.com/youtube/v3/commentThreads?key=ENTER_KEY_HERE&textFormat=plainText&part=snippet&videoId=kffacxfA7G4&maxResults=50 but don't know how to call it in ubot? (just navigate?) I am trying to figure out how to do this with the youtube plugin from pash, but with no luck so far. The documentation for that plugin is non-existent sadly.

So, to my current understanding the function "scrape attribute" is used to get/find code that is "inside" of an attribute, but not to actually pull anything specific out of it? And "find regular expression" is used to actually pull the desired text from some scraped code/text? (when you already know the regular expression needed) Using the element editor, it seems that the regular expression selector is used to target matches, but only under specific conditions? This is the part where I am getting lost. The same regex that works for me using "find regular expression", will not work in the

Thank you helloinsomnia. You know I really am horrible at regex, I even tried using your tool to generate something that would work, but I don't know what I am doing wrong. I will try to study this and see what I can learn from it. Did you use your tool to make this or just your own knowledge? So essentially I have to clean the html out before I use the regex?

Hmm, I am actually more confused by this than anything, lol! I have pash's api plugin, so I could probably do something with that, but I don't understand what I am looking at here, could you possibly explain what is going on here?

I have tried to come up with a good regex to pick up emails from youtube comments, but to no avail. I ended up using xpath to get all of the emails I was testing with, but it was a simple xpath that also picked up the text of each comment. I am hoping that I can find a way to filter just the email addresses from the scrape, but regex is really a pain for me, xpath seems better, but that was the best I could do... Here is an example of what I did: <div class="comment-renderer-text-content">hey bud could u send over the spred sheet please too somefool@<a href="http://gmail.com/" clas

Sorry if this has been asked before, are we able to use extensions and plugins on Firefox and Chrome? And if so, how? I just started using exbrowser and the tutorials on YouTube, but all I have done so far is launched the browser and navigated to google. When I checked the browser that it brought up, it was not using my own configuration (I already have Firefox as my main browser, with many plugins active). EDIT: Never mind, with a little digging I answered my own question! Here it is for others who want to know the answer: https://youtu.be/xf4RUzGaf-A?list=PLun8zsNfYLfIWrYEDRk14AwU2UdASbexp

Very nice indeed! Thanks! EDIT: Damn, it isn't working anymore! It goes to 404 page and I never get the mail...

Firstly, why are updates so few and very far in between? I subscribe to updates and pay quarterly, but am lucky to see an update 1 time in a quarter. Secondly, when is there going to be an actual feature update or major update to the software? Lastly, why are there still old bugs and issues still around? It is as if next to nothing is being worked on from the tracker. There is a severe lack of features to keep up with current trends in online automation and keeping up with websites (such as java advancements, motion UI, advanced conditional logic, etc), and a lack of convenience features

PM sent.

Posting to bookmark this, the solution is quite helpful to me! I have been wondering how to achieve this for a long time now!

Ahh! Now I understand that second part better! It calls itself, brilliant! That is something that I never even thought of or conceived could be done with a define. A nice jump in my skill can be gained from such an example! I will now approach it better and continue my breakdown in the post above, both for my own analytical understanding, and to help others who may think as I do. Thank you for explaining it! Once I can understand enough myself to reiterate it in my own words, I will be quite happy! Indeed, you are and have been a great help to me in the past. I don't like to just be a "taker

Wow! So, it worked, and quickly! For the purposes of what I am trying to do, it is a much better solution than my own initial channel scraping method (Which was bulky, convoluted, and did not work on large channels!). I have to edit a bit of this to get it to work within my script, but that is trivial and easy. This replaces my clunky solution with a far better one. I have added slightly edited code below that gets the href as opposed to the titles, for anyone who may need that also. usersVideos("https://www.youtube.com/user/YaleCourses/videos") define usersVideos(#videoPage) { set(#get,$

404 on "Aymans Free JSON Parser" too, so I'm SoL there.

I am thoroughly confused and overwhelmed at this point, lol! Deliter you are great! But I am still quite far away from your skill and understanding. I need some time to try and understand this all, but I know it will help me in trying to finish this bot. Thank you!

You know, I was hoping for a "get" solution! Thanks so much Deliter!

I am having a hard time resolving this error in Ubot that is caused by loading more videos in YouTube. I am trying to get all of the videos of any channel, and my script works on nearly every channel I have tried it on. It merely starts by going to a channel, and hitting "load more" until there is nothing more to load. The problem is that on some channels with a HUGE amount of videos (for example Yale Courses), it returns the following error: "Error converting value True to type 'System.Collections.Generic.List`1 [system.String]'. My code to keep loading more as long as the button exists

In the example you told me to view, the search has a term and there is no channel ID. I don't want to search YouTube for videos based on a search term, I want to search a channel for all of its videos. The command itself says to leave the search field blank to get all of the videos of a channel. Yet if I do that, it returns the error I posted above. Furthermore, your example script does not work if run either. Can you please look into it and see what is the matter?

For getting info on any channel - What is the format of entering the channel Id, if it is not a character string like your example? The full youtube channel URL, or the trailing text after youtube.com/ or? Also, I keep getting the following error: Execute: Object reference not set to an instance of an object. I don't know why it won't run. Do I need to do anything else, besides just trying to run the command alone?

Is there a way to get all of a channel's video URL's with this? I have looked, but couldn't find a way. Right now I am doing it with just ubot, then using this to download all videos in a channel, but scraping a channel for URL's with ubot is not ideal. Try scraping all of Yale courses for example, and you run into a major error after 40 or so "load more" clicks.

Thanks Helloinsomnia, as usual it is you, and just a handful of others who are helpful here. Hence why I buy yours and their products, and my obvious frustration with so small a support group that the official forums offer. I am indeed admittedly a tad bitter over such a lack of real community here, but that's another story. You, and those few other guys (you guys know who you are!!!), are a great help, and you have no idea how much I truly appreciate it. My purchasing of your fine products is the least I could do to show that. Indeed, that was an overly detailed post, but it was also helpfu

Anyone know how to implement the status bar in this software? I tried setting it up, but it never works. I even tried it with the code from this thread with no success. What am I doing wrong? Or is this feature just broken? This guy sure lives up to the name "lazy", that's for sure...