Abs* 12 Posted April 7, 2011 Report Share Posted April 7, 2011 HI Guys - just wondering if anyone could help with matching a url when checking the search engines - so it wouldnt cause an issue if it was a .com or .co.uk etc thanks abs Quote Link to post Share on other sites
LoWrIdErTJ - BotGuru 904 Posted April 7, 2011 Report Share Posted April 7, 2011 have this one a go.. ^(http(?:s)?\:\/\/[a-zA-Z0-9\-]+(?:\.[a-zA-Z0-9\-]+)*\.[a-zA-Z]{2,6}(?:\/?|(?:\/[\w\-]+)*)(?:\/?|\/\w+\.[a-zA-Z]{2,4}(?:\?[\w]+\=[\w\-]+)?)?(?:\&[\w]+\=[\w\-]+)*)$ Quote Link to post Share on other sites
Abs* 12 Posted April 8, 2011 Author Report Share Posted April 8, 2011 have this one a go.. ^(http(?:s)?\:\/\/[a-zA-Z0-9\-]+(?:\.[a-zA-Z0-9\-]+)*\.[a-zA-Z]{2,6}(?:\/?|(?:\/[\w\-]+)*)(?:\/?|\/\w+\.[a-zA-Z]{2,4}(?:\?[\w]+\=[\w\-]+)?)?(?:\&[\w]+\=[\w\-]+)*)$ Hi - Thanks alot - will give it a shot - Just wondering how on earth your writing such complex regex I dont have any experience with regex and find it a nightmare to try and understand I understand parts of it like it could have a-z lowercase or A-Z uppercase 0-9 http or could have the s But for the life of me im lost with all this complex code man A few pro tips would go a long way thanks abs Quote Link to post Share on other sites
UBotBuddy 331 Posted April 8, 2011 Report Share Posted April 8, 2011 Just look in the pinned thread in this sub-forum and you will see a link to Rubular. http://www.rubular.com/r/nidQpOizwC Aww heck there it is Quote Link to post Share on other sites
JohnB 255 Posted April 8, 2011 Report Share Posted April 8, 2011 abs, it can be very intimidating at first, but if you play with it for a few days it becomes second nature. It is actually nowhere near as difficult as it looks. Let us know if we can help out! John Quote Link to post Share on other sites
UBotBuddy 331 Posted April 8, 2011 Report Share Posted April 8, 2011 Oh I hate regex. That's why I have friends like John. I'll whine about it and then he'll do it for me. LOL Or make some fatal attempt at it and then someone with provide a Winner of a regex formula. Mu ah ha ha {evil laugh} Quote Link to post Share on other sites
Pete 121 Posted April 8, 2011 Report Share Posted April 8, 2011 I can see these regex posts will keep popping up again and again would it not help user’s if the expression generated where to be kept in a library? Quote Link to post Share on other sites
LoWrIdErTJ - BotGuru 904 Posted April 8, 2011 Report Share Posted April 8, 2011 I've used php for web development for around 14 years now. Doing so I have had to use a lot of regex on things ranging from sign up to filtering. So it kinda comes second nature once you get the hang of it. There are programs out there though to make it a lot easier for building regex code as well thoughHere is a tester for regex codehttp://regexlib.com/RETester.aspxEverything you want to know about regex (cheat sheet)http://regexlib.com/CheatSheet.aspx Regex Resourceshttp://regexlib.com/Resources.aspx Quote Link to post Share on other sites
Abs* 12 Posted April 11, 2011 Author Report Share Posted April 11, 2011 Hay thanks alot guys Yes will certainly look into all the regex guides - Im sure it will only help so better to get stuck in asap rather then waiting Thanks for all your help guys Will be trying to work on the regex provided today Abs Quote Link to post Share on other sites
Abs* 12 Posted April 11, 2011 Author Report Share Posted April 11, 2011 hi guys Ok I guess I aint got a clue how to use these regex codes with evals Ive looked at the tutorials and also the tutorial on ubotjunkie by frank The issue that im having is checking to see if the url is on the page - So I have a UI field for the keyword and also the url field - The url field may not be the full url - Just want to see if the actual url ranks regardless of it being a home page or a inner page SO am a little confused where to place the reg expression to evaluate if it is noted Quote Link to post Share on other sites
Abs* 12 Posted April 11, 2011 Author Report Share Posted April 11, 2011 How about a $50 donation to anyone that can PM me a working google rank checker using the regex noted. It should do a google search based on a keyword entered in the UI field - and search for a url noted regardless of it being a home or inner page. It should then bring back the google position it ranks in - Based on Organic searches - You can do this based on a while command so that it indefinatly loops till found- I think there is a free google rank checker .ubotfile which already does this but not based on regex and it doesnt work with .co.uk or similar domains I will be working with the .ubot file to enable it to work with proxies and from lists maybe even sockets. If you want to include this in the bot then feel free to do so -Would help me loads - If you feel $50 is too less then let me know how much you feel is enough and we can work somehting out . Would love to do it myself but just aint got the hours in the day at the moment- Thanks Abs Quote Link to post Share on other sites
LoWrIdErTJ - BotGuru 904 Posted April 11, 2011 Report Share Posted April 11, 2011 Here is the specific Regex for looking up all sub domains, http, https, ftp, imap etc for UK domains example: .co.uk or .me.uk or .plc.uk or .gov.uk etc etc etc Uk based URLs ^(ht|f)tp((?<=http)s)?://((?<=http://)www|(?<=https://)www|(?<=ftp://)ftp)\.(([a-z][0-9])|([0-9][a-z])|([a-z0-9][a-z0-9\-]{1,2}[a-z0-9])|([a-z0-9][a-z0-9\-](([a-z0-9\-][a-z0-9])|([a-z0-9][a-z0-9\-]))[a-z0-9\-]*[a-z0-9]))\.(co|me|org|ltd|plc|net|sch|ac|mod|nhs|police|gov)\.uk$ US based URLs ^(http(?:s)?\:\/\/[a-zA-Z0-9\-]+(?:\.[a-zA-Z0-9\-]+)*\.[a-zA-Z]{2,6}(?:\/?|(?:\/[\w\-]+)*)(?:\/?|\/\w+\.[a-zA-Z]{2,4}(?:\?[\w]+\=[\w\-]+)?)?(?:\&[\w]+\=[\w\-]+)*)$ Using both in a scan will produce all possible Urls for your searchI will try to put together a sample bot for you. TJ Quote Link to post Share on other sites
Abs* 12 Posted April 11, 2011 Author Report Share Posted April 11, 2011 Oh thanks man Quote Link to post Share on other sites
LoWrIdErTJ - BotGuru 904 Posted April 11, 2011 Report Share Posted April 11, 2011 Let me ask this. Are you searching google for a specific url (UI inserted) and trying to find what page its on? I guess let me know a bit more functionality and the information that is going to be supplied to the bot for it to search. Quote Link to post Share on other sites
LoWrIdErTJ - BotGuru 904 Posted April 11, 2011 Report Share Posted April 11, 2011 I have made the attached for ya. Have a look and see if this works for you. Set domain your looking for on pagesSet keyword to search bySet number of pages back in Engine you want to look for itSet which engine you want to use However regex was not a matter in this case. I was able to set it up without having to use regex for the search.Let me know if im doing something incorrectly for what your wanting from the engines? Bot will return where you are found on what pages of the rankings for the keyword searchedif no results within the pages back it will tell you so at the end of the bot. TJfind site in serps.ubot Quote Link to post Share on other sites
LoWrIdErTJ - BotGuru 904 Posted April 12, 2011 Report Share Posted April 12, 2011 Did the above work for you? Quote Link to post Share on other sites
Abs* 12 Posted April 12, 2011 Author Report Share Posted April 12, 2011 Did the above work for you? HI - Thanks alot I have been speaking to John and it looks like I got myself very confused - Regex cant be used to in the way I wanted it to - No need for regex ive looked at the google code you have noted - Its kind of cool the way your using the regex to scrape the result stats - Thanks Quote Link to post Share on other sites
UBotBuddy 331 Posted April 12, 2011 Report Share Posted April 12, 2011 I only use regex as a last resort. Quote Link to post Share on other sites
LoWrIdErTJ - BotGuru 904 Posted April 12, 2011 Report Share Posted April 12, 2011 yeah regex is unforgiving but when absolutely needed to be very specific on the data your trying to find it is good to use. TJ Quote Link to post Share on other sites
James J 3 Posted April 14, 2011 Report Share Posted April 14, 2011 @Abs I sent you a pm. James Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.