Removing Duplicates from List ReGex

[sales 1]

Being new to uBot...EVERYTHING is a challenge! I'm sure the uBot "gurus" here on the forum don't have this problem...but I sure did.

 

I scraped a list of URLs from Google. There were lots of duplicates in my list so I wanted to use the $replace regular expression feature to clean my list. I found several regex's online, both for sorted and unsorted lists. Anyways, here is the code I used in uBot v4. It may not be pretty...but it works! I hope it will help some other poor newbie soul out. This regex deletes duplicates in lists that are either sorted or unsorted.

 

My code:

 

clear list(%dupesremoved)

add list to list(%dupesremoved, $list from file("C:\\GoogleURLs\\GoogleURLs.txt"), "Delete", "Global")

save to file("C:\\GoogleURLs\\DupesRemoved.txt", $replace regular expression(%dupesremoved, "^(.*)(?:\\r?\\n|\\r)(?=[\\s\\S]*^\\1$)", ""))

 

sales

[Legend 181]

I'm still trying to wrap my head around using simple regex in v.4... for example if I want to trim the white space from the left side or right side (or both) only of a variable what do I need to do to this code to make it work?

 

ui text box("Text to Trim", #test)

ui button("Trim Left") {

set(#test, $replace regular expression(#test, "/^\\s+/, \'\'", $nothing), "Global")

}

ui button("Trim Right") {

set(#test, $replace regular expression(#test, "/\\s+$/, \'\'", $nothing), "Global")

}

ui button("Trim All") {

set(#test, $replace regular expression(#test, "/^\\s+|\\s+$/g, \'\'", $nothing), "Global")

}

 

http://ubotstudio.com/forum/public/style_emoticons/default/blink.gif

[Bob The Builder 62]

I'm still trying to wrap my head around using simple regex in v.4... for example if I want to trim the white space from the left side or right side (or both) only of a variable what do I need to do to this code to make it work?

 

ui text box("Text to Trim", #test)

ui button("Trim Left") {

set(#test, $replace regular expression(#test, "/^\\s+/, \'\'", $nothing), "Global")

}

ui button("Trim Right") {

set(#test, $replace regular expression(#test, "/\\s+$/, \'\'", $nothing), "Global")

}

ui button("Trim All") {

set(#test, $replace regular expression(#test, "/^\\s+|\\s+$/g, \'\'", $nothing), "Global")

}

 

just use $trim

[Legend 181]

Actually, I'm trying to learn how to do it with regex because with $trim you can't specify between left and right trim and I'd like to know how to do this... as well as using regex in a set command for a hundred other regular expressions that aren't hotlisted... yet... http://ubotstudio.com/forum/public/style_emoticons/default/wink.gif

[Bob The Builder 62]

Actually, I'm trying to learn how to do it with regex because with $trim you can't specify between left and right trim and I'd like to know how to do this... as well as using regex in a set command for a hundred other regular expressions that aren't hotlisted... yet... http://ubotstudio.com/forum/public/style_emoticons/default/wink.gif

 

Here ya go, this should help:

 

set(#text, " the cow and the dog ", "Global")

set(#trimleft, $replace regular expression(#text, "^\\s+", ""), "Global")

set(#trimright, $replace regular expression(#text, "\\s+$", ""), "Global")

set(#trimboth, $replace regular expression(#text, "^\\s+|\\s+$", ""), "Global")

[Legend 181]

There we go... awesome!!

 

I feel the wonderful world of regex opening up right before my eyes...

 

Thanks Bro!

 

http://ubotstudio.com/forum/public/style_emoticons/default/smile.gif

[Bob The Builder 62]

There we go... awesome!!

 

I feel the wonderful world of regex opening up right before my eyes...

 

Thanks Bro!

 

http://ubotstudio.com/forum/public/style_emoticons/default/smile.gif

 

np, if you don't have it, I highly recommend RegEX Buddy.

Great tool, I don't use it often, but for the tricky ones it is freaking awesome.

[Guest michelsmith]

If you have a file in which all lines are sorted (alphabetically or otherwise), you can easily delete (subsequent) duplicate lines. really nice tool buddy.