Unix tools native to windows

[Super Dave 26]

Needed a solution to removing duplicate lines in massive text files (+15mb) before processing in uBot. uBot's own ignore-list-duplicate function worked but with the file sizes I was dealing with it wouldn't scale. I've used 'sort' to do this through a terminal session but now I needed a way to use sort with ubot.

 

After a little research I found a developer's site with this description:

Here are some ports of common GNU utilities to native Win32. In this context, native means the executables do only depend on the Microsoft C-runtime (msvcrt.dll) and not an emulation layer like that provided by Cygwin tools.

 

For a little perspective, I killed a bot after an hour into processing a 600k line file... with the "sort.exe -u" utility processed the file in less than 15 seconds.

 

Enjoy: http://sourceforge.net/projects/unxutils/