READ Just One Line of a Text File, Do Something and then DELETE that Line from the File

[Chainsaw 9]

I've got some huge text files of data, too large to simply import as a table. So instead I'd like to read just one line from the file, do something with the data and then delete that one line. Rinse and repeat.

 

Reading the entire file is out too, so ideally I'd like to read just the first or last line of the file.

 

Any suggestions welcome.

[Gogetta 263]

There is no way to read one line of a file unless you read the entire file. If your file is that big break it up into chunks and use the $get files function and add the file names into a list. That way you can call them one by one for use in your bot..

[wilriv21 16]

With huge files - any reason to not use MySQL or SQLite?  UBot supports both.

[Chainsaw 9]

There is no way to read one line of a file unless you read the entire file. If your file is that big break it up into chunks and use the $get files function and add the file names into a list. That way you can call them one by one for use in your bot..

.

Thanks, that is the solution I'm probably going to go with for now.

[Chainsaw 9]

With huge files - any reason to not use MySQL or SQLite?  UBot supports both.

 

Tried adding the files to a SQLite database but at the moment there seems to be an issue with connecting to SQLite databases. Perhaps that will be working in 5.01? MySQL might of course work but last time I tried MySQL, I was getting a connection error as well and so haven't yet tested MySQL.

[a2mateit 395]

With all the free and paid plugins out there nowadays is there anyway to do this yet?

 

I need to be able to read the first line of a file, then delete that line.

 

I've been searching around for a while now to try and find this out...

 

TIA

[HelloInsomnia 1103]

Well you can read the first line of the file using this: http://network.ubotstudio.com/forum/index.php/topic/15579-free-plugin-advanced-file-clipboardhandle-big-lists-in-text-files-and-more/

 

Not sure about deleting it though unless you add it to a list and remove that list item.

[a2mateit 395]

Well you can read the first line of the file using this: http://network.ubotstudio.com/forum/index.php/topic/15579-free-plugin-advanced-file-clipboardhandle-big-lists-in-text-files-and-more/

 

Not sure about deleting it though unless you add it to a list and remove that list item.

 

Thanks Nick.

 

Currently what I'm doing is loading it to a list, then removing the first list item, then resaving.

 

But when the list has 50k lines, this really slows things down.

[HelloInsomnia 1103]

Thanks Nick.

 

Currently what I'm doing is loading it to a list, then removing the first list item, then resaving.

 

But when the list has 50k lines, this really slows things down.

 

Why do you need to delete the line after reading it? In the end you would end up with a blank text file. Instead you can just save the last line number read if the bot doesn't do them all at once or in case it crashes or something.

[a2mateit 395]

Why do you need to delete the line after reading it? In the end you would end up with a blank text file. Instead you can just save the last line number read if the bot doesn't do them all at once or in case it crashes or something.

 

That's the whole idea.

 

I'm cleaning one scrape file, and if it's successful it gets appended to a different list.

 

So the blank text file will is ok in this instance.

 

But I see what your saying about just saving the list position each time it runs and recalling it if there's a crash.

[LordPorker 7]

Kinda stumped with this too; why can't UB simply take a line (even better, would be a choice to select a random one), directly from a file, then once it's taken, delete it?

 

Saving a large file into UB must take a toll on resources, no?

 

Anyway, so I've saved the file into UB using: add list to list with list from file. Now that I have my (virtual) list, I need to take a url and then delete it. Can anyone explain the process?

[Code Docta (Nick C.) 638]

Hey LP,

the large data plugin or advance file plugin will handle this and you wont have to delete just cycle thru with an index. Start at zero(set #index 0, outside loop) and then inside the loop increment the #index.

 

or what you asked specifically

use remove list item 0 everytime

then instead of $next list item

 

use $list item and 0 everytime

 

so in affect you are deleting the first item and the next rolls to position 0 which will be the next.

 

CD

[LordPorker 7]

I kinda follow you, but the issue is that I'm taking a random url from an actual file. If I delete position 0, then the random url is still in the text file.

 

So far at the moment, I've managed to grab a random line from a list using this:

set(#listing,$random list item($list from file("C:random.txt")),"Global")

The next issue is deleting said line from the list, ideally using a variable search for and to delete the line.

[abbas786 78]

Hey Tony, 

 

Try this logic

 

Set #TotalRecords = List Total
Set #RandomRecord = $Rand(1, #TotalRecords )

 

Now Select ur item from list

set(#listing,$random list item(#RandomRecord),"Global")

and than Delete it

 

Hope it helps

 

 

 

 

[bangali_beta 11]

Chainsaw,

 

Can you not dump the file data onto a list ($list from file) and then grab the $List Item: 0 and put it into a 2nd list and then use the "Remove From List" command to remove $List Item: 0 from the 1st list and so on ?

[LordPorker 7]

Hey Tony, 

 

Try this logic

 

Set #TotalRecords = List Total

Set #RandomRecord = $Rand(1, #TotalRecords )

 

Now Select ur item from list

set(#listing,$random list item(#RandomRecord),"Global")

 

and than Delete it

 

Hope it helps

 

Thanks, but my issue is working directly with a text file (not UB's virtual list). I need to delete an item from an actual .txt file.

 

Tried it with remove from file, but no luck there (as this command is only meant for UB's own list). Why can't we work directly with a txt. file? It's a joke, as getting an item from a list, and then deleting it directly, is a basic function in programming, lol. Oh well.

[bangali_beta 11]

Thanks, but my issue is working directly with a text file (not UB's virtual list). I need to delete an item from an actual .txt file.

 

Tried it with remove from file, but no luck there (as this command is only meant for UB's own list). Why can't we work directly with a txt. file? It's a joke, as getting an item from a list, and then deleting it directly, is a basic function in programming, lol. Oh well.

 

Correct. It's a joke having to deal with $Lists in the middle. I suggest you suggest a feature to deal with the file. Maybe the following functions would be good so suggest them to the Ubot feature suggestion section:

$file

$delete from file

$Add To Line Number "so" In File

$Delete From Line Number "so" In File

$Replace From Line Number "so" To "so" In File

And so on.