flamerz 2 Posted February 13, 2020 Report Share Posted February 13, 2020 (edited) Hello, I'm just creating a blog post about reading some "big" csv file. Lets say, 100.000 lines, 2 columns, name and age. I wish to load the data and iterate/sum the age variable, just for documenting it. set(#total,0,"Global") set(#row,0,"Global") create table from file("C:\\Users\\Usuario\\Desktop\\names.csv",&dataTable) loop($table total rows(&dataTable)) { set(#total,$add(#total,$table cell(&dataTable,#row,1)),"Global") increment(#row) } This is my UBot code. My issue is, with 100k lines this goes a bit slow, maybe its not good idea to load it on a table. this is my mockup post:https://translate.google.com/translate?hl=en&sl=es&tl=en&u=https%3A%2F%2Fscraper.es%2Fabrir-archivos-csv%2F any guidance? my post is not a competition for speed, just a guide to read/iterate a csv file in a good way. I wish to have the data loaded and do not use a driver or a database query to do a SUM. Just a simple loop. thanks in advance. PD: names file attached. names.csv Edited February 13, 2020 by flamerz Quote Link to post Share on other sites
Bot-Factory 602 Posted February 13, 2020 Report Share Posted February 13, 2020 Ubot will have lot of memory issues with that. And performance is awful. Take a look at this:http://network.ubotstudio.com/forum/index.php/topic/16308-free-plugin-large-data/ Dan Quote Link to post Share on other sites
ATuringtest 14 Posted February 14, 2020 Report Share Posted February 14, 2020 Ubot will have lot of memory issues with that. And performance is awful. Take a look at this:http://network.ubotstudio.com/forum/index.php/topic/16308-free-plugin-large-data/ Dan That plugin link is dead Quote Link to post Share on other sites
flamerz 2 Posted February 14, 2020 Author Report Share Posted February 14, 2020 not sure why the plugin its not working on my ubot works: create table from file("H:\\FileCenter\\test.csv",&test)does nothing: plugin command("Bigtable.dll", "Large Table From file", "H:\\FileCenter\\test.csv", &test2) Quote Link to post Share on other sites
flamerz 2 Posted February 14, 2020 Author Report Share Posted February 14, 2020 That plugin link is dead yes.. seems gone. I still have the version 1.7.0.0, not sure that one was the latest. Quote Link to post Share on other sites
ATuringtest 14 Posted February 14, 2020 Report Share Posted February 14, 2020 Personally I moved to SQLlite far easier in the long run. There's a plugin for it and it integrates really well with C#. Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.