billywizz 0 Posted December 26, 2009 Report Share Posted December 26, 2009 Is there a way to remove a item from a list.say I have a list of urls that I have scraped,then once that url is visited ,either remove it or better still mark it as visted,and save until the bot is used again. Quote Link to post Share on other sites
sardwyn 0 Posted December 26, 2009 Report Share Posted December 26, 2009 save to file ---> clear list? ---> load as necessary? Quote Link to post Share on other sites
tooltrainer 12 Posted December 27, 2009 Report Share Posted December 27, 2009 You could save the current item to a new list and write that list to a file, replacing it with the updated list each time the node is executed. That would get you an ever-growing list of "visited" URLs. As far as removing them from the current list once visited, that's trickier I think. You could loop through the full list and write the values to a new one as long as they don't equal the current URL. I'm doing something sortof similar with my shopping.com scraper and it's working out. Jonathan Quote Link to post Share on other sites
billywizz 0 Posted December 27, 2009 Author Report Share Posted December 27, 2009 I think what I'm looking for is some sort of $save list postion ,so when I restart the bot it carries on from where it left off. Quote Link to post Share on other sites
tooltrainer 12 Posted December 27, 2009 Report Share Posted December 27, 2009 Ah... in that case I would write the list position to a file and read it back in again later. UBot doesn't really have any other way to store persistent data like that as far as I know... Jonathan Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.