Geek Answers Handbook

Efficient way to delete lines from a file in python (and keep the same file name)?

My program stores a log for the user. If the log is ever larger than the set amount, I want to delete the first 20% of the lines.

From similar questions, I saw suggestions for reading in the old file and write out all the lines that I want to save in the new file. However, my files may be too large to be read continuously, and using this method would not allow me to keep the same file name.

Is it possible to delete lines from a file without reading in my old file?

+4
source share
3 answers

The general way to achieve this for log files is to "rotate" - when the log files get older or reach a certain size, you rename it and start writing a new one. If you use the logging module, there is even a preconfigured - RotatingFileHandler , which does this automatically.

As for your question: you can crop only from the back, and not from the very beginning. An approximate solution would be to search () for up to 20% of the file, first find "\ n" and copy it, but it will be slow and subject to race conditions. Go with logging and RotatingFileHandler.

+8
source

As others have said, the traditional way to solve this problem is to save 5 different files instead of one big one. When you need to delete 20%, just delete the oldest file and rename the rest.

As convenient in text files, you can also view the database. It is designed to delete any piece of data at any time.

+1
source 
if size > MAX_SIZE: f = open(your_file, 'r') lines = f.readlines() f.close() f = open(your_file, 'w') f.write('\n'.join(lines[len(lines)/5:])) f.close()

It could be like that. Although, as the first two said, it is much better to use several files or even a database if you can. This code has not been verified.

0
source

More articles:


All Articles