Add text to file using python (without replacing it)

Question

I have a file with identifiers and information, for example:

1oMZgkoaz3o 2011-12-29T01:23:00.000Z 9 503 ApolloIsMyCoPilot nUW1TomCSQg 2011-12-29T01:23:15.000Z 9 348 grea7stuff tJuLnRrAcs0 2011-12-29T01:26:20.000Z 9 123 AdelGaming tyi5g0mnPIs 2011-12-29T01:28:07.000Z 9 703 PreferredGaming 

and I want to add a flag on some lines, so if I have a dictionary

 flags = {'1oMZgkoaz3o': flag1, 'tJuLnRrAcs0': flag2} 

the result I want is

 1oMZgkoaz3o 2011-12-29T01:23:00.000Z 9 503 ApolloIsMyCoPilot flag1 nUW1TomCSQg 2011-12-29T01:23:15.000Z 9 348 grea7stuff tJuLnRrAcs0 2011-12-29T01:26:20.000Z 9 123 AdelGaming flag2 tyi5g0mnPIs 2011-12-29T01:28:07.000Z 9 703 PreferredGaming 

So I made this code

 l = True while l is True: a = f.readline() try a.split(' ')[0] in flags.iterkeys(): f.seek(-1,1) f.write(' '+str(flags[a.split(' ')[0]])+'\n') del flags[a.split(' ')[0]] except IndexError: l = False 

therefore my Python code may be bad, but the problem is that with this code I replace the text, so the file is all corrupted. How can I write without replacement? and if you have better ideas for code, you can ...

Answer 1

Here I see two problems:

Reading and writing from / to the same file

This does not work too well. It is better to read from one file and write to another (in this way, you also will not lose data if something goes wrong in your program). Example:

 input_file = open('infile.txt', 'r') output_file = open('outfile.txt', 'w') for line in input_file: line += "transformed" output_file.write(line) 

Syntax / semantic errors

You have a syntax error in the code snippet, line

 try a.split(' ')[0] in flags.iterkeys(): 

invalid (and Python should complain about it!).

Some other notes:

• in flags.iterkeys() is semantically equivalent to in flags
• Alternatively, you can use while l instead of while l is True . Even better, you can completely remove the l flag and jump out of the loop with break if an error occurs.

My attempt

 input_file = open('infile.txt', 'r') output_file = open('outfile.txt', 'w') flags = { ... } for line in input_file: parts = line.strip().split() if parts[0] in flags: line = line + ' ' + flags[parts[0]] output_file.write(line + "\n") 

If you know how to use the shell, you can make your life easier by simply using STDIN / STDOUT to input and output data. Then you save your file handling and leave the user more flexibility in how he can use your script.

Answer 2

You cannot write to a file and paste. The best approach is to read your file and write the modified contents to a new file, and then rename if necessary.

Answer 3

I would like to use stdin/stdout redirection:

 #!/usr/bin/env python3 import sys flags = {'1oMZgkoaz3o': 'flag1', 'tJuLnRrAcs0': 'flag2'} for line in sys.stdin: line = line.rstrip() k = line.split()[0] if k in flags: print(line, flags[k]) else: print(line) 

---

 $ python3 script.py <input.txt >output.txt $ cat output.txt 1oMZgkoaz3o 2011-12-29T01:23:00.000Z 9 503 ApolloIsMyCoPilot flag1 nUW1TomCSQg 2011-12-29T01:23:15.000Z 9 348 grea7stuff tJuLnRrAcs0 2011-12-29T01:26:20.000Z 9 123 AdelGaming flag2 tyi5g0mnPIs 2011-12-29T01:28:07.000Z 9 703 PreferredGaming 

Answer 4

Changing a file in place is possible using the fileinput module:

 from fileinput import FileInput f = FileInput(the_filename, inplace=True) for line in f: line = modify_line() # do whatever modifications you need to do print line # this writes the line to the file f.close() 

Answer 5

First, let the script clear a bit:

 for line in f.readlines(): line = line.strip() parts = line.split() if parts[0] in flags: f.write(line + flags[parts[0]] + "\n"); else: f.write(line + "\n"); 

It has been some time since the last time I worked with reading and writing to the same file, so that I could turn off a little.

Answer 6

The root of your problem is that you are trying to read and write to the same file. You need to create a new file.

However, there are several other things that you must first clean up ...

First, you can simplify the process by simply looping through the file object directly, instead of using a while loop. For example:.

 flags = {'1oMZgkoaz3o': 'flag1', 'tJuLnRrAcs0': 'flag2'} # The "with" statement automatically closes the file when we're done with it with open('test.txt', 'r') as infile: # If we just iterate over the open file, we're iterating over the lines in it for line in infile: line = line.strip().split() key = line[0] # I'm using "flags.get" with a default arugment here. If "key" isn't in # "flags", then an empty string will be returned. line.append(flags.get(key, '')) print ' '.join(line) 

In this example, we simply print the desired result. If the file is small, we can easily do something like this

 flags = {'1oMZgkoaz3o': 'flag1', 'tJuLnRrAcs0': 'flag2'} with open('test.txt', 'r') as infile: # Load the entire contents of the file into memory... lines = infile.readlines() with open('test.txt', 'w') as outfile: for line in lines: print line line = line.strip().split() line.append(flags.get(line[0], '')) outfile.write(' '.join(line) + '\n') 

However, if it is a large file, we may not want to read the entire copy in memory.

In this case, we want to iterate over the source file and write to another file. Then we need to rename the new file to the original file name.

If we want to be very careful, we would do something like the following:

 import os flags = {'1oMZgkoaz3o': 'flag1', 'tJuLnRrAcs0': 'flag2'} infile = open('test.txt', 'r') outfile = open('test2.txt', 'w') try: # Try to do this... for line in infile: line = line.strip().split() line.append(flags.get(line[0], '')) outfile.write(' '.join(line) + '\n') finally: # Do this no matter what... infile.close() outfile.close() # If nothing goes wrong, do this... os.remove('test.txt') os.rename('test2.txt', 'test.txt') 

The try:... finally:... essentially manually does what the with statement does for the file object. This may be a little cleaner than nesting two with statements in this particular case, but I mostly use it to show an alternative (older) syntax for this. Ideally, you probably write this piece of code similar to this:

 import os def main(): flags = {'1oMZgkoaz3o': 'flag1', 'tJuLnRrAcs0': 'flag2'} with open('test.txt', 'r') as infile: with open('test2.txt', 'w') as outfile: append_flags(infile, outfile, flags) os.remove('test.txt') os.rename('test2.txt', 'test.txt') def append_flags(infile, outfile, flags): for line in infile: line = line.strip().split() line.append(flags.get(line[0], '')) outfile.write(' '.join(line) + '\n') main() 

However, we are clearly becoming more complex the further we go with this.

In your case, the second example (reading the entire file into memory and then writing over the original file) is probably what you want.