How to read a CSV file in reverse order in Python?

Question

I know how to do this for a TXT file, but now I am having problems creating it for a CSV file.

How can I read a CSV file from below in Python?

Answer 1

Just like a text file: read all of this in a list, and then go back:

import csv with open('test.csv', 'r') as textfile: for row in reversed(list(csv.reader(textfile))): print ', '.join(row) 

If you want a fantasy, you can write a lot of code that reads blocks from the end of the file and runs backward, emits a line at a time, and then passes that value to csv.reader , but this will only work with a file that can search, i.e. with disk files, but not with normal input.

---

Some of us have files that do not fit into memory, can someone come up with a solution that does not require storing the entire file in memory?

This is a little trickier. Fortunately, all csv.reader expects an iterator-like object that returns a line (line) for each call to next() . Therefore, we borrow the Darius Bacon technique presented in " The most efficient way to find the last x lines of a file in python ", to read the lines of the file backwards, without pulling out the whole file:

 import os def reversed_lines(file): "Generate the lines of file in reverse order." part = '' for block in reversed_blocks(file): for c in reversed(block): if c == '\n' and part: yield part[::-1] part = '' part += c if part: yield part[::-1] def reversed_blocks(file, blocksize=4096): "Generate blocks of file contents in reverse order." file.seek(0, os.SEEK_END) here = file.tell() while 0 < here: delta = min(blocksize, here) here -= delta file.seek(here, os.SEEK_SET) yield file.read(delta) 

and feed reversed_lines to the code to cancel the lines before they switch to csv.reader , removing the need for reversed and list :

 import csv with open('test.csv', 'r') as textfile: for row in csv.reader(reversed_lines(textfile)): print ', '.join(row) 

There is a more feasible solution for Pythonic that does not require changing the symbolic character of a block in memory (hint: just get a list of indexes where there are line itertools in the block, vice versa, and use it to cut the block) and uses the chain from itertools to glue the string clusters of consecutive blocks together, but this is left as an exercise for the reader.

---

It is worth noting that the reversei () idiom mentioned above does not work only if the columns in the CSV file do not contain newlines.

Aargh! There is always something. Fortunately, it is not so bad to fix it:

 def reversed_lines(file): "Generate the lines of file in reverse order." part = '' quoting = False for block in reversed_blocks(file): for c in reversed(block): if c == '"': quoting = not quoting elif c == '\n' and part and not quoting: yield part[::-1] part = '' part += c if part: yield part[::-1] 

Of course, you will need to change the quotation mark if your CSV dialogs do not use. "

Answer 2

Based on @ mike-desimone answer. Here's a solution that provides the same structure as the python file object, but reads in reverse order, line by line:

 import os class ReversedFile(object): def __init__(self, f, mode='r'): """ Wraps a file object with methods that make it be read in reverse line-by-line if ``f`` is a filename opens a new file object """ if mode != 'r': raise ValueError("ReversedFile only supports read mode (mode='r')") if not type(f) == file: # likely a filename f = open(f) self.file = f self.lines = self._reversed_lines() def _reversed_lines(self): "Generate the lines of file in reverse order." part = '' for block in self._reversed_blocks(): for c in reversed(block): if c == '\n' and part: yield part[::-1] part = '' part += c if part: yield part[::-1] def _reversed_blocks(self, blocksize=4096): "Generate blocks of file contents in reverse order." file = self.file file.seek(0, os.SEEK_END) here = file.tell() while 0 < here: delta = min(blocksize, here) here -= delta file.seek(here, os.SEEK_SET) yield file.read(delta) def __getattribute__(self, name): """ Allows for the underlying file attributes to come through """ try: # ReversedFile attribute return super(ReversedFile, self).__getattribute__(name) except AttributeError: # self.file attribute return getattr(self.file, name) def __iter__(self): """ Creates iterator """ return self def seek(self): raise NotImplementedError('ReversedFile does not support seek') def next(self): """ Next item in the sequence """ return self.lines.next() def read(self): """ Returns the entire contents of the file reversed line by line """ contents = '' for line in self: contents += line return contents def readline(self): """ Returns the next line from the bottom """ return self.next() def readlines(self): """ Returns all remaining lines from the bottom of the file in reverse """ return [x for x in self] 

Answer 3

Go for it. This is a simple program for changing lines from a CSV file.

 import csv BC_file = open('Master.csv', 'rb') BC_reader = csv.reader(BC_file) next(BC_reader) for row in reversed(list(BC_reader)): print row[0]