Read all files in all directories

Question

I have code that works for reading in the values ​​of a single text file, but I am having difficulty reading all the files from all directories and compiling all the contents.

Here is what I have:

filename = '*' filesuffix = '*' location = os.path.join('Test', filename + "." + filesuffix) Document = filename thedictionary = {} with open(location) as f: file_contents = f.read().lower().split(' ') # split line on spaces to make a list for position, item in enumerate(file_contents): if item in thedictionary: thedictionary[item].append(position) else: thedictionary[item] = [position] wordlist = (thedictionary, Document) #print wordlist #print thedictionary 

note that I am trying to insert a pattern * for the file name, as well as a pattern for the file suffix. I get the following error:

"IOError: [Errno 2] There is no such file or directory:" Test /. ""

I'm not sure if this is even the right way to do this, but it seems that if I somehow get wildcards - this should work.

I got this example to work: Python - reading files from a directory file not found in a subdirectory (which is there)

Which is a little different - but I don’t know how to update it to read all the files. I think in this initial set of code:

 previous_dir = os.getcwd() os.chdir('testfilefolder') #add something here? for filename in os.listdir('.'): 

That I would need to add something where I have an external for loop, but I don’t know what to insert into it ..

Any thoughts?

Answer 1

Python does not support wildcards directly in file names for open() calls. You will need to use the glob module instead to load files from the same level of subdirectories or use os.walk() to execute an arbitrary directory structure.

Opening all text files in all subdirectories at the same level:

 import glob for filename in glob.iglob(os.path.join('Test', '*', '*.txt')): with open(filename) as f: # one file open, handle it, next loop will present you with a new file. 

Opening all text files in an arbitrary directory attachment:

 import os import fnmatch for dirpath, dirs, files in os.walk('Test'): for filename in fnmatch.filter(files, '*.txt'): with open(os.path.join(dirpath, filename)): # one file open, handle it, next loop will present you with a new file.