Geek Answers Handbook

Recursive grep using python

I am new to python and am trying to learn. I am trying to implement a simple recursive grep using python for processing, and this is what I have come up with so far.

p = subprocess.Popen('find . -name [ch]', shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) for line in p.stdout.readlines(): q = subprocess.Popen('grep searchstring %s', line, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) print q.stdout.readlines()

Can someone tell me how to fix this to do what it should?

+4
source share
3 answers

You must use the os.walk function to view your files. Use the string or regex methods to filter the results. Check out http://docs.python.org/library/os.html for information on how to use os.walk.

 import os import re def findfiles(path, regex): regObj = re.compile(regex) res = [] for root, dirs, fnames in os.walk(path): for fname in fnames: if regObj.match(fname): res.append(os.path.join(root, fname)) return res print findfiles('.', r'my?(reg|ex)')

Now for the grep part you can iterate over the file using the open function

 def grep(filepath, regex): regObj = re.compile(regex) res = [] with open(filepath) as f: for line in f: if regObj.match(line): res.append(line) return res

If you want to get line numbers, you can look in the enumerate function.

edited to add grep function

+7
source 
 p = subprocess.Popen('find . -name [ch]', shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) for line in p.stdout.readlines(): q = subprocess.Popen('grep searchstring %s', line, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) print q.stdout.readlines()
  • The indentation in line 2 will exclude, for should be aligned with p above
  • 'grep searchstring %s', line will not replace the string, you need to replace , with %

With these changes and real search values, it works in my OS X box. Final script:

 import subprocess p = subprocess.Popen('find . -name *.py', shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) for line in p.stdout.readlines(): print line q = subprocess.Popen('grep import %s' % line, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) print q.stdout.readlines()
0
source

Maybe an example can help you, the find . -print | grep "python" find . -print | grep "python" find . -print | grep "python" equivalent to this:

 import subprocess pc1 = subprocess.Popen('find . -print', stdout=subprocess.PIPE, shell=True) pc2 = subprocess.Popen('grep "python"', stdin=pc1.stdout, shell=True, stdout=subprocess.PIPE) print pc2.communicate()[0]
0
source

More articles:


All Articles