Various results obtained with the Python os.walk function and the ls command

Question

#!/bin/python import os pipe=os.popen("ls /etc -alR| grep \"^[-l]\"|wc -l") #Expr1 a=int(pipe.read()) pipe.close() b=sum([len(files) for root,dirs,files in os.walk("/etc")]) #Expr2 print a print b print "a equals to b ?", str(a==b) #False print "Why?" 

What is the difference between Expr1 and Expr2 ? I think Expr1 gives the correct answer, but not sure.

Answer 1

Short answer:

ls -laR | grep "^[-l]" ls -laR | grep "^[-l]" counts symbolic links to directories. It matches any line starting with l and including symbolic links in directories.

In contrast, [files for root, dirs, files in os.walk('/etc')] does not refer to directory links . It ignores all directories and lists only files.

---

Long answer:

Here is how I determined the discrepancies:

 import os import subprocess import itertools def line_to_filename(line): # This assumes that filenames have no spaces, which is a false assumption # Ex: /etc/NetworkManager/system-connections/Wired connection 1 idx = line.rfind('->') if idx > -1: return line[:idx].split()[-1] else: return line.split()[-1] 

line_to_filename trying to find the file name in the output ls -laR .

This defines expr1 and expr2 and essentially matches your code.

 proc=subprocess.Popen( "ls /etc -alR 2>/dev/null | grep -s \"^[-l]\" ", shell = True, stdout = subprocess.PIPE) #Expr1 out, err = proc.communicate() expr1 = map(line_to_filename, out.splitlines()) expr2 = list(itertools.chain.from_iterable( files for root,dirs,files in os.walk('/etc') if files)) #Expr2 for expr in ('expr1', 'expr2'): print '{e} is of length {l}'.format(e = expr, l = len(vars()[expr])) 

This removes the names from expr1 , which are also in expr2 :

 for name in expr2: try: expr1.remove(name) except ValueError: print('{n} is not in expr1'.format(n = name)) 

After deleting the file names common to expr1 and expr2 ,

 print(expr1) 

gives

 ['i386-linux-gnu_xorg_extra_modules', 'nvctrl_include', 'template-dkms-mkdsc', 'run', '1', 'conf.d', 'conf.d'] 

Then I used find to search for these files in /etc and tried to guess what was unusual in these files. They were symbolic links to directories (not files).

Answer 2

If you use walk, errors are ignored (see this ), and ls sends a message for each error. They are considered words.

Answer 3

On my machine, / etc is a symbolic link to / private / etc, so ls /etc has only one line of output. ls /etc/ give the expected equivalence between ls and os.walk .