Recursively process the same files inside many subdirectories in Java

Question

Hi, I would like to process files inside many subdirectories using Java. Psuedo code will be

while(mainDir.hasMoreDirectory()) { getFilesFromCurrentDirectory() passThoseFilesAsArgumentToProcess() } 

I am currently using the following code

 public void list(File file) { System.out.println(file.getName()); File[] children = file.listFiles(); for (File child : children) { list(child); } } 

Files are simply listed above the code. Another thing I can do is to keep the list of files and directories in the list, and then process it in another loop. But I can’t come up with what I want, how to show in pseudo-code. I am new to file directories, please help. Thanks in advance.

Answer 1

If you use Java 7, you can use the advanced NIO features in the form of the Files.walkFileTree method. Passing the file system has never been easier in Java.

There is a short usage tutorial here .

It implements a visitor pattern, so you don’t have to worry about the crawl algorithm itself, just indicating what you want to do with each entry.

Answer 2

When moving the directory tree in Java 7, use the Paths and Files functions. They not only make reading directories and files easier, but also faster than the "old" File method.

Suppose you have two directories: mainDir and otherDir , and you want to go through all the directories of mainDir to its leaves. With each entry in maiondir (file, subdirectory, symbolic link, ...) you want to compare this entry and its attributes (size, modification time ...) with the entry in the same position in otherDir . Then this will be your code:

 public final void test() throws IOException, InterruptedException { final Path mainDir = Paths.get("absolute path to your main directory to read from"); final Path otherDir = Paths.get("absolute path to your other directory to compare"); // Walk thru mainDir directory Files.walkFileTree(mainDir, new FileVisitor<Path>() { @Override public FileVisitResult preVisitDirectory(Path path, BasicFileAttributes atts) throws IOException { return visitFile(path, atts); } @Override public FileVisitResult visitFile(Path path, BasicFileAttributes mainAtts) throws IOException { // I've seen two implementations on windows and MacOSX. One has passed the relative path, one the absolute path. // This works in both cases Path relativePath = mainDir.relativize(mainDir.resolve(path)); BasicFileAttributes otherAtts = Files.readAttributes(otherDir.resolve(relativePath), BasicFileAttributes.class); // Do your comparison logic here: compareEntries(mainDir, otherDir, relativePath, mainAtts, otherAtts); return FileVisitResult.CONTINUE; } @Override public FileVisitResult postVisitDirectory(Path path, IOException exc) throws IOException { // TODO Auto-generated method stub return FileVisitResult.CONTINUE; } @Override public FileVisitResult visitFileFailed(Path path, IOException exc) throws IOException { exc.printStackTrace(); // If the root directory has failed it makes no sense to continue return (path.equals(mainDir))? FileVisitResult.TERMINATE:FileVisitResult.CONTINUE; } }); } 

What is he not :

• Find entries that exist in otherDir but not in mainDir
• Path and BasicFileAttributes not Serializable , so there is no easy way to do this on two different machines.

Answer 3

Will follow?

 public void list(File file) { File[] children = file.listFiles(); if (children != null) { process(children); for (File child : children) { if (child.isDirectory()) { list(child); } } } else { process(new File[]{file}); } } private void process(File[] children) { for (File child : children) { if (child.isFile()) { // process normal file } } } 

Answer 4

A similar method will return a List all files recursive to the directory. You can work with the returned List or replace rtn.add calls rtn.add processing.

Remember that this method has nothing to stop it getting stuck in circular symbolic links.

 public static List<File> getFilesRecursive(File s) { ArrayList<File> rtn = new ArrayList<File>(); File[] contents = s.listFiles(); for(int i = 0; i<contents.length; i++) { if(contents[i].isDirectory()){ rtn.addAll(getFilesRecursive(contents[i])); }else{ rtn.add(contents[i]); } } return rtn; } 

Answer 5

Perhaps this piece of code will help you:

 public void traverse(String path) { File root = new File(path); File[] list = root.listFiles(); if (list == null) return; for (File file : list) { if (file.isDirectory()) { traverse(file.getAbsolutePath()); System.out.println("Directory: " + file.getAbsoluteFile()); } else { System.out.println("File: " + file.getAbsoluteFile()); } } } 

Answer 6

 private static List<File> allFiles = new ArrayList<File>(); private static void processFiles(String rootDirectory) { File rootDir = new File(rootDirectory); if (rootDir.exists()) { traverseDirectories(rootDir); } } private static void traverseDirectories(File file) { // add all files and directories to list. allFiles.add(file); if (file.isDirectory()) { File[] fileList = file.listFiles(); for (File fileHandle : fileList) { traverseDirectories(fileHandle); } } else { // call to process file System.out.println("Call to process file " + file.getAbsolutePath()); } }