Breaking changes in GetFiles using 0.15.2 cannot exclude folders

Question

I can’t remember where, but I found an example of how to exclude a folder from the search. Our problem was finding node_modules leading to long path exceptions.

Func<IFileSystemInfo, bool> exclude_node_modules = fileSystemInfo=>!fileSystemInfo.Path.FullPath.Contains("node_modules"); var solutions = GetFiles("./**/*.sln", exclude_node_modules); 

Any help to solve this problem would be helpful.

Answer 1

To speed up the recursive walking of the file system, Cake uses the built-in .NET functions to do this, however it is limited to the 260 char Windows prefix. Thus, when it becomes faster in most use cases, it fails with folder structures that are too deep, such as, for example, Node modules.

You can solve this problem by iterating the folder by folder and applying the predicate in which folder to exclude it before entering.

In my example, the following folder structure is used

 Repo directory | build.cake | test.sln | \---src | test.sln | +---proj1 | | test.sln | | | \---node_modules | node.sln | +---proj2 | | test.sln | | | \---node_modules | node.sln | +---proj3 | | test.sln | | | \---node_modules | node.sln | \---proj4 | test.sln | \---node_modules node.sln 

We want to find all solutions recursively from the repo directory not in the node_modules directory, and not find node.sln

The suggested solution is as follows: create a RecursiveGetFile utility method that does this for you:

 // find and iterate all solution files foreach(var filePath in RecursiveGetFile( Context, "./", "*.sln", path=>!path.EndsWith("node_modules", StringComparison.OrdinalIgnoreCase) )) { Information("{0}", filePath); } // Utility method to recursively find files public static IEnumerable<FilePath> RecursiveGetFile( ICakeContext context, DirectoryPath directoryPath, string filter, Func<string, bool> predicate ) { var directory = context.FileSystem.GetDirectory(context.MakeAbsolute(directoryPath)); foreach(var file in directory.GetFiles(filter, SearchScope.Current)) { yield return file.Path; } foreach(var file in directory.GetDirectories("*.*", SearchScope.Current) .Where(dir=>predicate(dir.Path.FullPath)) .SelectMany(childDirectory=>RecursiveGetFile(context, childDirectory.Path, filter, predicate)) ) { yield return file; } } 

The result of this script will be something like

 RepoRoot/test.sln RepoRoot/src/test.sln RepoRoot/src/proj1/test.sln RepoRoot/src/proj2/test.sln RepoRoot/src/proj3/test.sln RepoRoot/src/proj4/test.sln 

This fixes the 260 char problem, skipping known problems will not be fixed if other unknown paths have the same problem.