Retrieving a directory hierarchy using go

Question

I am trying to extract a directory hierarchy into a go data directory. filepath.Walk seems to be the way to go, but all I can do so far is print the file and folder names. Here is what I use:

 func main() { visit := func(path string, info os.FileInfo, err error) error { if info.IsDir() { fmt.Println("dir: ", path) } else { fmt.Println("file: ", path) } return nil } err := filepath.Walk("./", visit) if err != nil { log.Fatal(err) } } 

this prints the names of folders like:

 dir: folder1 file: folder1/file1.txt file: folder1/file2.txt file: folder1/file3.txt file: folder1/file4.txt dir: folder1/folder2 file: folder1/folder2/file5.txt file: folder1/folder2/file6.txt file: folder1/folder2/file7.txt file: folder1/folder2/file8.txt file: folder1/folder2/file9.txt 

for a tree structure I was thinking of using something like:

 type File struct { Name string Content string } type Folder struct { Name string Files []File Folders []Folder } 

but of course, any suggestions are welcome.

How can I convert this to a tree structure in go? Is there an easier way to do this?

Answer 1

AFAIK there is nothing ready for this in the standard version of Go.

Tree structures lend themselves well to a recursive approach. I have defined the addFile and addFolder in your file and folder types. Starting from the root folder, you can call these methods in Walk. If you get a / b / c, we will call root.addFile(a, b, c) , a.addFile(b, c) , b.addFile(c) .

I also changed Folder.Folders to a map, because filepath.Walk always gives us full paths, so we can split them and find their components on the folder map.

Here is some quick and dirty code that probably has errors and does not perform full error checking. It only works for the current directory, but this should be easy to fix.

I also added the String () method in the folder, which is recognized by the compiler and will be used when printing instances of the type.

 package main import ( "log" "os" "path/filepath" "strings" ) type File struct { Name string } type Folder struct { Name string Files []File Folders map[string]*Folder } func newFolder(name string) *Folder { return &Folder{name, []File{}, make(map[string]*Folder)} } func (f *Folder) getFolder(name string) *Folder { if nextF, ok := f.Folders[name]; ok { return nextF } else { log.Fatalf("Expected nested folder %v in %v\n", name, f.Name) } return &Folder{} // cannot happen } func (f *Folder) addFolder(path []string) { for i, segment := range path { if i == len(path)-1 { // last segment == new folder f.Folders[segment] = newFolder(segment) } else { f.getFolder(segment).addFolder(path[1:]) } } } func (f *Folder) addFile(path []string) { for i, segment := range path { if i == len(path)-1 { // last segment == file f.Files = append(f.Files, File{segment}) } else { f.getFolder(segment).addFile(path[1:]) return } } } func (f *Folder) String() string { var str string for _, file := range f.Files { str += f.Name + string(filepath.Separator) + file.Name + "\n" } for _, folder := range f.Folders { str += folder.String() } return str } func main() { startPath := "." rootFolder := newFolder(startPath) visit := func(path string, info os.FileInfo, err error) error { segments := strings.Split(path, string(filepath.Separator)) if info.IsDir() { if path != startPath { rootFolder.addFolder(segments) } } else { rootFolder.addFile(segments) } return nil } err := filepath.Walk(startPath, visit) if err != nil { log.Fatal(err) } log.Printf("%v\n", rootFolder) } 

Answer 2

I needed something similar for my small application, so I wrote a small separate library, which is available for your pleasure on Github . Since I need inline JSON serialization for the returned os.FileInfo, I added it as well.

I know that it is too late for the original author of this question, but to post it here anyway if someone is looking for something like that. Pull asks easy to accept :)

Answer 3

Small change

 package main import ( "fmt" "path" "strings" ) type File struct { Id string Name string } type Folder struct { Name string Files []File Folders map[string]*Folder } func newFolder(name string) *Folder { return &Folder{name, []File{}, make(map[string]*Folder)} } func (f *Folder) getFolder(name string) *Folder { if nextF, ok := f.Folders[name]; ok { return nextF } else if f.Name == name { return f } else { return &Folder{} } } func (f *Folder) existFolder(name string) bool { for _, v := range f.Folders { if v.Name == name { return true } } return false } func (f *Folder) addFolder(folderName string) { if !f.existFolder(folderName) { f.Folders[folderName] = newFolder(folderName) } } func (f *Folder) addFile(fileName string, fileId string) { f.Files = append(f.Files, File{fileId, fileName}) } func (f *Folder) getList() (result []map[string]interface{}) { for _, v := range f.Folders { result = append(result, map[string]interface{}{ "name": v.Name, "type": "folder", }) } for _, v := range f.Files { result = append(result, map[string]interface{}{ "id": v.Id, "name": v.Name, "type": "file", }) } return } func isFile(str string) bool { if path.Ext(str) != "" { return true } return false } func DeleteEmptyElements(s []string) []string { var r []string for _, str := range s { if str != "" { r = append(r, str) } } return r } type IS map[string]string func main() { arrayPaths := []interface{}{ IS{ "id": "1", "filePath": "/print/some/com.png", }, IS{ "id": "2", "filePath": "/print/some2/com412412.png", }, IS{ "id": "3", "filePath": "/print/some2/41241241241.png", }, } breadcrumb := "/print/some2" startPath := "/" rootFolder := newFolder(startPath) for _, path := range arrayPaths { filePath := path.(IS)["filePath"] fileId := path.(IS)["id"] splitPath := DeleteEmptyElements(strings.Split(filePath, "/")) tmpFolder := rootFolder for _, item := range splitPath { if isFile(item) { tmpFolder.addFile(item, fileId) } else { if item != startPath { tmpFolder.addFolder(item) } tmpFolder = tmpFolder.getFolder(item) } } } currentFolder := rootFolder.getFolder("/") breadcrumbElements := DeleteEmptyElements(strings.Split(breadcrumb, "/")) for i, v := range breadcrumbElements { if currentFolder.existFolder(v) { currentFolder = currentFolder.getFolder(v) if i == len(breadcrumbElements)-1 { break } } else { currentFolder = currentFolder.getFolder(v) } } fmt.Println(currentFolder.getList()) } 

Answer 4

use only one for loop and filepath.Walk

 package main import ( "encoding/json" "fmt" "log" "os" "path" "path/filepath" ) func main() { tree := BuildTree(os.Args[1]) fmt.Println(tree) } type File struct { Name string } type Folder struct { Name string Files []*File Folders map[string]*Folder } func (f *Folder) String() string { j, _ := json.Marshal(f) return string(j) } func BuildTree(dir string) *Folder { dir = path.Clean(dir) var tree *Folder var nodes = map[string]interface{}{} var walkFun filepath.WalkFunc = func(p string, info os.FileInfo, err error) error { if info.IsDir() { nodes[p] = &Folder{path.Base(p), []*File{}, map[string]*Folder{}} } else { nodes[p] = &File{path.Base(p)} } return nil } err := filepath.Walk(dir, walkFun) if err != nil { log.Fatal(err) } for key, value := range nodes { var parentFolder *Folder if key == dir { tree = value.(*Folder) continue } else { parentFolder = nodes[path.Dir(key)].(*Folder) } switch v := value.(type) { case *File: parentFolder.Files = append(parentFolder.Files, v) case *Folder: parentFolder.Folders[v.Name] = v } } return tree }