How to print results from parallel and recursive function?

Question

I went through a review tour and I completed the Internet search exercise, but I think that the method I used to print all the results may be ineffective.

Here is my code. I just edited the workaround and main features, so I will just post this. Here is a link to the exercise ( http://tour.golang.org/#70 )

var used = make(map[string]bool) func Crawl(url string, depth int, fetcher Fetcher, results chan string) { if depth <= 0 { return } body, urls, err := fetcher.Fetch(url) if err != nil { results <- fmt.Sprintf("%v",err) return } results <-fmt.Sprintf("\nfound: %s %q\n", url, body) for _,u := range urls { if used[u] == false { used[u] = true go Crawl(u, depth-1, fetcher, results) } } return } //------------------------------------------------------------ func main() { used["http://golang.org/"] = true results := make(chan string) go Crawl("http://golang.org/", 4, fetcher, results) for i := 0; i < len(used); i++ { fmt.Println(<-results) } } 

I use the line "for i <len (used)" in the main to ensure that the value of the results will be printed only if there is a result to print. I can't just use

  for i := range results 

because it’s difficult to use “close (results)” in the traversal function, since it is recursive, but with how I do it, I need to find the length of the variable each time.

Is there a better way to do this?

Answer 1

To wait for the goroutines collection to finish, use sync.WaitGroup.

I believe you'll find the example in the official documentation very familiar ..

http://golang.org/pkg/sync/#example_WaitGroup

Quoting:

  var wg sync.WaitGroup
 var urls = [] string {
     "http://www.golang.org/",
     "http://www.google.com/",
     "http://www.somestupidname.com/",
 }
 for _, url: = range urls {
     // Increment the WaitGroup counter.
     wg.Add (1)
     // Launch a goroutine to fetch the URL.
     go func (url string) {
         // Fetch the URL.
         http.Get (url)
         // Decrement the counter.
         wg.Done ()
     } (url)
 }
 // Wait for all HTTP fetches to complete.
 wg.Wait ()

This will be blocked until all work is completed.

If you really want to print the results gradually as they are collected, the easiest way is to do this in the set itself.