How to parallelize 2 functions and catch errors?

Question

I do golan, and I have no idea how to catch mistakes.

What I expect:

• FetchTickerData works
• It calls two different functions at the same time: fetchPriceTicket and fetchWhatToMine
• If one of the functions returns an error, then FetchTickerData returns this error.
• If everything is in order, processes data from both sources and returns them.

I can't figure out how to catch bugs. I wrote this code, but I do not think this is the right solution, and it does not work. What is the best way to do this?

 package main import "net/http" import ( "github.com/tidwall/gjson" "time" "io/ioutil" "fmt" ) var client = &http.Client{Timeout: 10 * time.Second} type Ticker struct { } func FetchTickerData() (error, *gjson.Result, *gjson.Result) { whatToMine := make(chan *gjson.Result) currency := make(chan *gjson.Result) err := make(chan error) counter := 0 // This variable indicates if both data was fetched go func() { innerError, json := fetchWhatToMine() fmt.Print(innerError) if innerError != nil { err <- innerError // Stop handler immediately whatToMine <- nil currency <- nil return } whatToMine <- json counter = counter + 1 if counter == 2 { fmt.Print("err pushed") err <- nil } }() go func() { innerError, json := fetchPriceTicket() fmt.Print(innerError) if innerError != nil { err <- innerError whatToMine <- nil currency <- nil return } currency <- json counter = counter + 1 if counter == 2 { fmt.Print("err pushed") err <- nil } }() return <-err, <-whatToMine, <-currency } func fetchPriceTicket() (error, *gjson.Result) { resp, err := client.Get("https://api.coinmarketcap.com/v1/ticker/") if err != nil { return err, nil } defer resp.Body.Close() body, _ := ioutil.ReadAll(resp.Body) json := gjson.GetBytes(body, ""); return nil, &json; } func fetchWhatToMine() (error, *gjson.Result) { resp, err := client.Get("https://whattomine.com/coins.json") if err != nil { return err, nil } defer resp.Body.Close() body, _ := ioutil.ReadAll(resp.Body) json := gjson.GetBytes(body, ""); return nil, &json; } 

UPD: if I replace return <-err, <-whatToMine, <-currency with return nil, <-whatToMine, <-currency , it returns the data that I expect, but does not return an error, if any.

UPD: there is a second version of the code:

 package main import "net/http" import ( "github.com/tidwall/gjson" "time" "io/ioutil" "context" "fmt" ) var client = &http.Client{Timeout: 10 * time.Second} type Ticker struct { } func main() { ticker, coins, err := FetchTickerData() fmt.Print("Everything is null! ", ticker, coins, err) if err != nil { fmt.Print(err) return } fmt.Print("Bitcoin price in usd: ", ticker.Array()[0].Get("price_usd")) } func FetchTickerData() (*gjson.Result, *gjson.Result, error) { ctx, cancel := context.WithCancel(context.Background()) defer cancel() var result1, result2 *gjson.Result var err1, err2 error go func() { result1, err1 = fetchJson(ctx, "https://api.coinmarketcap.com/v1/ticker/") if err1 != nil { cancel() // Abort the context, so the other function can abort early } }() go func() { result2, err2 = fetchJson(ctx, "https://whattomine.com/coins.json") if err2 != nil { cancel() // Abort the context, so the other function can abort early } }() if err1 == context.Canceled || err1 == nil { return result1, result2, err2 } return result1, result2, err1 } func fetchJson(ctx context.Context, url string) (*gjson.Result, error) { req, err := http.NewRequest(http.MethodGet, url, nil) if err != nil { return nil, err } req = req.WithContext(ctx) resp, err := client.Do(req) if err != nil { return nil, err } defer resp.Body.Close() body, err := ioutil.ReadAll(resp.Body) if err != nil { return nil, err } fmt.Print("I don't know why this body isn't printed ", string(body)) json := gjson.ParseBytes(body) return &json, nil } 

For some reason, HTTP requests do not work here, and there are no errors. Ideas?

 Everything is null! <nil> <nil> <nil>panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x11f7843] goroutine 1 [running]: main.main() /Users/andrey/go/src/tickerUpdater/fetchTicker.go:25 +0x183 

Answer 1

Since you no longer use channels, the main procedure ends before the start of the execution of the other two goroutines, therefore, exit the program. you must use waitgroups to lock the main goroutine until the other two have finished their work.

 package main import "net/http" import ( "context" "fmt" "io/ioutil" "sync" "time" "github.com/tidwall/gjson" ) var client = &http.Client{Timeout: 10 * time.Second} type Ticker struct { } func main() { ticker, coins, err := FetchTickerData() fmt.Print("Everything is null! ", ticker, coins, err) if err != nil { fmt.Print(err) return } fmt.Print("Bitcoin price in usd: ", ticker.Array()[0].Get("price_usd")) } func FetchTickerData() (*gjson.Result, *gjson.Result, error) { var wg sync.WaitGroup ctx, cancel := context.WithCancel(context.Background()) defer cancel() var result1, result2 *gjson.Result var err1, err2 error wg.Add(2) go func() { defer wg.Done() result1, err1 = fetchJson(ctx, "https://api.coinmarketcap.com/v1/ticker/") if err1 != nil { cancel() // Abort the context, so the other function can abort early } }() go func() { defer wg.Done() result2, err2 = fetchJson(ctx, "https://whattomine.com/coins.json") if err2 != nil { cancel() // Abort the context, so the other function can abort early } }() wg.Wait() if err1 == context.Canceled || err1 == nil { return result1, result2, err2 } return result1, result2, err1 } func fetchJson(ctx context.Context, url string) (*gjson.Result, error) { req, err := http.NewRequest(http.MethodGet, url, nil) if err != nil { return nil, err } req = req.WithContext(ctx) resp, err := client.Do(req) if err != nil { return nil, err } defer resp.Body.Close() body, err := ioutil.ReadAll(resp.Body) if err != nil { return nil, err } fmt.Print("I don't know why this body isn't printed ", string(body)) json := gjson.ParseBytes(body) return &json, nil } 

Answer 2

This is an ideal use case for the context package. I deleted part of your template and the second function; You will want to add this for your actual code.

 func FetchTickerData() (*gjson.Result, *gjson.Result, error) { ctx, cancel := context.WithCancel(context.Background()) defer cancel() var result1, result2 *gjson.Result var err1, err2 error var wg sync.WaitGroup wg.Add(1) go func() { defer wg.Done() result1, err1 := fetchPriceTicket(ctx) if err1 != nil { cancel() // Abort the context, so the other function can abort early }() } wg.Add(1) go func() { defer wg.Done() result2, err2 := fetchWhatToMine(ctx) if err2 != nil { cancel() // Abort the context, so the other function can abort early } }() wg.Wait() // if err1 == context.Canceled, that means the second goroutine had // an error and aborted the first goroutine, so return err2. // If err1 == nil, err2 may still be set, so return it in this case // as well. if err1 == context.Canceled || err1 == nil { return result1, result2, err2 } return result1, result2, err1 } func fetchPriceTicket(ctx context.Context) (*gjson.Result, error) { req, err := http.NewRequest(http.MethodGet, "https://api.coinmarketcap.com/v1/ticker/", nil) if err != nil { return nil, err } req = req.WithContext(ctx) resp, err := client.Do(req) if err != nil { return nil, err } defer resp.Body.Close() body, err := ioutil.ReadAll(resp.Body) if err != nil { return nil, err } json := gjson.GetBytes(body, "") return &json, nil }