I would like to reduce the access time to my table on my Golang RestApi.
I have a restAPI link with enpoint that requires access to the bigtable database for each code specified in the request body. My key access design is "provider | customer | productCode | date | .." (and the other 4 parameters).
For each productCode of the request body, I make an asynchronous call to read the line with the specified key.
In my test, I asked for 1000 product codes, and the ones I received on my local machine about 800 ~ 900 ms. The time on the cloud is about 550 ~ 450 ms.
I am using a bigtable package to access bigTable and I am using the following code:
package main import ( "log" "time" "golang.org/x/net/context" "google.golang.org/cloud/bigtable" ) func main() { start2 := time.Now() listPKs := []string{"PROV|CLI|6030|20160714|8|30301.30302|ES", "PROV|CLI|44103|20160714|8|30301.30302|ES", "PROV|CLI|1454871|20160714|8|30301.30302|ES"} providerRS := getBDresponse(listPKs, 50000) if providerRS != nil { ///do something } elapsed2 := time.Since(start2) log.Printf("Time access BT: %s", elapsed2) } func getBDresponse(listPKs []string, timeoutMiliseconds int) []string { resp := make(chan string) table := myClient.Client.Open(TABLE_NAME) //async call to BT for _, key := range listPKs { go asyncDoUniqueCall(key, resp, myContext, table) } //get all responses providerRS := getResponses(resp, timeoutMiliseconds, len(listPKs)) return providerRS } func asyncDoUniqueCall(pk string, ch chan string, ctx *context.Context, table *bigtable.Table) { ch <- GetRowValue(pk, ctx, table) } func getResponses(resp chan string, timeoutMiliseconds int, totalNumRQ int) []string { var ret []string for j := 0; j < totalNumRQ; j++ { select { case rowResponse := <-resp: //catch the response if rowResponse != "" { ret = append(ret, rowResponse) } case <-time.After(time.Duration(timeoutMiliseconds) * time.Millisecond): // timeout control return nil } } return ret } //GetRowValue open the table and get the row indexed by pk and returns the stored message func GetRowValue(pk string, ctx *context.Context, table *bigtable.Table) string { var response string r, err := table.ReadRow(*ctx, pk) if err != nil { return "" } if len(r) > 0 { row := r[COLUMN_FAMILY_NAME] numCol := len(row) response = string(row[0].Value) } return response }
I tried this example, and time is no better:
Do I use gorutines and correctyl chanels? Is there a way in which I access BT correctly? Maybe the key design is wrong?