How to find, "invalid character", "search for start value" error message

Question

I have a short Go program that runs the go list -json for several packages, saves the output of each command run in json.RawMessage, adds each json.RawMessage to the json.RawMessages piece, and then returns the result to the server after merging each of json.RawMessages together and json composites. However, an error message appears that is generated when json.Compact starts that I cannot find the source. As a result of this error message, it can be seen that most people who seem to encounter this - regardless of whether it has an invalid character or some other character - have difficulty finding its source.

 invalid character ',' looking for beginning of value 

The code with comments can be viewed here on play.golang.org (although it will not work there), as well as below,

Question: Can you explain the source of this error and how to prevent it?

(Please note that some of the packages were included for testing purposes only)

 package main import ( "expvar" "encoding/json" "bytes" "fmt" "github.com/go-martini/martini" "github.com/zenazn/goji" "github.com/zenazn/goji/web" "go/build" "log" "math/rand" "net/http" _ "net/http/pprof" "os/exec" ) type myType struct { J []json.RawMessage } var pack map[string]string type GoList struct { Imports []string } type Import struct { Dir string ImportPath string Name string Target string Standard bool Root string GoFiles []string Imports []string Deps []string } const contentTypeJSON = "application/json" func main() { http.HandleFunc("/importgraph", func(w http.ResponseWriter, r *http.Request) { importGraph(w, r) }) http.HandleFunc("/", handler) http.ListenAndServe(":8080", nil) } func handler(w http.ResponseWriter, r *http.Request) { fmt.Println("Inside handler") fmt.Fprintf(w, "Hello world from my Go program!") } func importGraph(w http.ResponseWriter, r *http.Request) { pack = make(map[string]string) var t myType cmd := exec.Command("go", "list", "-json") stdout, err := cmd.Output() if err != nil { println(err.Error()) return } var list GoList err = json.Unmarshal(stdout, &list) for _, d := range list.Imports { //get the imports for each of the packages listed by go list -json t.imports(d) } var buff bytes.Buffer //concatenate the separate json.RawMessages together into json buff.WriteByte('[') for i, j := range tJ { if i != 0 { buff.WriteByte(',') } buff.Write([]byte(j)) } buff.WriteByte(']') var buffer bytes.Buffer if err := json.Compact(&buffer, buff.Bytes()); err != nil { println(err.Error()) //error message: invalid character ',' looking for beginning of value return } w.Header().Set("Content-Type", contentTypeJSON) w.Write(buffer.Bytes()) } func (myObj *myType) imports(pk string) error { cmd := exec.Command("go", "list", "-json", pk) stdout, _ := cmd.Output() pack[pk] = pk var deplist Import json.Unmarshal(stdout, &deplist) var newj json.RawMessage json.Unmarshal(stdout, &newj) myObj.J = append(myObj.J, newj) for _, imp := range deplist.Imports { if _, ok := pack[imp]; !ok { myObj.imports(imp) //recursive call to get the imports of the imports etc } } return nil } 

Answer 1

Firstly, as was commented, are you sure you cannot use the go/build package and not run the go list ?

I would not use println (or fmt.Println ) inside the HTTP handlers. It is much better to use log.Println and / or get an error in ResponseWriter . It is also recommended that you make a call to ListenAndServe using log.Fatal .

When printing / registering error values, you can simply use err , you do not need to have err.Error() .

In addition, when you really want to do something more detailed than just reporting / logging an error message, you can see its type and other information. For example, log.Printf("verbose error info: %#v", err) gives:

 &json.SyntaxError{msg:"invalid character ',' looking for beginning of value", Offset:0} 

I tried this because I know that the json package returns various types of errors with additional information, and I was hoping the offset value would be useful. If that were the case, perhaps it would be helpful:

 if err := json.Compact(…) { if err != nil { log.Println("json.Compact:", err) if serr, ok := err.(*json.SyntaxError); ok { log.Println("Occurred at offset:", serr.Offset) // … something to show the data in buff around that offset … } } } 

But a zero offset is not useful :(

So, although this does not identify you, I hope this can help you in your further investigation.

Edit:

So by adding:

 log.Println("Write file:", ioutil.WriteFile("data.json", buff.Bytes(), 0600)) 

to the above error handling block, I then ran JSON authentication in the resulting file, and it identified this part:

  "XTestImports": [ "io", "log", "net" ] },,{ "Dir": "/usr/local/go/src/mime", "ImportPath": "mime", "Name": "mime", 

Pay attention to double ,, .

This should tell you that there is an error in your code. But if not, you need to skip empty entries, either when processing tJ , or when creating it. Later it is better and simply includes:

  if len(newj) > 0 { myObj.J = append(myObj.J, newj) } 

(where btw you do not check for errors from json.Unmarshal so that it does not clear if it should be empty or if it is empty due to a previous error. Never ignore the error returns!)

Answer 2

I also came across the same error message in the Go program, but the error message was in the HTML response of the HTML response when I was expecting my HTTP JSON parser.

For me, the solution was to modify my request by including the value application/json in the Content-Type header. How you do this depends on which http client library you use; if you have access to the http.Header type, you can set the header using .Set(...) .

I understand that the scope of this fix for me may not relate to the original question, but I came here first after searching on Google and thought that this would help others, since the message was not especially obvious at first glance. The hint is that the invalid character < is the first HTML character in the error / response, which is most likely the result of the request type not set to application/json , so the server responds with a non-JSON response.

Answer 3

If this helps someone,

For me, the problem was that I was trying to parse already parsed JSON.

Answer 4

And in case someone has the same problem as mine, I had to call JSON.stringify for my post data.

Answer 5

I ran into a similar problem with my error message:

Invalid self symbol seeks the beginning of a value

In my case, I tried to decode BSON using json.Unmarshal . Json does not recognize the type of ISODate that caused this error.