Newtonsoft Converter FromJson - unexpected token

Question

I tried to write a JSON deserializer for a while, but could not find my error. Why does Newtonsoft tell me Unexpected token when deserializing object: StartObject , after deserializing it?

 type ThisFails = { a : string * string m : Map<string, string> } type ThisWorks = { y : Map<string, string> z : string * string } testCase "failing test - array before object" <| fun _ -> let res = deserialise<ThisFails> Serialisation.converters """{"a":["xyz","zyx"],"m":{}}""" Assert.Equal("should be eq to res", { a = "xyz", "zyx"; m = Map.empty }, res) testCase "passing test - array after object" <| fun _ -> let res = deserialise<ThisWorks> Serialisation.converters """{"y":{},"z":["xyz","zyx"]}""" Assert.Equal("should be eq to res", { y = Map.empty; z = "xyz", "zyx" }, res) 

The object is a TupleArrayConverter .

Trace of this converter:

 reading json [Newtonsoft.Json.FSharp.TupleArrayConverter] type => System.Tuple`2[System.String,System.String] value token, pre-deserialise [Newtonsoft.Json.FSharp.TupleArrayConverter] path => "a[0]" token_type => String value token, post-deserialise [Newtonsoft.Json.FSharp.TupleArrayConverter] path => "a[1]" token_type => String value token, pre-deserialise [Newtonsoft.Json.FSharp.TupleArrayConverter] path => "a[1]" token_type => String value token, post-deserialise [Newtonsoft.Json.FSharp.TupleArrayConverter] path => "a" token_type => EndArray after EndArray token, returning [Newtonsoft.Json.FSharp.TupleArrayConverter] path => "m" token_type => PropertyName 

In the converter, I use the last token, the final array, as you can see in the final case:

 match reader.TokenType with | JsonToken.EndArray -> read JsonToken.EndArray |> req |> ignore acc 

And I first use the StartArray token ...

So: why does this code not work? (Newtonsoft.Json 6.0.8)

This is mistake:

 map tests/failing test - array before object: Exception: Newtonsoft.Json.JsonSerializationException: Unexpected token when deserializing object: StartObject. Path 'm', line 1, position 24. at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ResolvePropertyAndCreatorValues (Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty containerProperty, Newtonsoft.Json.JsonReader reader, System.Type objectType, IDictionary`2& extensionData) [0x00000] in <filename unknown>:0 at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObjectUsingCreatorWithParameters (Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty containerProperty, Newtonsoft.Json.Serialization.ObjectConstructor`1 creator, System.String id) [0x00000] in <filename unknown>:0 at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateNewObject (Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonObjectContract objectContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, Newtonsoft.Json.Serialization.JsonProperty containerProperty, System.String id, System.Boolean& createdFromNonDefaultCreator) [0x00000] in <filename unknown>:0 at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x00000] in <filename unknown>:0 at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x00000] in <filename unknown>:0 at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize (Newtonsoft.Json.JsonReader reader, System.Type objectType, Boolean checkAdditionalContent) [0x00000] in <filename unknown>:0 (00:00:00.1500996) 1 tests run: 0 passed, 0 ignored, 0 failed, 1 errored (00:00:00.2482623) 

Answer 1

Debugging your code with JSON.Net.

It turns out that after you consume the EndArray token in your unsuccessful case, the reader then points to the PropertyName token, which is all well and good.

Then after completing your converter, JSON.Net will do this.

 } while (!exit && reader.Read()); 

Read() then takes the reader to the next token, which in your case with a StartObject error causes the deserializer to fail.

So, I am not an expert in JSON.Net, but I am thinking of providing a string value provider in JSON.Net. I probably won't promote the reader after the conversion, which means the reader is still pointing to a string value. In the same line of thinking, it makes sense when it consumes an array in order to leave the reader the last character of the array value, i.e. the EndArray marker.

So my suggestion is simply the following:

 match reader.TokenType with | JsonToken.EndArray -> // read JsonToken.EndArray |> req |> ignore Logger.debug logger <| fun _ -> LogLine.sprintf [ "path", reader.Path |> box "token_type", reader.TokenType |> box ] "after EndArray token, returning" acc 

This makes my test program:

 [<EntryPoint>] let main argv = let works = deserialize<ThisWorks> """{"y":{},"z":["xyz","zyx"]}""" printfn "%A" works let fails = deserialize<ThisFails> """{"a":["xyz","zyx"],"m":{}}""" printfn "%A" fails 0 

Print

 {y = map []; z = ("xyz", "zyx");} {a = ("xyz", "zyx"); m = map [];} 

We hope this helps you resolve this error (you may already have done this)