Destroying base64 byte arrays based on 64 returns different results in 2 versions of the JSON.net library

Question

JObject j = JObject.Parse("{'responseArray':'AAAAAAAAAAAAAAAAAAAAAAAAAAABAAAA'}"); byte[] r = j["responseArray"].ToObject<byte[]>(JsonSerializer.Create(new JsonSerializerSettings())); 

Expected Result (Works Great with JSON.net v4.5.6.14930)

 r = {byte[24]} [0]: 0 [1]: 0 [2]: 0 [3]: 0 [4]: 0 [5]: 0 [6]: 0 [7]: 0 [8]: 0 [9]: 0 [10]: 0 [11]: 0 [12]: 0 [13]: 0 [14]: 0 [15]: 0 [16]: 0 [17]: 0 [18]: 0 [19]: 0 [20]: 1 [21]: 0 [22]: 0 [23]: 0 

Deamination of the above code in JSON.net v6.0.8.18111 gives an erroneous result

Invalid output (JSON.net v6.0.8.18111)

 r= {byte[16]} [0]: 170 [1]: 170 [2]: 170 [3]: 170 [4]: 170 [5]: 170 [6]: 170 [7]: 170 [8]: 170 [9]: 170 [10]: 170 [11]: 170 [12]: 170 [13]: 171 [14]: 170 [15]: 170 

Am I doing something wrong here?

Note- The snippet in the question is part of a complex piece of code where the data is included as a JSON parameter for WebMethod. The responder is part of a complex object sent to the web method, so we use the JSON.net library for decoding.

Answer 1

Update

This is fixed in Json.NET 7.0 Release 1 .

Original answer

This is a Json.NET problem. The problem is that your string AAAAAAAAAAAAAAAAAAAAAAAAAAABAAAA simply successfully recognized as a GUID in JsonReader.ReadAsBytesInternal() :

  if (s.Length == 0) { data = new byte[0]; } else if (ConvertUtils.TryConvertGuid(s, out g)) { data = g.ToByteArray(); } else { data = Convert.FromBase64String(s); } 

In this trace:

 Newtonsoft.Json.JsonReader.ReadAsBytesInternal() Line 517 C# Newtonsoft.Json.Linq.JTokenReader.ReadAsBytes() Line 74 + 0x9 bytes C# Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ReadForType(Newtonsoft.Json.JsonReader reader = {Newtonsoft.Json.Linq.JTokenReader}, Newtonsoft.Json.Serialization.JsonContract contract = {Newtonsoft.Json.Serialization.JsonPrimitiveContract}, bool hasConverter = false) Line 1853 + 0x8 bytes C# Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(Newtonsoft.Json.JsonReader reader = {Newtonsoft.Json.Linq.JTokenReader}, System.Type objectType = {Name = "Byte[]" FullName = "System.Byte[]"}, bool checkAdditionalContent = false) Line 144 + 0x2f bytes C# Newtonsoft.Json.JsonSerializer.DeserializeInternal(Newtonsoft.Json.JsonReader reader = {Newtonsoft.Json.Linq.JTokenReader}, System.Type objectType = {Name = "Byte[]" FullName = "System.Byte[]"}) Line 710 + 0x52 bytes C# Newtonsoft.Json.JsonSerializer.Deserialize(Newtonsoft.Json.JsonReader reader = {Newtonsoft.Json.Linq.JTokenReader}, System.Type objectType = {Name = "Byte[]" FullName = "System.Byte[]"}) Line 689 + 0x11 bytes C# Newtonsoft.Json.Linq.JToken.ToObject(System.Type objectType = {Name = "Byte[]" FullName = "System.Byte[]"}, Newtonsoft.Json.JsonSerializer jsonSerializer = {Newtonsoft.Json.JsonSerializer}) Line 1837 + 0x11 bytes C# Newtonsoft.Json.Linq.JToken.ToObject(System.Type objectType = {Name = "Byte[]" FullName = "System.Byte[]"}) Line 1811 + 0x3c bytes C# Newtonsoft.Json.Linq.JToken.ToObject<byte[]>() Line 1698 + 0x39 bytes C# 

As you can see, this closes the base 64 decoding. An ugly mistake, and quite a failure on your part, to get this exact string.

To block incorrect Json.NET GUID recognition, you can create your own global JsonConverter array for byte arrays. When you do this, Json.NET will give you the raw string without matching the “useful” GUID pattern:

 public class ByteConverter : JsonConverter { public override bool CanConvert(Type objectType) { return objectType == typeof(byte[]); } public override bool CanWrite { get { return false; } } // Use the default implementation for serialization, which is not broken. public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer) { var value = (string)JToken.Load(reader); if (value == null) return null; if (value.Length == 0) return new byte[0]; return Convert.FromBase64String(value); } public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer) { throw new NotImplementedException(); } } 

And then to install it globally:

  JsonConvert.DefaultSettings = () => new JsonSerializerSettings { Converters = new List<JsonConverter> { new ByteConverter() } }; 

This workaround restores proper deserialization for byte arrays.

Answer 2

The problem is that JObject has already parsed AAAAAAAAAAAAAAAAAAAAAAAAAAABAAAA as string ( j["responseArray"].Type == JTokenType.String ), so j["responseArray"].ToObject<byte[]> does not do de-base64.

You need to parse it directly on byte[] , for example:

 public class MyObject { public byte[] responseArray { get; set; } } MyObject cl = JsonConvert.DeserializeObject<MyObject>("{'responseArray':'AAAAAAAAAAAAAAAAAAAAAAAAAAABAAAA'}"); 

Clearly, you can do Convert.FromBase64String "manually":

 JObject j = JObject.Parse("{'responseArray':'AAAAAAAAAAAAAAAAAAAAAAAAAAABAAAA'}"); byte[] r = Convert.FromBase64String((string)j["responseArray"]);