Empty JSON Array

Question

I am trying to parse some JSON that comes back from a REST web service. The return from the get () call is TStringStream. I am using dbxjson to work with data. To simplify the demo here, I created a test project that reproduces the error without calling the web service (instead, it uses a text file to output the web service instead). Here is the code:

var SL : TStringStream; LJsonObj : TJSONObject; begin SL := TStringStream.Create; try SL.LoadFromFile('output.txt'); LJsonObj := TJSONObject.ParseJSONValue(TEncoding.ASCII.GetBytes(SL.DataString), 0) as TJSONObject; finally SL.Free; end; end; 

Sometimes the phone_numbers array in the JSON data is empty. In a streaming object coming from a web service call, it looks like this:

 { "Contact Information Service": { "response": { "phone_numbers": [ ] } } } 

This causes ParseJSONValue to return nil.

However, if I change the empty phone_numbers array to this in my test txt file:

 { "Contact Information Service": { "response": { "phone_numbers": [] } } } 

works fine (i.e. returns a TJSONObject). The difference is a space in an empty array. For some reason, the first JSON response with a space in an empty array causes ParseJSONValue to return zero. It works fine without spaces between square brackets.

What am I doing wrong with my JSON analysis? Is there some kind of preliminary parsing I need to do before calling ParseJSONValue?

Answer 1

This problem does not exclude the implementation of Delphi JSON (DBXJSON), I worked with some PHP JSON parsers with the same restriction.

Now, since all spaces outside of double-quoted literal literals (and should be) are ignored by JSON parsers, you can safely remove these white spaces. So a possible workaround is to Minify your Json string before parsing it.

Try this sample, which uses regular expressions to remove extra spaces from the string.

 {$APPTYPE CONSOLE} {$R *.res} uses System.RegularExpressions, System.Classes, System.SysUtils, Data.DBXJSON; const JsonString= '{'+ ' "Contact Information Service": {'+ ' "response": {'+ ' "phone_numbers": [ ]'+ ' }'+ ' }'+ '}'; function JsonMinify(const S: string): string; begin Result:=TRegEx.Replace(S,'("(?:[^"\\]|\\.)*")|\s+', '$1'); end; procedure TestJSon; var s : string; SL : TStringStream; LJsonObj : TJSONObject; begin SL := TStringStream.Create; try s:=JsonMinify(JsonString); SL.WriteString(s); LJsonObj := TJSONObject.ParseJSONValue(TEncoding.ASCII.GetBytes(SL.DataString), 0) as TJSONObject; Writeln(LJsonObj.Size); finally SL.Free; end; end; begin try TestJSon; except on E:Exception do Writeln(E.Classname, ':', E.Message); end; Writeln('Press Enter to exit'); Readln; end. 

Answer 2

Take a look at TJsonObject.ParseArray . You will find the following:

 while ValueExpected or (Br.PeekByte <> Ord(']')) do begin ConsumeWhitespaces(Br); Pos := ParseValue(Br, JsonArray); if Pos <= 0 then Exit(Pos); 

So, at the top of the array (right after it reads the open bracket), if the next character is not a close bracket, use a space and then try to read the actual JSON value. A close bracket is not a valid JSON value, so a failure occurs at this point.

This seems to be valid JSON (I can make my browser accept it as a valid JavaScript object), so this should be considered an error in the DBXJSON library. You may need to pre-analyze this, use a different JSON library (there are several for Delphi), or find a way to ensure that the information sent to you does not contain this template.

In any case, you should report this to QC as an error.