How can I completely sort arbitrary JSON using jq?

Question

I want to split two JSON text files. Unfortunately, they are built in random order, so I get diff when they are semantically identical. I would like to use jq (or something else) to sort them in any order in order to eliminate differences related only to the ordering of the elements.

- sort keys solve half the problem, but don't sort arrays.

I am pretty versed in jq and don’t know how to write a recursive jq filter that saves all the data; Any help would be appreciated.

I understand that line output of "diff" is not always the best way to compare two complex objects, but in this case I know that the two files are very similar (almost identical) and in turn diffs are good for my purposes.

Using jq or alternative command line tools to delimit JSON files answers a very similar question, but does not print the difference. Also, I want to save the sorted results, so what I really want is just a filtering program to sort JSON.

Answer 1

Here is a solution using the generic function sorted_walk / 1 (named for the reason described in the postscript below).

normalize.jq:

# Apply f to composite entities recursively using keys[], and to atoms def sorted_walk(f): . as $in | if type == "object" then reduce keys[] as $key ( {}; . + { ($key): ($in[$key] | sorted_walk(f)) } ) | f elif type == "array" then map( sorted_walk(f) ) | f else f end; def normalize: sorted_walk(if type == "array" then sort else . end); normalize 

An example of using bash:

 diff <(jq -S -f normalize.jq FILE1) <(jq -S -f normalize.jq FILE2) 

POSTSCRIPT: The built-in definition of walk/1 was revised after this answer was first published: it now uses keys_unsorted , not keys .

Answer 2

I want to split two JSON text files.

Use jd with the -set option:

No conclusion means a difference.

 $ jd -set A.json B.json 

Differences are displayed as the path @ and + or -.

 $ jd -set A.json C.json @ ["People",{}] + "Carla" 

Output differentials can also be used as patch files with the -p option.

 $ jd -set -o patch A.json C.json; jd -set -p patch B.json {"City":"Boston","People":["John","Carla","Bryan"],"State":"MA"} 

https://github.com/josephburnett/jd#command-line-usage