Bash: Iterate over JSON array elements selected by index

Question

I use jq to parse a JSON file, extracting each JSON array from a series into a shell array.

My current code is as follows:

 for ((i = 0; i < ${#nvars[@]}; i++)); do v1=($(cat $INPUT | jq '."config"[i]."var1"[]')) echo $v1 done 

error message:

 error: i is not defined 

I also replaced

 v1=($(cat $INPUT | jq '."config"[i]."var1"[]')) 

from

 v1=($(cat $INPUT | jq '."config"[$i]."var1"[]')) 

still not working. Any ideas? Any help is appreciated!

---

Edit: sample input

 { "config-vars":[ { "var1":["v1","v2"], "var2":"" }, { "var1":["v3",""], "var2":"v4" } ] } 

Answer 1

There is little room for improvement. Start here:

 v1=($(cat $INPUT | jq '."config"[$i]."var1"[]')) 

... firstly, you really don't need to use cat ; this slows down your performance because it forces jq to read from the pipe, not from your input file directly. Just running jq <"$INPUT" would be more reliable (or better, <"$input" to avoid using all uppercase names, which are reserved by convention for the shell's built-in shells and environment variables).

Secondly, you need to quote all variable extensions, including the extension of the input file name, otherwise you will get errors when your file name contains spaces.

Thirdly, array=( $(stuff) ) splits the output of stuff into all IFS characters and extends the results of this splitting as a series of glob expressions (so if the output contains *.txt and you run this script in a directory containing text files, you get the names of these files in the result array). Dividing into new lines would only mean that you could parse verbose lines correctly, and disabling the glob extension is necessary before you can reliably use this technique in the presence of glob characters. One way to do this is to set IFS=$'\n' and run set -h before running this command; the other is to redirect the output of your command to the while read (shown below).

Fourth, replacing a string with code is bad practice in any language - thus (local equivalents) of Bobby Tables , allowing someone who was only supposed to change the data passed into your process to provide content that is processed as executable code ( although in this case, like a jq script, which is less dangerous than arbitrary execution of code in a more fully functional language, but this may allow you to add additional data to the output).

Further, as soon as you get jq to release content separated by a newline, you do not need to read it into an array: you can jq over the contents as you write it with jq and read your shell, thereby preventing the shell from having to allocate memory for buffering of this content:

 while IFS= read -r; do echo "read content from jq: $REPLY" done < <(jq -r --arg i "$i" '.config[$i | tonumber].var1[]' <"$input") 

Finally, let's say you want to work with an array. There are two ways to do this to avoid pitfalls. One of them is to explicitly set IFS and disable the glob extension before destination:

 IFS=$'\n' # split only on newlines set -f result=( $(jq -r ... <"$input") ) 

Another is to assign a loop to your array:

 result=( ) while IFS= read -r; do result+=( "$REPLY" ) done < <(jq -r ... <"$input") 

... or, as @JohnKugelman suggested, use read -a to read the entire array in one operation:

 IFS=$'\n' read -r -d '' -a result < <(jq -r ... <"$input") 

Answer 2

Variables are not interpolated inside single quotes. Instead, use double quotes and remove existing quotes.

 v1=($(cat $INPUT | jq ".config[$i].var1[]")) 

Or use the --arg , and then you can use single quotes.

 v1=($(cat $INPUT | jq --arg i "$i" '.config[$i].var1[]')) 

You can also fix the useless use of cat:

 v1=($(jq ".config[$i].var1[]" "$INPUT")) 

Also see @CharlesDuffy's comment for a detailed explanation of why assigning to an array as this is unsafe.

Answer 3

jq is able to retrieve the structure in one pass, so the whole loop is redundant. If the input JSON contains more entries than you have values ​​in nvars , use the index to grind.

 jq -r '."config-vars"[]."var1"' "$INPUT" | head -n "${#nvars[@]}" # If you need just the #nvars first values 

Answer 4

If you already saved the result of some JSON in a variable named $ MY_VAR:

 while IFS= read -r; do echo "$REPLY" done < <(echo $MY_VAR | jq -r '.[]') 

It took me too long to figure this out. All the examples that I saw were collapsed, and I had to put it together.