Reasonably executable hex decoding in native bash?

Question

I have a 2 MB file, which is a sequence of hexadecimal values ​​separated by spaces. For instance:

3F 41 56 00 00

Easy peasy for this in bash:

cat hex.txt | tr -s " " $'\n' | while read a; do 
    echo $a | xxd -r -p | tee -a ascii
done

or

f=$(cat hex.txt)
for a in $f; do
    echo $a | xxd -r -p | tee -a ascii
done

Both are painfully slow.

I hacked into a C program that converted the file in about two seconds, and later realized that I could just do this:

cat hex.txt | xxd -r -p

Since I already converted the file and found the optimal solution, my question is not about the conversion process itself, but about how to optimize my first two attempts, as if the third were impossible. Is there anything that could be done to speed up these single-liners or is this Bash too slow for that?

Answer 1

, , xxd tee, .

printf , tee:

tr -s " " '\n' < hex.txt | 
while read seq; do printf "\x$seq"; done |
tee -a ascii 

( -a tee).

( , ruby - awk:

tr -s " " '\n' < hex.txt |  ruby -pe '$_ = $_.to_i(16).chr'

( , bash). )

Answer 2

: , awk:

# BSD/OSX awk
xargs printf '0x%s ' < hex.txt | awk -v RS=' ' '{ printf "%c", $0 }' > ascii

# GNU awk; option -n needed to support hex. numbers
xargs printf '0x%s ' < hex.txt | awk -n -v RS=' ' '{ printf "%c", $0 }' > ascii

# mawk - sadly, printf "%c" only works with letters and numbers if the input is *hex*
awk  -v RS=' ' '{ printf "%c", int(sprintf("%d", "0x" $0)) }' < hex.txt

2 - iMac 2012 Intel Core i5 3,2 Fusion Drive, OSX 10.10.3, :

• BSD/OSX awk: ca. 1s
• GNU awk: ca. 0.6s
• mawk: ca. 0.5s

PSkocik optimized- bash -loop solution: ca. 11s

^{, mawk, , , awk, . , -n GNU awk, : awk $([[ $(gawk --version 2>/dev/null) = GNU* ]] && printf %s -n) -v RS=' ' '{ printf "%c", int(sprintf("%d", "0x" $0)) }' < hex.txt}

- bash :

• xargs printf '0x%s ' < hex.txt hex.txt 0x, awk hexadecimals.
  • , , xargs stdin , , getconf ARG_MAX - , xargs , , .
• awk -v RS=' ' '{ printf "%c", $0 }'
  • awk -v RS=' ' , , .. . value -
  • printf "%c", $0 ASCII- printf.

---

:

• Bash .
• , .
  ^{. .}

, bash .

Answer 3

Well, you can remove the first one catand replace it with tr < hex.txt. Then you can also create a static conversion table and drop echoand xxd. But the cycle will still be slow, and you cannot get rid of it, I think.