In the shell (bash) How to execute several commands in the pipeline?

Question

I am trying to convey the awk command output through several commands at once in the Bash shell, following my knowledge, I come up with this:

awk '$13 ~ /type/ {print $15}' filename.txt | (wc -l || sort -u) 

I want the result of the awk command to be counted and sorted, how can I do this? Even with && the command does not work, it executes the first command and then exits. I think this is my knowledge of Bash that is failing.

Thanks in advance.

Answer 1

If you want to send the output to two different commands on the same line, you will need to replace the process.

Try the following:

 awk '$13 ~ /type/ {print $15}' filename.txt | tee >(wc -l >&2) | sort -u 

This prints the number of lines to stderr and the sorted result to stdout. If you need the number of lines in stdout, you can do this, leave the value >&2 , but then it will be passed to the sort call and most likely sorted at the top of the output.

EDIT: A revised description of what happens based on further testing.

Answer 2

In this case, do your count in awk, why do you need a pipe? don't complicate

 awk '$13 ~ /type/ {print $15;c++}END{print c} ' filename.txt | sort -u 

Answer 3

If the size of the output is not too large to fit in memory, and you do not need the wc and sort commands to work in parallel for performance reasons, here's a fairly simple solution:

 output=$(awk '$13 ~ /type/ {print $15}' filename.txt; echo a) printf "%s" "${output%a}" | sort -u printf "%s" "${output%a}" | wc -l 

This difficulty with optional a is that the awk command can print several empty lines at the end of the input, which will be broken by the $() construct. You can easily choose which of sort or wc should be displayed first.

---

Here you can work with any POSIX shell (ash, bash, ksh, zsh, ...), but only on systems with /dev/fd (which includes reasonably recent Linux, * BSD and Solaris). Like Walter’s similar construction using the simpler method available in bash, ksh93, and zsh , the wc output and the sort output can be mixed.

 { awk '$13 ~ /type/ {print $15}' filename.txt | tee /dev/fd3 | wc -l } 3>&1 1>&3 | sort -u 

---

If both of you have to deal with an intermediate output that does not fit comfortably into memory and does not want the output of the two commands to be mixed, I do not think that there is an easy way in the POSIX shell, although it should be done using ksh or zsh coprocesses.

Answer 4

I think the main question is: what do you expect from the conclusion?

If you are trying to do two things, do two things:

 awk '$13 ~ /type/ {print $15}' filename.txt > tempfile wc -l < tempfile sort -u < tempfile rm tempfile 

Answer 5

You want to use named pipes created with mkfifo in combination with tee. Example: http://www.softpanorama.org/Tools/tee.shtml