Bash when reading reload value values ​​- using built-in pipeline read

Question

initiate () { read -p "Location(s) to look for .bsp files in? " loc find $loc -name "*.bsp" | while read do if [ -f "$loc.bz2" ] then continue else filcount=$[$filcount+1] bzip $loc fi if [ "$scan" == "1" ]; then bzipint $loc fi echo $filcount #Correct counting echo $zipcount #Correct counting echo $scacount #Correct counting echo $valid #Equal to 1 done echo $filcount #Reset to 0 echo $zipcount #Reset to 0 echo $scacount #Reset to 0 echo $valid #Still equal to 1 } 

I am writing a bash shell script to use bzip2 to .bsp all .bsp files inside a directory. In this script, I have several variables for counting totals (files, successful zip codes, successful integrity checks), however I seem to run into a problem.

When find $loc -name "*.bsp" runs out of files to print while read and while read outputs, it discards $filcount , $zipcount and $scacount (all of which change (increase) inside initiate () , bzip () ( which is called during initiate () ) or bzipint () (which is also called in initiate () ).

To check if this is due to variables changing inside initiate () or other functions accessible from it, I used echo $valid , which is defined outside initiate () (e.g. $filcount , $zipcount , etc.), but does not change from another function inside initiate () or inside initiate () .

Interestingly, $valid does not get reset to 0, like other variables inside initiate.

Can someone tell me why my variables magically get reset when reading outputs?

Answer 1

I ran into this problem yesterday.

The problem is that you are doing find $loc -name "*.bsp" | while read find $loc -name "*.bsp" | while read . Since this is pipe-related, the while read cannot actually work in the same bash process as in the rest of your script; bash should output the subprocess so that it can connect stdout find to the stdin of the while .

This is very smart, but it means that any variables set in the loop cannot be visible after the loop, which completely defeated the whole purpose of the while that I wrote.

You can either try to feed the input into a loop without using a channel, or get out of a loop without using variables. I have finished the terrible abomination associated with writing to a temporary file and I assure the entire cycle in $(...) , for example:

 var="$(producer | while read line; do ... echo "${something}" done)" 

Which got me var, set to all things that were an echo from the loop. I probably messed up the syntax of this example; I do not have the code that I have written at the moment.

Answer 2

if you use bash

 while read do if [ -f "$REPLY.bz2" ] then continue else filcount=$[$filcount+1] bzip $REPLY fi if [ "$scan" == "1" ]; then bzipint $REPLY fi echo $filcount #Correct counting echo $zipcount #Correct counting echo $scacount #Correct counting echo $valid #Equal to 1 done < <(find $loc -name "*.bsp") 

Answer 3

To summarize the parameters for using read at the end of the [conceptual equivalent] pipeline in POSIX-like wrappers:

Recall: in bash, by default and in strictly compatible POSIX shells, all commands in the pipeline are always executed in a subshell , so the variables that they create or modify will not be visible to the current shell (will not exist after the pipeline completes).

The following cover bash , ksh , zsh and sh ([mainly] shells for POSIX functions only, such as dash ) and show ways to avoid creating a subshell to save variables created / modified with read .

^{If the minimum version number is not specified, suppose that even "fairly old" versions are supported (such functions have existed for a long time, but I don’t know exactly when they were introduced.}

^{Note that as an [POSIX-compatible] alternative to the solutions below, you can always capture the output of a command in a [temporary] file and then pass it to read as < file , which also avoids subnets.}

---

ksh and zsh : no workaround / configuration change required at all:

read inline is launched by default in the current shell when used as the last command in the pipeline.

^{Apparently, ksh and zsh by default run any command at the last stage of the pipeline in the current shell.
Observed in ksh 93u+ and zsh 5.0.5 .
If you know specifically in which version this feature was introduced, let me know.}

 #!/usr/bin/env ksh #!/usr/bin/env zsh out= # initialize output variable # Pipe multiple lines to the `while` loop and collect the values in the output variable. printf '%s\n' one two three | while read -r var; do out+="$var/" done echo "$out" # -> 'one/two/three/' 

---

bash 4.2+ : use the lastpipe shell option

In bash version 4.2 or later, enabling the lastpipe wrapper lastpipe makes the last pipeline segment work in the current wrapper, allowing reads to create visible variables for the current wrapper.

 #!/usr/bin/env bash shopt -s lastpipe # bash 4.2+: make the last pipeline command run in *current* shell out= printf '%s\n' one two three | while read -r var; do out+="$var/" done echo "$out" # -> 'one/two/three/' 

---

bash , ksh , zsh : use process overrides

In short, process substitution is a way for command output to act as a temporary file.

 out= while read -r var; do out+="$var/" done < <(printf '%s\n' one two three) # <(...) is the process substitution echo "$out" # -> 'one/two/three' 

---

bash , ksh , zsh : use here is a line with command replacement

 out= while read -r var; do out+="$var/" done <<< "$(printf '%s\n' one two three)" # <<< is the here-string operator echo "$out" # -> 'one/two/three' 

Note the need for double quotation marks for command substitution to protect its output from shell extensions .

---

POSIX-compatible solution ( sh ): use a document here with command substitution

 #!/bin/sh out= while read -r var; do out="$out$var/" done <<EOF # <<EOF ... EOF is the here-doc $(printf '%s\n' one two three) EOF echo "$out" # -> 'one/two/three' 

Note that by default you need to put the trailing delimiter - EOF , in this case - at the very beginning of the line and that no characters should follow it.