How to process every line received as a result of grep command

Question

I have a number of lines extracted from a file after running the grep command as follows:

var=`grep xyz abc.txt` 

Suppose I got 10 lines that consist of xyz as a result.

Now I need to process each line received as a result of the grep command. How to do it?

Answer 1

One of the easiest ways is not to store the output in a variable, but to iterate over it directly using the while / read loop.

Something like:

 grep xyz abc.txt | while read -r line ; do echo "Processing $line" # your code goes here done 

There are various options for this scheme depending on what you are after.

If you need to change the variables inside the loop (and so that the change can be seen outside of it), you can use the replacement process, as indicated in answer.fonorqui :

 while read -r line ; do echo "Processing $line" # your code goes here done < <(grep xyz abc.txt) 

Answer 2

You can do the following while read , which will be passed as the result of the grep using the so-called process replacement:

 while IFS= read -r result do #whatever with value $result done < <(grep "xyz" abc.txt) 

Thus, you do not need to save the result in a variable, but directly “enter” its output into the loop.

---

Pay attention to using IFS= and read -r as recommended in BashFAQ / 001: How can I read a file (data stream, variable) in turn (and / or field by field)? :

The -r read option prevents the backslash from being interpreted (usually used as a newline backslash pair to extend multiple lines or avoid delimiters). Without this option, any irreplaceable backslashes at the input will be discarded. You should almost always use -r with reading.

In the scenario above, IFS = prevents trimming of leading and trailing whitespace. Remove it if you want this effect.

As for replacing the process, this is explained on the hackers bash page:

Process substitution is a form of redirection where the input or output of a process (some sequence of commands) is displayed as a temporary file.

Answer 3

I would suggest using awk instead of grep + for something else.

awk '$0~/xyz/{ //your code goes here}' abc.txt

Answer 4

Often the processing order does not matter. In parallel with GNU for this situation:

 grep xyz abc.txt | parallel echo do stuff to {} 

If you process more like:

 grep xyz abc.txt | myprogram_reading_from_stdin 

and myprogram slower than you can run:

 grep xyz abc.txt | parallel --pipe myprogram_reading_from_stdin