'grep -q' does not exit with 'tail -f'

Question

I am trying to implement scripts that wait for a specific message in a log file. After the message is logged, I want to continue the script.

Here is what I am trying to use with tail -f and grep -q :

 # tail -f logfile | grep -q 'Message to continue' 

grep never leaves, and so it waits forever, even if the message "Continue" is entered in the file.

When I run it without -f , it works fine.

Answer 1

tail -f will read the file and the displayed lines added later, it will not be completed (if no signal is sent, for example SIGTERM ). grep is not a blocking part here, tail -f is. grep will read from the pipe until it is closed, but this never happens because tail -f does not exit and does not close the pipe.

---

A solution to your problem will probably be (not tested and most likely not working well):

 tail -f logfile | while read line; do echo $line | grep -q 'find me to quit' && break; done 

Answer 2

After some experimentation, I believe the problem is that bash expects all processes in the pipeline to terminate in some form or form.

With a simple qqq file of approximately 360 lines of C source (several programs concatenated several times), and using grep -q return, I observe:

• tail -n 300 qqq | grep -q return tail -n 300 qqq | grep -q return exits almost immediately.
• tail -n 300 -f qqq | grep -q return tail -n 300 -f qqq | grep -q return does not exit.

• tail -n 300 -f qqq | strace -o grep.strace -q return tail -n 300 -f qqq | strace -o grep.strace -q return does not exit until interrupted. The grep.strace file ends with:

   read(0, "#else\n#define _XOPEN_SOURCE 500\n"..., 32768) = 10152 close(1) = 0 exit_group(0) = ? 

  This one leads me to think that grep came out before the interrupt kills tail ; if he was waiting for something, there would be an indication that he received a signal.

• A simple program that mimics the actions of a shell, but without waiting, indicates that everything is ending.

   #define _XOPEN_SOURCE 600 #include <stdlib.h> #include <unistd.h> #include <stdarg.h> #include <errno.h> #include <string.h> #include <stdio.h> static void err_error(const char *fmt, ...) { int errnum = errno; va_list args; va_start(args, fmt); vfprintf(stderr, fmt, args); va_end(args); if (errnum != 0) fprintf(stderr, "%d: %s\n", errnum, strerror(errnum)); exit(1); } int main(void) { int p[2]; if (pipe(p) != 0) err_error("Failed to create pipe\n"); pid_t pid; if ((pid = fork()) < 0) err_error("Failed to fork\n"); else if (pid == 0) { char *tail[] = { "tail", "-f", "-n", "300", "qqq", 0 }; dup2(p[1], 1); close(p[0]); close(p[1]); execvp(tail[0], tail); err_error("Failed to exec tail command"); } else { char *grep[] = { "grep", "-q", "return", 0 }; dup2(p[0], 0); close(p[0]); close(p[1]); execvp(grep[0], grep); err_error("Failed to exec grep command"); } err_error("This can't happen!\n"); return -1; } 

  With a fixed-size file, tail -f not going to exit, so the shell ( bash ) seems to be hanging around.

• tail -n 300 -f qqq | grep -q return tail -n 300 -f qqq | grep -q return freezes, but when I used another terminal to add another 300 lines to the qqq file, the command came out. I interpret this as happening because grep exited, so when tail wrote new data to the pipe, it received SIGPIPE and exited, and bash therefore acknowledged that all the processes in the pipeline were dead.

I observed the same behavior with both ksh and bash . This suggests that this is not a mistake, but some expected behavior. Testing on Linux (RHEL 5) on x86_64 machine.

Answer 3

 tail -f logfile | grep --max-count=1 -q 'Message to continue' 

Admittedly, it exits when the next line is read, and not immediately on the agreed one.

Answer 4

This is because tail with the -f (follow) option does not exit and continues to provide grep output. Waiting for lines in a log file is likely to be easier with perl / python.

Run tail -f using the Python subprocess module. Read the output from tail in a loop until you see the lines you need, and then exit the Python script. Put this solution inside your shell script.

The Python script blocks the shell of the script until the required lines are visible.

Answer 5

I thought that I would send this as an answer, because it explains why the command exits after the second write to the file:

 touch xxx tail -f xxx | grep -q 'Stop' ps -ef |grep 'grep -q' # the grep process is there echo "Stop" >> xxx ps -ef|grep 'grep -q' # the grep process actually DID exit printf "\n" >> xxx # the tail process exits, probably because it receives a signal when it # tries to write to a closed pipe 

Answer 6

I was looking for the answer to this question for my own project. Trying to check exactly when the one passed through the GPU is active on the VMware ESXi VM. Many variations of the same question are everywhere. This is quite recent. I figured out a way to trick him, and if you can live with your interesting line repeated in the magazine, then:

tail -n 1 -f / var / log / vmkernel.log | grep -m 1 IOMMUIntel → / var / log / vmkernel.log

This is the log header, one line at a time, grep checks each line for the first appearance and adds it to the log, and then the tail immediately exits.

If you like VMware passthough cracker, read here: http://hackaday.io/project/1071-the-hydra-multiheaded-virtual-computer