Apply formatting to unix shell

Question

I have been looking at several server logs using tail -f recently, and thought it would be much easier to see some things if I could format the output. In fact, all I'm looking for is a way, perhaps, to color certain words (defined by a regular expression) and, possibly, delete certain words (again, defined by a regular expression).

I know there are programs that visualize server logs in real time and much more, but I'm more interested in this.

Answer 1

Output tail -f to sed and add escape codes to the ANSI code . For example, the following will colorize all the numbers in red (color 31) and all the quoted lines in bright yellow (color 93):

 RED=`echo -en '\e[31m'` YELLOW=`echo -en '\e[93m'` RESET=`echo -en '\e[00m'` tail -f file | sed -E "s/([0-9]+)/$RED\1$RESET/g;s/(\"[^\"]*\")/$YELLOW\1$RESET/g" 

Answer 2

I think you are looking for a kind of sed script that will surround the words that you select with the ANSI Color escape sequences. (hmm ... lemme look).

EDIT OK, it turned out:

Here is an example of the output of "Hello" in dark red:

 echo -e "\033[31mHello\033[0m" 

What's happening? First of all, I use echo -e so that echo does not convert slashes to screen slashes, but rather reads the \033 escape sequence as a single escaped character. Actually it is only 33 octal, which is 27 (serial number of the ESC key).

So what really gets sent to the screen is something like:

 <ESC>[32mHello<ESC>[0m 

Which ANSI screen is interpreted as "first run the 32m command, print" Hello ", and then run the 0m command.

In this case, the 32m command means “set the background color to 2”, and since color No. 2 is dark red, the “feather” used by the terminal will now be dark red. This means that from now on all the text that will be displayed on the screen will be dark red.

When we finish the output of the text, which should be red, we want to reset the color, so we call the 0m command, which resets the colors to normal.

For a list of all escape codes, search in [ http://en.wikipedia.org/wiki/ANSI_escape_code Wikipedia] or just go to Google.

So all you need to do a sed script is to replace the words you have chosen with words surrounded by colors. For example, to replace the word "Feb" in /var/log/messages , follow these steps:

 tail /var/log/messages | sed -e "s/Feb/\\o033[31m&\\o033[0m/" 

(You may need to do this as root to actually read /var/log/messages )

All sed did was search for the word “Feb” and its surroundings with the same escape sequence as we used above.

You can expand it to color a few words:

 tail /var/log/messages | sed -e "s/\(Feb\|Mar\|Apr\)/\\o033[31m&\\o033[0m/g" 

What will be the color of "Feb", "Mar", "Apr" - each in dark red.

Hope this gives you some insight into how to do what you need!

Answer 3

Thanks to both scraimer and Adam, I came up with quite a lot that I needed, and I thought that I would share it here for someone else:

 RED=`echo -en '\e[31m'` YELLOW=`echo -en '\e[93m'` RESET=`echo -en '\e[00m'` # line breaks here are just for formatting tail -f ~/access-logs/access.log | sed -e " s@ \([0-9\.]\+\) - - \[[0-9]\+/[a-zA-Z]\+/[0-9]\+:\([0-9]\+:[0-9]\+:[0-9]\+\) +[0-9]\+] \" \(.\+\) HTTP/1\.[01]\" \([0-9]\+\) \([-0-9]\+\) \"\([^\"]\+\)\".* @\n\2 $YELLOW\4$RESET (\1)\n$RED\3$RESET\nBytes: \5\nFrom: \ 6@g " # here the full line if you wanted to copy it # tail -f ~/access-logs/access.log | sed -e " s@ \([0-9\.]\+\) - - \[[0-9]\+/[a-zA-Z]\+/[0-9]\+:\([0-9]\+:[0-9]\+:[0-9]\+\) +[0-9]\+] \"\(.\+\) HTTP/1\.[01]\" \([0-9]\+\) \([-0-9]\+\) \"\([^\"]\+\)\".*@\n\2 $YELLOW\4$RESET (\1)\n$RED\3$RESET\nBytes: \5\nFrom: \ 6@g " 

What does it mean:

 182.108.1.20 - - [04/Feb/2009:23:24:41 +1000] "GET /images/exam_room.jpg HTTP/1.0" 200 8559 "http://www.myserver.com/courses/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" 

In this (but with some color):

 23:24:41 200 (182.108.1.20) GET /images/exam_room.jpg Bytes: 8559 From: http://www.myserver.com/courses/