Respect the last line if it does not end with a new line char (\ n) when using read

Question

For a while, I noticed that read never reads the last line of a file unless there is a newline at the end of it. This is understandable when you consider that as long as there is no “new line” character in the file, it seems to contain 0 lines (which is rather difficult to recognize!). See, for example, the following:

 $ echo 'foo' > bar ; wc -l bar 1 bar 

But...

 $ echo -n 'bar' > foo ; wc -l foo 0 foo 

The question is: how can I handle such situations when using read to process files that were not created or changed by me, and which I don’t know about, do they really end with a "new line", a symbol?

Answer 1

read , in fact, reads an unused line into the assigned var ( $REPLY by default). It also returns false on such a line, which simply means "the end of the file; directly using its return value in the classic while skips this last line. If you slightly change the logic of the cycle, you can process files not related to the new line, correctly, without the need for prior sanitation, with read :

 while read -r || [[ -n "$REPLY" ]]; do # your processing of $REPLY here done < "/path/to/file" 

Please note that this is much faster than solutions that rely on external ones.

hint for a hat Gordon Davisson to improve loop logic.

Answer 2

POSIX requires that any line in the file has a newline at the end to indicate that it is a line. But this site offers a solution for exactly the scenario that you are describing. The end product is this chunklet.

 newline=' ' lastline=$(tail -n 1 file; echo x); lastline=${lastline%x} [ "${lastline#"${lastline%?}"}" != "$newline" ] && echo >> file # Now file is sane; do our normal processing here... 

Answer 3

If you must use reading, try the following:

 awk '{ print $0}' foo | while read line; do echo the line is $line done 

since awk seems to recognize strings even without a new char string

Answer 4

This is more or less a combination of the answers received so far.

It does not modify files in place.

 (cat file; tail -c1 file | grep -qx . && echo) | while read line do ... done 

Answer 5

Shorter bash way :

To use the entire file without changing it:

 cat file <(echo) | wc -l 

or

 wc -l < <(cat file <(echo) ) 

To verify the correct completion of the file and fix it:

 [ $(tail -n +$(( $(wc -l <$file) +1)) <$file | wc -c) -gt 0 ] && echo >>$file 

Explanation: Skip the number of lines from the $ file than leading characters should be considered: must be 0 if the last line is complete.

Another approach:

 [ $(tail -c1 $file | wc -l) -eq 0 ] && echo >>$file 

Eplained: find the last character in the file, if it is not a new line, linecount from this will be 0.