Detecting that a log file has been deleted or truncated on POSIX systems?

Question

Suppose a lengthy process is written to a log file. Assume that the log file remains open indefinitely. Suppose a careless system administrator deletes this log file. Can a program detect that this happened?

Is it possible to assume that fstat() will report the number of null references for the remote file?

Truncation, it seems to me, is a little more complicated. This partly depends on whether the file descriptor is in O_APPEND mode. If the log file does not work with O_APPEND , then the current position of the log entry of the program log descriptor does not change, and truncation removes the leading bytes, but the program continues to write at the "end", leaving a space from phantom zero bytes (they are considered zeros, but do not necessarily take up space by drive).

If the program works with O_APPEND , then it will write at the end of the file how it exists. The only way to observe truncation is to note that the file position is not where the program was expecting, which in turn means tracking this position explicitly.

In general, truncation as removal does not bother me as much, but any thoughts would be welcome.

Answer 1

Checking that fstat() returns a null reference count will fail if the file is hardlinked or renamed. I would probably instead periodically compare the number of inode stat() with fstat() instead.

I'm not sure about truncation.

tail -F checks for deletion and possibly truncation, so I would check its source to see how it implements it.

Answer 2

Suppose a careless system administrator killed a process. Are you sure you want to protect against an administrator doing random things? I think you're just looking for a way to run a new log file from time to time, for example using logrotate . There, it’s sufficient to provide a way to manually allow the program to reopen the log file. The standard way to do this is to listen to the HUP signal in the program and reopen the log file if it arrives:

 #include <signal.h> volatile int f_sighup; void sighup_handler() { f_sighup = 1; } void trap_sighup() { struct sigaction sa; int rv; memset(&sa, 0, sizeof(struct sigaction)); sa.sa_handler = &sighup_handler; rv = sigaction(SIGHUP, &sa, NULL); if (-1 == rv) { fprintf(stderr, "warning: setting SIGHUP signal handler failed"); } } int main() { f_sighup = 0; trap_sighup(); ... } 

Then check the f_sighup flag f_sighup in the main program to see if the log file should be reopened. This works well with tools like logrotate , which can rename an old log file and then call kill -s HUP $PID . And sloppy sysadmin can do this manually after deleting (or better renaming) the old log file.

Answer 3

You can use inotify to view the log file by tracking it for file system events.

Answer 4

In response to søren-holm's answer

When the file is closed, the change time changes.

which does not look correct:

 import os from time import sleep TMPF = '/tmp/f' def print_stats(): print("%s, %s" % (os.stat(TMPF).st_mtime, os.stat(TMPF).st_ctime)) sleep(1.1) print("Opening...") with open(TMPF, 'w') as f: print_stats() print("Writing...") os.write(f.fileno(), 'apple') print_stats() print("Flushing...") f.flush() print_stats() print("Closing...") print_stats() 

It produces:

 Opening... 1483052647.08, 1483052647.08 Writing... 1483052648.18, 1483052648.18 Flushing... 1483052648.18, 1483052648.18 Closing... 1483052648.18, 1483052648.18 

Admittedly, there is a bit of Python magic there; that write() not guaranteed to be automatically cleared, but the dot means that mtime is updated when the file changes, and not when the file is closed. The behavior of ctime will depend on your file system and its mount options.

Answer 5

When the file is closed, the change time changes. Therefore, periodically check the time in mtime using stat ().