Geek Answers Handbook

Using pyinotify to keep track of file creation, but expecting it to be completely written to disk

I use pyinotify to view a folder when files are created in it. And when certain files are created, I want to move them. The problem is that as soon as the file is created (obviously), my program tries to move it, even before it is completely written to disk.

Is there any way to make pyinotify until the file is completely written to disk before telling me that it was created? Or is there any simple way, after I received the notification, to make python wait to move it until it is written?

+7
python linux file pyinotify
source share
3 answers

Have pyinotify respond to IN_CLOSE_WRITE events:

wm.add_watch(watched_dir, pyinotify.IN_CLOSE_WRITE, proc_fun=MyProcessEvent())

This is from man 5 incrontab , but it is equally suitable for pyinotify:

  IN_ACCESS File was accessed (read) (*) IN_ATTRIB Metadata changed (permissions, timestamps, extended attributes, etc.) (*) IN_CLOSE_WRITE File opened for writing was closed (*) IN_CLOSE_NOWRITE File not opened for writing was closed (*) IN_CREATE File/directory created in watched directory (*) IN_DELETE File/directory deleted from watched directory (*) IN_DELETE_SELF Watched file/directory was itself deleted IN_MODIFY File was modified (*) IN_MOVE_SELF Watched file/directory was itself moved IN_MOVED_FROM File moved out of watched directory (*) IN_MOVED_TO File moved into watched directory (*) IN_OPEN File was opened (*)
+14
source share

It is hard to say at this level if the file is being written. What you can do is to check if the file is being opened by any other process.

1) Of the various flags that are used when opening a file, the O_EXLOCK flag can help. If the O_EXLOCK flag is set, the file descriptor has an exclusive lock on the file. Therefore, I understand that if you can do os.open () with the O_EXLOCK flag, it is not opened by another process. This should work on all posix compatible OS, but I have not tested them. If the file is open, you can close, wait and try again.

2) You can also try os.stat and see a change in the timestamp and try to interpret the information safely. Although this is not stupid evidence.

3) On Unix systems, you can try "lsof"

4) The next page describes the use of symbolic links from / proc / PID / fd to check open files

[Edit: links updated]

+1
source share

If you have control over the recording process, you can call the file "foo.part" while it is being written and rename it to "foo" when it was closed.

+1
source share

More articles:


All Articles