How to get file close event in python

Question

Using python 2.7 on a 64 bit Windows 7 machine.

How to get file close event:

• when a file is opened in a new process of opening a file (for example, notepad, a text panel that opens a file each time in a new text panel process).
• when a file is opened on the tab of the file opener (for example, notepad ++, which opens all the files on a new tab, but there is only one process of working with notepad ++)

So, how to get the file close event in the above cases? Is it possible to achieve the above cases with common code? I deal with different types of files

Answer 1

This turned out to be a very simple task for * nix systems, but on Windows, receiving a file close event is not an easy task. The following is a summary of common methods grouped by OS.

For linux

On Linux, changes to the file system can be easily tracked and in great detail. The best tool for this is a kernel function called inotify , and there is a Python implementation that uses it, called Pynotify.

• Pynotify

  Pyinotify is a Python module for monitoring file system changes. Pyinotify relies on a Linux kernel function (integrated in the 2.6.13 kernel) called inotify , which is an event driven notifier. Its notifications are exported from kernel space to user space through three system calls. Pyinotify binds these system calls and provides an implementation on top of them, offering a general and abstract way to manage these functions.

  Here you can find a list of events that can be tracked using Pynotify .

  Using an example:

  import pyinotify

   class EventHandler(pyinotify.ProcessEvent): def process_IN_CLOSE_NOWRITE(self, event): print "File was closed without writing: " + event.pathname def process_IN_CLOSE_WRITE(self, event): print "File was closed with writing: " + event.pathname def watch(filename): wm = pyinotify.WatchManager() mask = pyinotify.IN_CLOSE_NOWRITE | pyinotify.IN_CLOSE_WRITE wm.add_watch(filename, mask) eh = EventHandler() notifier = pyinotify.Notifier(wm, eh) notifier.loop() if __name__ == '__main__': watch('/path/to/file') 

For windows

The situation for Windows is rather complicated than for Linux. Most libraries rely on the ReadDirectoryChanges API, which is limited and cannot detect finer details, such as a file close event. However, there are other ways to detect such events, so read on to find out more.

• Watcher

  Note: Watcher was last updated in February 2011, so it could probably be skipped.

  Watcher is a low-level C extension for receiving file system updates using the ReadDirectoryChangesW API on Windows systems. The package also includes a high-level interface for emulating most of the API.NET FileSystemWatcher .
  The closest one can be file closure detection using Watcher to monitor FILE_NOTIFY_CHANGE_LAST_WRITE and / or FILE_NOTIFY_CHANGE_LAST_ACCESS .

  Using an example:

   import watcher w = watcher.Watcher(dir, callback) w.flags = watcher.FILE_NOTIFY_CHANGE_LAST_WRITE w.start() 

• Watchdog

  Python APIs and shell utilities for monitoring file system events. Easy to install: $ pip install watchdog . For more information, visit the documentation .
  Watchdog on Windows relies on the ReadDirectoryChangesW API, which provides its reservations, as with Watcher and other libraries based on the same API.

• Pywatch

  Python is almost a clone of the Linux watch team. The pywatch.watcher.Watcher class can be specified to view a set of files and given a set of commands to run when any of these files change. It can only track the file change event because it relies on polling stat st_mtime .

Bonus for Windows with NTFS:

• NTFS USN Magazine

  NTN USN (Update Sequence Number) is an NTFS feature that supports recording changes made to a volume. The reason it is listed as a bonus is that, unlike other entries, this is not a specific library, but rather a function that exists in the NTFS system. Therefore, if you use other Windows file systems (e.g. FAT, ReFS, etc.), this is not applicable.
  How it works, the system records all changes made to the volume in the USN Journal file, each volume having its own instance. Each change log entry contains a USN, file name, and information about what happened.

  The main reason this method is interesting for this question is because, unlike most other methods, this method allows you to detect the event of closing a file defined as USN_REASON_CLOSE . More information with a complete list of events can be found in this MSDN article . For complete documentation on USN Journaling, visit the ntfsjournal page .

The "right" way for Windows:

• File system filter driver

  As described on the MSDN page:

  The file system filter driver is an optional driver that adds value to or changes the behavior of the file system. The file system filter driver is a kernel mode component that runs as part of the Windows executive. A file system filter driver can filter I / O for one or more file systems or file system volumes. Depending on the nature of the filter, the filter may mean logging, observing, changing, or even preventing. Typical applications for file system filter drivers include antivirus utilities, encryption programs, and hierarchical system storage management.

  It is not an easy task to create a file system filter driver, but for those who would like to try, there is a good introduction to the CodeProject tutorial.

  PS Check out @ ixe013 answer for more information on this method.

Multiplatform

• Qt QFileSystemWatcher

  The QFileSystemWatcher class provides an interface for monitoring files and directories for modifications. This class was introduced in Qt 4.2 .
  Unfortunately, its functionality is quite limited, as it can only determine when a file has been modified, renamed or deleted, and also when a new file has been added to the directory.

  Using an example:

   import sys from PyQt4 import QtCore def directory_changed(path): print('Directory Changed: %s' % path) def file_changed(path): print('File Changed: %s' % path) app = QtCore.QCoreApplication(sys.argv) paths = ['/path/to/file'] fs_watcher = QtCore.QFileSystemWatcher(paths) fs_watcher.directoryChanged.connect(directory_changed) fs_watcher.fileChanged.connect(file_changed) app.exec_() 

Answer 2

The problem you are facing is not with Python, but with Windows. This can be done, but you will have to write non-trival C / C ++ code for it.

A file or file closing notification opens in user mode in Windows. Therefore, libraries offered by others do not have a notification about closing the file. The Windows API for detecting changes in userland is ReadDirectoryChangesW . He will warn you of one of the following notifications :

• FILE_ACTION_ADDED if the file was added to the directory.
• FILE_ACTION_REMOVED if the file was deleted from the directory.
• FILE_ACTION_MODIFIED if the file was modified. This may be a change in timestamp or attributes.
• FILE_ACTION_RENAMED_OLD_NAME if the file has been renamed, and this is the old name.
• FILE_ACTION_RENAMED_NEW_NAME if the file has been renamed, and this is a new name.

No amount of Python can change what Windows provides you.

To receive notification of file closure, tools such as Process Monitor , install the Minifilter, which lives in the kernel , next to other filters, such as EFS.

To achieve what you want, you need to:

• Install Minifilter, which has code to send events back to userland. Use the Microsoft Minispy sample , it is stable and fast.
• Convert the code from the user program to make it a Python extension ( minispy.pyd ), which provides a generator that generates events. This is the difficult part, I will return to this.
• You will need to filter events, you will not accept the amount of input / output in an unoccupied Windows window!
• Your Python program can then import the extension and execute it.

Everything looks something like this:

Of course, you can have EFS via NTFS, this is just to show that your minifilter would be above everything.

Hard parts:

• Your minifilter must be digitally signed by an authority trusted by Microsoft. Verification comes to mind, but there are others.
• Debugging requires a separate (virtual) machine, but you can easily customize your interface.
• You will need to install a mini-filter with an account with administrator rights. Any user will be able to read events.
• You will have to deal with multiple users. For many users, there is only one mini-filter.
• You will need to convert the user program from the MiniSpy sample to a DLL that you will wrap with the Python extension.

The last two are the most difficult.

Answer 3

You can use Pyfanotyfi or butter .

I think you will find this link very useful: Linux file system events with C, Python and Ruby

Here you will find an example of exactly how you want (using pyinotify ), this is the code:

 import pyinotify DIR_TO_WATCH="/tmp/notify-dir" FILE_TO_WATCH="/tmp/notify-dir/notify-file.txt" wm = pyinotify.WatchManager() dir_events = pyinotify.IN_DELETE | pyinotify.IN_CREATE file_events = pyinotify.IN_OPEN | pyinotify.IN_CLOSE_WRITE | pyinotify.IN_CLOSE_NOWRITE class EventHandler(pyinotify.ProcessEvent): def process_IN_DELETE(self, event): print("File %s was deleted" % event.pathname) #python 3 style print function def process_IN_CREATE(self, event): print("File %s was created" % event.pathname) def process_IN_OPEN(self, event): print("File %s was opened" % event.pathname) def process_IN_CLOSE_WRITE(self, event): print("File %s was closed after writing" % event.pathname) def process_IN_CLOSE_NOWRITE(self, event): print("File %s was closed after reading" % event.pathname) event_handler = EventHandler() notifier = pyinotify.Notifier(wm, event_handler) wm.add_watch(DIR_TO_WATCH, dir_events) wm.add_watch(FILE_TO_WATCH, file_events) notifier.loop() 

Answer 4

I did not find a package that captures open and close events on Windows. As others have noted, pyinotify is a great option for Linux-based operating systems.

Since I could not observe the closed event, I decided to conduct a modified event. This is a very "type of solution" (i.e. I cannot stop until I see that the file is closed). But it works surprisingly well.

I used the watchdog package. Below is the code from the example and monitors the current directory if you do not pass the path on the command line, otherwise it keeps track of the path you go through.

Call example: python test.py or python test.py C:\Users\Administrator\Desktop

 import sys import time import logging from watchdog.observers import Observer from watchdog.events import LoggingEventHandler if __name__ == "__main__": logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(message)s', datefmt='%Y-%m-%d %H:%M:%S') path = sys.argv[1] if len(sys.argv) > 1 else '.' event_handler = LoggingEventHandler() observer = Observer() observer.schedule(event_handler, path, recursive=True) observer.start() try: while True: time.sleep(1) except KeyboardInterrupt: observer.stop() observer.join() 

This code will show you when files are created, modified, deleted, or renamed / moved. You can filter by simply changing by observing the on_modified event .