Linux I / O Error Implementation

Question

I have a Python and C application on Linux that should correctly handle I / O errors when reading files from disk. The bulk of the application is written in Python with a C extension that makes IO. Inside this extension, I / O errors are detected.

There are two cases of errors for me.

• Missing file.
• The file is larger on disk (using stat ) than can be read with fread .

I can check and handle case number 1 quite easily. However, I would also like to write unit test for case 2. However, I do not know how to cause a “false” I / O error for the test. Is it possible? Is there a better approach to testing this kind of error?

Answer 1

As far as I understand, the TDD classic warns us against writing mocks / stubs for third-party interfaces (including the standard library), see, for example, here . The main problem is that there is usually a gap between the application code and the universal third-party library, which is difficult to associate with mock objects. In addition, this prevents the use of tests to get design problems.

(Despite the fact that in your case, the C library is not a completely third party, unit testing means that you test the objects separately).

The idea is that instead you write an adapter class that encapsulates all of the low-level logic and provides an interface that is close to what your application needs (and, for example, raises more significant exceptions, such as FileIsTooBig ). Then you write mock objects in terms of your domain. As for testing the adapter itself, it tested several simple system tests.

Answer 2

errno (3) is set to EIO only for

  EIO Input/output error (POSIX.1) 

also according to read (2) for:

  EIO I/O error. This will happen for example when the process is in a background process group, tries to read from its controlling terminal, and either it is ignoring or blocking SIGTTIN or its process group is orphaned. It may also occur when there is a low-level I/O error while reading from a disk or tape. 

and in accordance with write (2) for:

  EIO A low-level I/O error occurred while modifying the inode. 

Thus, simulating this particular error code may be difficult; note that there are other system calls for I / O, especially writev (2) and (indirectly) mmap (2) , but read(2) and write(2) are the most common.

Note that file systems and the Linux kernel (for example, its VFS level ) cache data. You can get EIO much later or never. See sync (2) and fsync (2)

However, as a rule, most programs do not specifically handle EIO wrt other error codes; you are probably testing enough by getting another error code, like for example.

  EDQUOT The user quota of disk blocks on the filesystem containing the file referred to by fd has been exhausted. 

So, you will probably experience enough by limiting disk quotas (see quotactl (2) , setquota (8) , etc.) and file space (see setrlimit (2) with RLIMIT_FSIZE , prlimit (1) , ulimit built-in bash ( 1) etc.)

If you really want to fake EIO , you can physically damage the device (or maybe just unplug the USB drive at the wrong time) or write your own File System in User Space (FUSE) that simulates it. I don’t think it’s worth the effort (because when something gets EIO , the whole computer becomes unusable very quickly, and the user will notice it anyway ... and because most software tools handle all error codes in the same way - except EINTR )

In part C of your code, you can use strerror (3) (with syslog (3) ) and / or perror (3) . I'm not sure it is worth trying to handle EIO completely different way from most other errors.

NB: many critical domains have standards that determine how errors should be handled, and code must be developed and tested, for example. ISO26262 in automotive or DO-178B in avionics. Follow your domain standards.

Answer 3

Use fusepy .

fusepy is a python layer on top of FUSE that allows file systems to be implemented in Linux user space. fusepy is a Python module that provides a simple interface for FUSE and MacFUSE. This is just one file and is implemented using ctypes. With fusepy, you can change the behavior of the write function implementation and throw away EIO if you want. I would use the memory.py example as a base.