How to "defragment" an ext3 directory?

Question

I run a daemon that parses files in a directory and then deletes them. If the daemon does not work for any reason, the files are collected there. Today I had 90k files in this directory. After starting the daemon, it processed all the files.

However, the catalog remains large; "ls -dh". returns a size of 5.6M. How can I defragment this directory? I already realized that renaming this directory and creating a new one with the same name and permissions solves the problem. However, since files can be written there at any time, there seems to be no reliable way to rename a directory and create a new one for a moment, the target directory does not exist.

So, a) is there a way to / a (shell) a program that can defragment directories in the ext3 file system? or b) is there a way to create a lock in a directory in order to try to write file blocks before the renaming / creation is complete?

Answer 1

"Optimize directories on the file system. This option causes e2fsck to try to optimize all directories by reindexing them if the file system supports directory indexing or directory sorting and compression for small directories or for file systems using traditional linear directories." - fsck.ext3 -D

Of course, this should not be done on a mounted file system.

Answer 2

Not applicable for Ext3, but may be useful for users of other file systems:

• according to https://wiki.archlinux.org/index.php/Btrfs#Defragmentation , with Btrfs , it seems possible to defragment directory metadata: btrfs filesystem defragment / will defragment the root folder metadata. It uses Btrfs online defragmentation support.

• and Ext4 supports online defragmentation (using e4defrag ), this does not seem to apply to directory metadata (according to http://sourceforge.net/p/e2fsprogs/bugs/326/ ).

I have not tried any of these solutions.

Answer 3

I do not know how to return free space from a directory.

5MB is not a lot of space, so the easiest way to just ignore it. If this problem (files folding into a directory) occurs regularly, then this space will be reused at any time when the directory is full again.

If you desperately need the ability to compress a directory here (ugly) hack that might work.

Replace the directory with a symlink to the empty directory. If this problem persists, you can create a new empty directory and then change the symbolic link to point to the new directory. Changing a symbolic link must be atomic, so you won’t lose incoming files. Then you can safely delete and delete the old directory.

[Edited to add: Turns out this doesn't work. As Bud notes in the comments, you cannot atomize the change of the symbolic link as I suggested. This leaves me from my original point of view. The file systems I am familiar with do not provide a mechanism for restoring free space in directory blocks.]