Securely write to embedded Linux compact flash memory

Question

I am developing an embedded Linux system that runs on compact flash and tmpfs. The flash is read-only and usually should stay that way, but sometimes I need to write something to a USB flash drive.

What precautions should be taken when recording to flash (via PATA)? For reasons that I can’t remember, I use the ext4 file system mounted with barrier=1,data=ordered,nodelalloc,noatime,ro . Is there any terrible idea? The system should boot quickly with zero interference. I am tempted to do tune2fs -c 0 -i 0 . Is this an even worse idea?

Also, when I write something, I obviously need to remount flash-read-write, write, and then read-only remount. The problem is that there are several different processes (both C ++ binary and shell scripts) that may be required for this. Obviously, every process indiscriminately restores a read-only file system when this is done is a bad idea.

What is the best way to coordinate this? flock looks promising; is this the best way to go and what do i need to worry about? I do not want an outdated lock to block writing or leave the file system unlimited.

To clarify: “Sometimes” I write, I mean that the system can last for years without requiring anything to be written. When something is written, it may be a couple of hundred bytes. At the same time, the system must withstand unpredictable energy cycles without any intervention.

Answer 1

Ext4 security

ext4 is not ideal for this; your worst case scenario will always be that the power does not work while your RW partition is corrupting the file system.

Although they are designed for raw NAND devices, a file registration system such as jffs2 or yaffs can be useful. They have a useful characteristic that their format on the disk is more like a list rather than a tree, so newer data tends to be packed more closely and, therefore, is less likely to destroy relatively static parts of the file system.

I cannot comment on how to make ext4 “safe” in these circumstances; both the file system structure and fsck are not aware of the general failure modes of flash devices (in particular, that for any recording there is a long period when the entire data block is invalid.) This behavior, of course, is typical for the controller on the CF card, therefore it is not very predictable in any case.

The safest and most predictable way that I can think of is the same flip buffer data layout that is used in small embedded devices. That is, you have two zones for recording (two sections). At any given time, at least one of them must be consistent. When you want to write, select the eldest of the two. If you turn on the power and find an inconsistency in buffer A, copy the contents of buffer B across (and do not allow a single entry in B at this time.)

Mount / remount management

I would write a daemon that controls shutdown / reboot on behalf of all processes in the system. This has several advantages:

• you highlight umount / remount logic and should only execute it once
• processes that want to write do not need root privileges
• You can create some kind of watchdog timer to ensure that the partition does not remain RW for too long. If so, you can kill the offensive process (possibly a rollback) and reinstall RO.
• You can measure how long you have been installing RW, which gives you some idea of ​​the risk associated with each update.

Answer 2

I made an embedded system with a 4GB CF card. I configure it to 3 sections:

• / downloads; ext2; ~ 128
• root; SquashFS; 500 MB (since it is not writable, there is no way to accidentally put it into recording mode.)
• /data; ext2; ~ 3.5 GB any data I want to save

I use ext2 because I believe that any of the journal FSs will actually cause more blocks to be modified for each record. In addition, there is no flash file system on the CF card, because the system does not have direct access to flash blocks. The CF card contains its own block mapping logic.

Answer 3

Since you use CF, wear leveling is most likely built-in, and therefore you will not get any benefit using a file system such as yaffs2, jffs2 etc. that have built-in wear leveling. In fact, most height-level file systems are not recommended for CF cards that have a built-in level of wear. You need the file system that built the log so you can safely go through the power cycle without damaging the system. There are two options: choosing a journaling file system or choosing db (assuming the entries are db-centric) with leveling, like sqlite. You can use either ext3 or ext4 for the file system, although I personally recommend ext3, because ext4 still has bugs that can give you a headache. I would also recommend that others have two partitions, one read-only clean using a file system like squashfs, cramfs and the other read-write using ext3.

Having said that, I'm not sure if your device is built in EEPROM, but if it is, and you just need to write a few hundred bytes every few years, the simplest configuration would be to use a single reade filesyetem file with EEPROM as your data area for those several hundred bytes.

Answer 4

I would make a separate section that mounts rw.