Fixed-length structure with variable-length reserved space

Question

In the embedded world, we often have data structures that are passed through fixed-length buffers. They are relatively easy to use, using something like this:

#define TOTAL_BUFFER_LENGTH 4096 struct overlay { uint16_t field1; uint16_t field2; uint8_t array1[ARY1_LEN]; }; static_assert(sizeof(struct overlay) <= TOTAL_BUFFER_LENGTH); struct overlay* overlay = malloc(TOTAL_BUFFER_LENGTH); 

That is, we use the data structure as an overlay to provide easy access to that part of the buffer used.

However, we do have several buffer formats that also use the last few bytes of the buffer to store things like checksums. Currently we use such designs:

 struct overlay { uint16_t field1; uint16_t field2; uint8_t array1[ARY1_LEN]; char reserved[TOTAL_BUFFER_LENGTH - sizeof(uint16_t) - sizeof(uint16_t) - (sizeof(uint8_t) * ARY1_LEN) - sizeof(uint32_t)]; uint32_t crc; }; 

How ugly, as it searches for this simple data structure, it is an absolute monster when the structure grows to have dozens of fields. This is also a nightmare for repairability, since adding or removing a structure field means that the size calculation for reserved must be updated at the same time.

When the end of the structure contains only one element (for example, a checksum), we sometimes use a helper function to read / write the value. This makes the data structure clean and maintained, but it does not scale well when the end of the buffer has multiple fields.

This would help a lot if we could do something like this:

 struct overlay { uint16_t field1; uint16_t field2; uint8_t array1[ARY1_LEN]; char reserved[TOTAL_BUFFER_LENGTH - offsetof(struct overlay, reserved) - sizeof(uint32_t)]; uint32_t crc; }; 

Unfortunately, offsetof only works with full object types, and since it is in the middle of a struct overlay definition, this type is not yet complete.

Is there a cleaner, more convenient way to do this? I essentially need a fixed-length structure with margins at the beginning and at the end, and the remaining space in the middle is reserved / not used.

Answer 1

In your situation, I think that I will probably do this:

 typedef struct overlay_head { uint16_t field1; uint16_t field2; uint8_t array1[ARY1_LEN]; } overlay_head; typedef struct overlay_tail { uint32_t crc; } overlay_tail; enum { OVERLAY_RSVD = TOTAL_BUFFER_LENGTH - sizeof(overlay_head) - sizeof(overlay_tail) }; typedef struct overlay { overlay_head h; uint8_t reserved[OVERLAY_RSVD]; overlay_tail t; } overlay; 

Then you can work almost as before, except where you write p->field1 you now write p->h.field1 , and where you write p->crc , now you write p->t.crc .

Note that this handles arbitrarily large tail structures quite efficiently if the head and tail also fit into the overall size.

Answer 2

You can define a structure that simply has a buffer with a CRC field at the end:

 struct checked_buffer { char data[TOTAL_BUFFER_LENGTH - sizeof(uint32_t)]; uint32_t crc; }; 

and then put your overlay in the data field. Presumably, you have already specified casting to "convert" the raw char* buffer to overlay* , so you don't need to do anything other than overlay* to checked_buffer* when you want to access the CRC field.

But if you want to have the field in a consistent position through a bunch of structures, it would be easier to place it at the beginning of each structure. Thus, you can declare it directly in each structure without having to do anything strange, and you do not need any actions with pointers in order to access it.

Answer 3

How about this?

 union a256 { struct { int field_a; int field_b; char name[16]; // int crcshadow; }; struct { char buff[256-sizeof(int)]; int crc; }; } ; static_assert(offsetof(a256, crcshadow) < offsetof(a256, crc), "data too big"); 

The first structure contains data, the second defines a fixed size for this union.