FIFO Implementation

Question

Consider the following code:

writer.c

mkfifo("/tmp/myfifo", 0660); int fd = open("/tmp/myfifo", O_WRONLY); char *foo, *bar; ... write(fd, foo, strlen(foo)*sizeof(char)); write(fd, bar, strlen(bar)*sizeof(char)); 

reader.c

 int fd = open("/tmp/myfifo", O_RDONLY); char buf[100]; read(fd, buf, ??); 

My question is:

Since he does not know in advance how many bytes foo and bar will have, how can I find out how many bytes to read from reader.c?
Because if, for example, I read 10 bytes in the reader, and foo and bar together are less than 10 bytes, I will have both of them in the same variable and that I don’t want. Ideally, I would have one read function for each variable, but again I don’t know before myself how many bytes the data will have.
I was thinking of adding another entry in writer.c between the entry for foo and bar with a delimiter, and then I would not have a problem decrypting it with reader.c. Is this the way to it?

Thanks.

Answer 1

A separator is one way around this, and it will work fine if you know the order of your data and you use the separator as the only separator and never as part of your data.

Another way is to precede each write to the pipe with the number of subsequent bytes in a fixed width. This way you will know how much data is going to go down the pipe. Use a fixed width so that you know exactly how long the width field will be, so you know how to start and stop reading each piece of data.

Answer 2

Many other answers mention using some kind of protocol for your data, and I think this is the right approach. This protocol can be as simple or complex as possible. I have provided some examples that may be useful ¹ .

---

In the simple case, you can only have a byte of length followed by a byte of data (for example, string C).

  + -------------- +
 |  length byte |
 + -------------- +
 |  data byte (s) |
 + -------------- + 

Writer:

 uint8_t foo[UCHAR_MAX+1]; uint8_t len; int fd; mkfifo("/tmp/myfifo", 0660); fd = open("/tmp/myfifo", O_WRONLY); memset(foo, UCHAR_MAX+1, 0); len = (uint8_t)snprintf((char *)foo, UCHAR_MAX, "Hello World!"); /* The length byte is written first followed by the data. */ write(fd, len, 1); write(fd, foo, strlen(foo)); 

Reader:

 uint8_t buf[UCHAR_MAX+1]; uint8_t len; int fd; fd = open("/tmp/myfifo", O_RDONLY); memset(buf, UCHAR_MAX+1, 0); /* The length byte is read first followed by a read * for the specified number of data bytes. */ read(fd, len, 1); read(fd, buf, len); 

---

In a more complex case, you may have a byte of length followed by data bytes containing a simpler string C.

  + ---------------- +
 |  length byte |
 + ---------------- +
 |  data type byte |
 + ---------------- +
 |  data byte (s) |
 + ---------------- + 

General heading:

 #define FOO_TYPE 100 #define BAR_TYPE 200 typedef struct { uint8_t type; uint32_t flags; int8_t msg[20]; } __attribute__((aligned, packed)) foo_t; typedef struct { uint8_t type; uint16_t flags; int32_t value; } __attribute__((aligned, packed)) bar_t; 

Writer:

 foo_t foo; unsigned char len; int fd; mkfifo("/tmp/myfifo", 0660); fd = open("/tmp/myfifo", O_WRONLY); memset(&foo, sizeof(foo), 0); foo.type = FOO_TYPE; foo.flags = 0xDEADBEEF; snprintf(foo.msg, 20-1, "Hello World!"); /* The length byte is written first followed by the data. */ len = sizeof(foo); write(fd, len, 1); write(fd, foo, sizeof(foo)); 

Reader:

 uint8_t buf[UCHAR_MAX+1]; uint8_t len; uint16_t type; union data { foo_t * foo; bar_t * bar; } int fd; fd = open("/tmp/myfifo", O_RDONLY); memset(buf, UCHAR_MAX+1, 0); /* The length byte is read first followed by a read * for the specified number of data bytes. */ read(fd, len, 1); read(fd, buf, len); /* Retrieve the message type from the beginning of the buffer. */ memcpy(&type, buf, sizeof(type)); /* Process the data depending on the type. */ switch(type) { case FOO_TYPE: data.foo = (foo_t)buf; printf("0x%08X: %s\n", data.foo.flags, data.foo.msg); break; case BAR_TYPE: data.bar = (bar_t)buf; printf("0x%04X: %d\n", data.bar.flags, data.bar.value); break; default: printf("unrecognized type\n"); } 

_{1 - This code has been written from memory and not verified.}

Answer 3

A separator is indeed one way to do this - and quite conveniently, C lines come with such a separator - nul-terminator at the end of the line.

If you change the write() calls so that they also write nul-terminator (note that sizeof(char) defined as 1, so you can ignore it):

 write(fd, foo, strlen(foo) + 1); write(fd, bar, strlen(bar) + 1); 

Then you can select the lines after reading them (you still need to read them into one buffer, and then split them into pieces if you do not read them at a time).

Answer 4

To summarize the WhirlWind answer a bit, you must set the protocol of the hundredth grade. There must be an order in what you send, or you do not know from top to bottom how you indicate.

Both WhirlWind suggestions will work. You can also go so far as to implement your own (or standard) protocol over a pipe or FIFO, to subsequently port your code to a more distributed environment with disparate systems and simplify the task. The essence of the problem is that you must establish rules for communication before you can communicate.

Answer 5

You will need to define some kind of wired protocol or serialization / deserialization format so that your reader knows how to interpret the data that he reads from fifo. Using a separator is the easiest way to do this, but you will run into problems if your separator ever appears as part of your author’s output.

A little further on the complexity scale, your protocol can determine both the separator and the way to indicate the length of each "part" or "message" of the transmitted data.

Finally, this problem is more thoroughly solved by writing serialized messages that your writer will then deserialize after receiving. You may be interested in using something like Protocol Buffers or Thrift to achieve this (with an added bonus that you can implement your reader or writer in several programming languages ​​without changing your protocol).