How to buffer and burn low latency input to disk using C

Question

I need to write C code (not C # or C ++) to receive data from a hardware analyzer (via a 100 Mb TCP / IP network) and write it to disk.

Why do I say "Low latency", well, the hardware analyzer has an internal buffer of 9 KB, it is 2 ms of data storage, after which it is filled with a buffer, and I lose information packets.

My application can receive this buffer without packet loss, but I noticed that I could not write data to disk at this speed.

My code is as follows:

int main (int argc, char * argv [] )
{
  pthread_t th_rx; /* Thread for receiving packets */

  outputfile = fopen ("output.log", "wb");

  link_open(); // open device link

  pthread_create ( &th_rx, NULL, read_packets, 0 );

  // running loop
  fclose (outputfile);
  pthread_exit(NULL);
  link_close(); // close device link 
  return 0; 
}

static thread_result read_packets (void *arg)
{
  char  rxbuf[40960];

  while (receiving)
  {
    bytes_received = read_packet(); //read packet from device buffer
    rxbuf = extract_data(bytes_received);
    fwrite (rxbuf, 1, bytes_received, outputfile); 
  }
  return thread_return;
}

Here I need ideas on how to do this.

• 1: do not record the package after receiving, create a circular shape? Buffer
• 2: Creating a 2 circular stream buffer?

Any ideas on how to improve the code and what can I do to get a stable record?

, : (

Answer 1

, hw 9 kb 2 ms. 9*1024*500 = 4608000 b/s or 4.6 Mb/s. , 10 /. , . , 1 4,6 . , 1 hw, " " 500 . , .
ping-pong ( hw, HDD) . , /, .

---

Answer 2

, , , - .

• / ( ) , . , .

  • , , o , .

• .

• .

Answer 3

You should look at asynchronous I / O. However, the Async-I / O APIs are operating system dependent, and you did not specify which OS you are using. Also your use of threads does not make sense. If you want to use streams (instead of asynchronous I / O), then you will have one stream that reads the packet, and another stream that writes. Including it in a single thread does not give you any benefit.