I have a number of sensors connected to a PC that measure various physical parameters such as force, rotation speed and temperature. These sensors continuously produce samples at a specific sampling rate. A sample consists of a time stamp and the measurement itself; sampling frequencies are in single-digit kilohertz values (i.e., somewhere between 1 and 9000 samples per second).
The PC must read and store these samples for a certain period of time. Then the collected data is further processed and evaluated.
What would be a reasonable way to buffer samples? With some realistic settings, an acquisition can easily collect a couple of megabytes per second. Also, paging can be critical if the memory is allocated quickly, but requires replacement during recording.
I could think of a threading approach when a separate thread allocates and manages a pool of (blocked, so not to be replaced) memory blocks. Considering that these pieces are always allocated, their further distribution will be blocked (if the pages of other processes should be replaced) this memory pool thread, and data collection can continue without interruption.
This is basically a conceptual question. However, more specifically:
- It should rely only on portable features like POSIX. Qt's capabilities are still great.
- Sensors can be paired in various ways. IP is one of the possibilities. Usually, sensors are directly connected to a PC via local channels (RS232, USB, expansion cards, etc.). That is, fast enough.
- Timestamps are mainly used by the collection equipment itself, if possible, to avoid network jitter, etc.
Thinking over
Should I really worry? Apparently the problem translates into three scenarios:
- Very little data has been collected. It can be easily buffered in one large pre-allocated buffer.
- Data is being collected slowly. Highlighting buffers on the fly is excellent.
- , . , . , .