Rubber buffer for consumer producers in C

Question

In C, I have several threads producing long values, and one thread consuming them. Therefore, I need a fixed-size buffer implemented similarly to Wikipedia and methods that access it in a thread-safe manner.

At a general level, the following should be done:

• When added to a full buffer, the stream should be blocked (without overwriting old values).
• The consumer stream should be blocked until the buffer is full - its work will have a high constant cost and should do as much work as possible. (Does this call with two buffers?)

I would like to use a proven implementation, preferably from a library. Any ideas?

---

Motivation and explanation:

I am writing JNI code related to removing global links stored as tags in heap objects.

When the ObjectFree JVMTI event ObjectFree , I get a long tag representing a global link that I need to free using DeleteGlobalRef . For this I need a JNIEnv link - and getting it is really expensive, so I want to buffer requests and delete as many as possible at the same time.

There may be many threads receiving an ObjectFree event, and there will be one thread (mine) doing the removal of the link.

Answer 1

You can use one buffer with the mutex when accessing. You will need to keep track of how many items are being used. For "signaling" you can use state variables. One that is started by producer threads whenever they queue data; this frees up the consumer thread to process the queue to empty. Another that is triggered by the consumer stream when it has emptied the line; this signals any blocked producer threads to fill the queue. For the consumer, I recommend locking the queue and removing as much as possible before releasing the lock (to avoid too many locks), especially since the decompression operation is simple and quick.

Update
Some useful links:
* Wikipedia explanation
* POSIX Themes
* MSDN

Answer 2

Two possibilities:

a) malloc () a * Buffer structure with an array for storing some lengths and index - without blocking. Ask each malloc production chain to create its own buffer and start loading it. When the producer thread fills the last position of the array, the buffer queue for the consumer stream in the producer-consumer queue and immediately malloc () a new * Buffer. The consumer receives * Buffers and processes them, and then frees () them, (or queues them elsewhere or pushes them back into the pool for reuse by manufacturers). This avoids blocking the buffers themselves, leaving only the PC queue lock. The disadvantage is that producers who only occasionally generate their debts will not process their data until their buffer is full, which may take some time (you can push the buffer before the array is full in such a stream.

b) Declare a buffer structure with an array to hold several lengths and an index. Protect with mutex / futex / CS lock. malloc () only one common * Buffer and all threads receive a lock, click on them for a long time and release the lock. If the stream is placed at the last position of the array, the buffer queue for the consumer stream in the producer-consumer queue, immediately malloc new * Buffer and then release the lock. The consumer receives * Buffers and processes them, and then frees () them, (or queues them elsewhere or pushes them back into the pool for reuse by manufacturers).

Answer 3

You might want to accept the condition. Take a look at this consumer code snippet:

 while( load == 0 ) pthread_cond_wait( &notEmpty, &mutex ); 

What it does is check if the download is loaded (where you store the number of items in your list), but if it is zero, it will wait until the manufacturer displays a new item and places it in the list.
You must implement the same condition for the manufacturer (when he wants to put the item in the full list)