I have a recursive problem where consumers do some work at each level of the tree and then have to recursively flush the tree and do the same work at the next level.
I want to use ConcurrentBag / BlockingCollection etc. to run this in parallel. In this case, queue consumers are also queue producers!
My problem is this: with BlockingCollection, I can write very simple foreach logic to remove items and queue new ones. When the queue is empty, the blocking collection will block correctly and wait for a new job to be created by one of the other consumers.
But how do I know if all consumers are blocking ?!
I know about CompleteAdding (), but it doesn't seem to work, since the only time you actually finish is when all the producers are produced and the queue is empty - and since they all will block, no one is “free” to install CompleteAdding (). Is there any way to detect this? (Perhaps an event that might fire when it locks and fire again when it unlocks?)
I can handle this manually, not using foreach, but manually having a while (! Complete) loop and using TryTake, but then I need to manually sleep, which seems inefficient (all reason to have a lock collection using only matching collections!) Every time through loop, if TryTake is false, I can set the Idle flag and then check Master, if the queue is empty and all threads are inactive, set the full flag, but again, it seems kludgy.
Intuition tells me that there is a way to use the Blocking Collection to do this, but I cannot get there.
In any case, someone has a good example, when consumers are manufacturers and can discover when to release all the blocks, it would be awesome
Jason coyne
source share