Consider this sequential procedure in a data structure containing collections (for simplicity, call their lists) Doubles . As long as I feel that:
- Choose two different lists from any structure
- Calculate statistics based on these lists
- Flip a coin based on this statistic
- Perhaps change one of the lists based on the results of the coin toss.
The goal is to eventually achieve the convergence of something, so the “solution” is linear in the number of iterations. The implementation of this procedure can be seen in the SO question here , and here's an intuitive visualization:
It seems that this procedure can be better performed, that is, convergence can be achieved faster - by using several workers running simultaneously in separate OS threads, for example:
I suppose that a perfectly implemented implementation of this should be able to achieve a solution in O (n / P) time, for P the amount of available computing resources.
Reading at the Haskell concurrency left my head cool with terms like MVar , TVar , TChan , acid-state , etc. Obviously, a parallel implementation of this procedure will look very different from the one I linked above . But the procedure itself, apparently, is essentially a pretty tame algorithm in what is essentially a database in memory, which is a problem that I'm sure someone has come across before.
I assume that I will have to use some kind of mutable parallel data structure that supports decent random access (i.e. random elements of inactivity) and modification. I get a little lost when I try to collect everything that may be required in order to improve performance (for example, STM seems dubious).
What data structures, concurrency concepts, etc. suitable for this kind of task, if the goal is to increase productivity over a consistent implementation?