Is it possible to create fast large circular buffer arrays to write a stream in Haskell?

Question

I am considering turning a C # application into a Haskell as my first “real" Haskell project. However, I want to make sure that the project makes sense. The application collects data packets from ~ 15 consecutive streams that reach about 1 kHz, downloads these values ​​to the corresponding circular buffers on my "context" object, each with ~ 25000 elements, and then sends these arrays to OpenGL at 60 Hz to display the waveform . (Thus, it must be saved as an array, or at least converted to an array every 16 ms). My context object also has about 70 fields that only support the current (last) value, and not the flow waveform.

There are several aspects of this project that display Haskell well, but I'm worried about performance. If for each new datapoint in any of the threads I have to clone the entire context object with 70 fields and 15 arrays of 25,000 elements, it is obvious that there will be performance problems.

Is it possible to get around this by putting everything in the IO monad? But then this seems to belittle the purpose of using Haskell, right? Also, all my C # code is event driven; is there an idiom for this in Haskell? It seems that adding a listener creates a “side effect”, and I'm not sure exactly how this will be done.

Answer 1

Yes, you probably want to use the IO monad for mutable data. I do not think that the ST monad is well suited for this problem space, because data updates alternate with valid IO actions (reading input streams). Since you need to do IO in ST using unsafeIOToST , I find it preferable to just use IO directly. Another approach with ST is to constantly thaw and freeze the array; this is dirty because you need to ensure that old copies of the data are never used.

Although it shows that a clean solution (in the form of Data.Sequence.Seq ) is often faster than using mutable data, given your requirement, the data will be pushed out in OpenGL, you can get better performance from working with the array directly. I would use functions from Data.Vector.Storable.Mutable (from a vector package), since then you have access to ForeignPtr for export.

You can see the arrows ( Yampa ) for one very common event-driven code approach. Another area is functional reactivity (FRP). Some fairly mature libraries, such as Netwire or reactive banana ones, are beginning to appear in this area. I do not know if they will provide adequate performance for your requirements; I mainly used them for gui style programming.

Answer 2

Look at this link in the "ST Monad" section:

http://book.realworldhaskell.org/read/advanced-library-design-building-a-bloom-filter.html

In the section “Modifying array elements”, we mentioned that modifying an immutable array is excessively expensive because it requires copying the entire array. Using UArray does not change this, so what can we do to lower the cost to acceptable levels?

In an imperative language, we simply change the elements of the array in place; this will be our approach in Haskell.

Haskell provides a special monad called ST that allows us to work safely with volatile state. Compared to the state monad, it has powerful added features.

We can thaw an immutable array to give a mutable array; change the mutable array; and freeze a new immutable array when we are done.

...

Monk IO also provides these features. The main difference between the two is that the ST monad is intentionally designed so that we can escape from it back into pure Haskell code.

Thus, it should be possible to change the location in place, and it will not defeat the purpose of using Haskell.