Managing Agent Thread Pools in Clojure

Question

Is there a way to control thread pools that handle functions that are sent to agents? Since I understand things, if I go through, under the hood I use an unlimited thread pool. I would like to, say, run some functions in one thread pool and other functions on another. The reason for this is because I have some features that do IO and which are also less important. I would have thrown them into a limited pool of threads and would not have to worry if there is excessive blocking, and they are folded, as they are, well, less important. The main thing is that I would not want their crappy I / O lock to talk about affecting some of the more important functions that work in another thread pool.

I ask a question about something similar that I did with thread pools in Akka, and I'm just wondering if I can do the same with Clojure.

Answer 1

For Clojure versions prior to 1.4:

You cannot replace the built-in agent pools for sending and sending threads. They are hardcoded in Agent.java .

The send pool (used for calculation) has a fixed size = 2 + Runtime.getRuntime (). availableProcessors ().

The send pool (also used for futures) is a cached thread pool and will grow without restriction. This allows an arbitrary number of background tasks to wait for I / O. Cached streams will be reused and discarded if they are idle for one minute.

If you want to control the work in your thread pool, you need to plunge into java.util.concurrent or use the Clojure wrapper lib.

For Clojure 1.5 (upcoming):

You can provide your own ExecutorService using (send-through executor af), the default thread pools are no longer rigidly connected. See Agent.java in Clojure 1.5+ for more details.

Answer 2

The Clojure Claypoole library is designed for just that. It allows you to define thread flows and use (and reuse) them for futures, pmaps, etc.

Answer 3

Amit Rathor (from Runa inc) has published a library (called medusa) for managing thread pools. This seems like a pretty close match to what you are looking for.

http://s-expressions.com/2010/06/08/medusa-0-1-a-supervised-thread-pool-for-clojure-futures-2/