Go block vs thread in core.async

Question

From http://martintrojer.imtqy.com/clojure/2013/07/07/coreasync-and-blocking-io/ :

To get more specific information, let's see what happens when we try to issue some HTTP GET request using core.async. Let's start with a naive solution using IO locking via clj-http.

(defn blocking-get [url] (clj-http.client/get url)) (time (def data (let [c (chan) res (atom [])] ;; fetch em all (doseq [i (range 10 100)] (go (>! c (blocking-get (format "http://fssnip.net/%d" i))))) ;; gather results (doseq [_ (range 10 100)] (swap! res conj (<!! c))) @res ))) 

Here we are trying to get 90 pieces of code (in parallel) using go blocks (and IO lock). This took a long time, and this is because the threads of the blocks of moves are clogged with long I / O. The situation can be improved by switching go blocks to normal state flows.

 (time (def data-thread (let [c (chan) res (atom [])] ;; fetch em all (doseq [i (range 10 100)] (thread (>!! c (blocking-get (format "http://fssnip.net/%d" i))))) ;; gather results (doseq [_ (range 10 100)] (swap! res conj (<!! c))) @res ))) 

What does it mean that "go block flows are pulled by long I / O operations"?

Answer 1

Go blocks are designed to be a kind of light cooperative thread; they provide thread-like behavior with less overhead than full JVM threads, using multiple threads in the pool and switching blocks when parking - for example, when waiting on a channel using <! . Thread switching cannot work when you call a method in a block that blocks the JVM thread, so you quickly run out of JVM threads. Most of the standard Java (and Clojure) I / O operations block the current thread while waiting.

Answer 2

What does it mean that "go block flows are pulled by long I / O operations"?

There are a limited number of threads designed to serve go * blocks. If you perform an I / O blocking operation in one of these threads, then it cannot be used for any other purpose until this operation completes (if the thread is not interrupted). This is also true for non-thread threads (i.e., Threads that are returned from the thread function), but no-thread threads come from a thread pool with limited progress. Therefore, if you block I / O in the go block, you are "hogging" that block the stream from being used by other go blocks, although the stream does not do any real work (it just waits for the I / O operation).

* This number is currently 42 + the number of processors available to the JVM.