关于多线程:core.async中的块与线程

go block vs thread in core.async

来自http://martintrojer.github.io/clojure/2013/07/07/coreasync-and-blocking-io/:

To get a bit more concrete let's see what happens when we try to issue
some HTTP GET request using core.async. Let's start with the naive
solution, using blocking IO via clj-http.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
(defn blocking-get [url]
  (clj-http.client/get url))


(time
   (def data
     (let [c (chan)
           res (atom [])]
       ;; fetch em all
       (doseq [i (range 10 100)]
         (go (>! c (blocking-get (format"http://fssnip.net/%d" i)))))
       ;; gather results
       (doseq [_ (range 10 100)]
         (swap! res conj (<!! c)))
       @res
       )))

Here we're trying to fetch 90 code snippets (in parallel) using go
blocks (and blocking IO). This took a long time, and that's because
the go block threads are"hogged" by the long running IO operations.
The situation can be improved by switching the go blocks to normal
threads.

1
2
3
4
5
6
7
8
9
10
11
12
(time
   (def data-thread
     (let [c (chan)
           res (atom [])]
       ;; fetch em all
       (doseq [i (range 10 100)]
         (thread (>!! c (blocking-get (format"http://fssnip.net/%d" i)))))
       ;; gather results
       (doseq [_ (range 10 100)]
         (swap! res conj (<!! c)))
       @res
       )))

"长时间运行的IO操作阻碍了执行块线程"是什么意思?


Go块旨在成为一种轻量级的协作线程;通过在池中使用几个线程并在停放时切换go块(例如,在使用<!的通道上等待时),它们提供的类线程行为比完整的JVM线程具有更少的开销。当您在阻塞JVM线程的块中调用方法时,线程切换无法工作,因此您很快用完了JVM线程。等待时,大多数标准Java(和Clojure)IO操作都会阻塞当前线程。


What does it mean that"go block threads are hogged by the long running IO operations"?

专门用于服务go块的线程数量有限。如果您在其中一个线程上执行阻塞I / O操作,则除非该操作完成(除非线程被中断),否则它不能用于任何其他目的。对于非执行块线程(即从thread函数返回的线程)也是如此,但是非执行块线程并非来自有限的go块线程池。因此,如果您确实在go块中阻止了I / O,则即使该线程未做任何实际的工作(它只是在等待I / O),您也"希望"该go块的线程不被其他go块使用。操作)。

*当前该数目恰好是JVM可用的处理器数目的42。