Sunday 15 August 2010

Why does the CPU profile in Visual vm show the process spends all its time in a promise deref when using clojure's core async to read a kafka stream -



Why does the CPU profile in Visual vm show the process spends all its time in a promise deref when using clojure's core async to read a kafka stream -

i running clojure app reading kaka stream. using shovel github project https://github.com/l1x/shovel read kafka stream. when profiled application using visual vm looking hotspots noticed of cpu time 70% beingness spent in function clojure.core$promise$reify__6310.deref.

the shovel api consumer thinwrapper on kafka consumergroup api. reads kafka topic , publishes out core async channel. should concerned application latencies affected if continued using api. there explanation why reify on promise taking much cpu time.

in clojure, $ used in printed representation of class represent inner class. clojure.core$promise$reify__6310.deref means calling method deref on class created via reify inner class of clojure.core/promise. turns out, if @ class of promise, show inner reified class within clojure.core$promise.

a promise in clojure represents info may not yet available. can see behavior in repl:

user> (def p (promise)) #'user/p user> (class p) clojure.core$promise$reify__6363 user> (deref p)

this hang , give no result, , not give next repl prompt, until deliver promise repl connection, or interrupt deref call. fact time beingness spent on deref of promise means programme logic waiting on values not yet computed (or have not yet come in via network, etc.).

clojure apache-kafka

No comments:

Post a Comment