Primitive types support for fns coming to a Clojure branch near you

unsorted — cgrand, 10 June 2010 @ 8 h 57 min

Rich Hickey is working on primitive support in the prim branch. As of now, one can write:

(defn ^:static fib ^long [^long n]
  (if (>= (long 1) n)
    (long 1)
    (+ (fib (dec n)) (fib (- n (long 2))))))

and computes (fib 38) on my laptop in 650ms where my (or even Rich’s) best attempt took about 2s! If I tweak the above code to use unchecked ops (regular arithmetic ops on primitive types in Clojure throw an exception on overflow, unchecked ones don’t), the performance comes real real close (< 5%) to the equivalent Java code.

What’s this ^:static thing?

Primitive (double and long only: they subsume any other type) args and return values are only allowed on statics (note that the return type hint is put on the arglist so as to allow different hints for different arities). Statics are still fns in vars but they are backed by a static method and, when called by name, a direct static method call is emitted rather than going through the var and the IFn interface — as such static calls replace direct binding.

(my-static 42) ; direct call
(apply my-static 42 nil) ; regular call through the var

About the syntax: ^:keyword is a new reader shorthand for ^{:keyword true} (and keep in mind that in Clojure 1.2 ^ is the new #^), and metadata are merged: ^:static ^long ^{:doc "docstring"} x is now equivalent to ^{:static true :tag long :doc "docstring"} x.

Interesting times… as usual in the Clojure community. Thanks Rich!

Primitive types support for fns

unsorted — cgrand, 4 June 2010 @ 10 h 25 min

Follow up See here.


Update: I was wrong and Rich Hickey set me right: I didn’t measure the gains I’m expecting because the inline-expanded form still go through the var lookup. See here (or the comments) for real gains (around 800ms on my laptop).


This is a quick and dirty hack to emulate primitive types support for globally-bound fns:

(defmacro defhintedfn [name args & body]
  (let [iface (gensym "HintedFn")
        hname (vary-meta name assoc :tag iface)
        vname (vary-meta name assoc 
                :inline (fn [& args] `(.invoke ~hname ~@args)))]
    `(do
       (definterface ~iface
         (~(with-meta 'invoke {:tag (:tag name Object)}) ~args))
       (def ~vname nil) ; workaround for a soon-to-be-fixed bug
       (def ~vname
          (reify 
            ~iface
              (invoke [this# ~@args] 
                ~@body) 
            clojure.lang.IFn
              (^Object invoke [this# ~@(map #(vary-meta % dissoc :tag) args)]
                 ~@body))))))

As you can see the trick is to generate a dedicated interface and to inline calls to the defined fn to go through the specific interface rather than through IFn.

Let’s define a dumb numeric fn generating tons of calls: the fibonacci function! Below it comes in three flavors: hinted as int, hinted as long and plain.

(defhintedfn ^int hfib [^int n] (if (>= (int 1) n) (int 1) (+ (hfib (dec n)) (hfib (- n (int 2))))))
(defhintedfn ^long lfib [^long n] (if (>= (long 1) n) (long 1) (+ (lfib (dec n)) (lfib (- n (long 2))))))
(defn fib [n] (if (>= 1 n) 1 (+ (fib (dec n)) (fib (- n 2)))))

And here are the timings:

user=> (time (hfib 38))
"Elapsed time: 2016.128098 msecs"
63245986
user=> (time (lfib 38))
"Elapsed time: 2896.46198 msecs"
63245986
user=> (time (fib 38))
"Elapsed time: 4704.449867 msecs"
63245986

Please note that hinted fns are still regular fns:

user=> (map hfib (range 10))
(1 1 2 3 5 8 13 21 34 55)

European Clojure Training, Brussels, late June

unsorted — cgrand, 14 April 2010 @ 12 h 29 min

Following a recent tweet, I’m setting up a Clojure training session with Lau B. Jensen and we’d like to know more about our potential attendees. Please fill this short survey:


(click here if you don’t see the above form.)

Pipe dreams aren’t necessarily made of promises

unsorted — cgrand, 2 April 2010 @ 14 h 31 min

Because of the spinning nature of atoms, it’s kind of a hack (a fun hack but still a hack) to build queues on it. Here is the same pipe function built on Java queues:

(defn pipe []
  (let [q (java.util.concurrent.LinkedBlockingQueue.)
        EOQ (Object.)
        NIL (Object.)
        s (fn s [] (lazy-seq (let [x (.take q)] 
                               (when-not (= EOQ x) 
                                 (cons (when-not (= NIL x) x) (s))))))]
    [(s) (fn ([] (.put q EOQ)) ([x] (.put q (or x NIL))))]))

The reasons why Enlive templates return seqs of strings

unsorted — cgrand, 10 February 2010 @ 12 h 03 min

The main reasons behind Enlive templates returning seqs of strings are:

  • Ring does not expose the output stream (and I think it is a good thing)
  • I don’t want to allocate the whole response as a single string

Hence it seemed to me the simplest thing to do. (Simpler than implementing InputStream or making templates and Ring to communicate through a kind of reduce-based IO.)

Nodes serializations are cached when the source of the template is read:

user=> (deftemplate hello (-> "<h1>This is a static title<h2>This is a dynamic title" java.io.StringReader. html-resource) [name] [:h2] (content name))
#'user/hello
user=> (hello "Replaced dynamic title")
("<html>" "<body>" "<h1>This is a static title</h1>" "<h2>" "Replaced dynamic title" "</h2>" "</body>" "</html>")
user=> (hello "Replaced dynamic title II")
("<html>" "<body>" "<h1>This is a static title</h1>" "<h2>" "Replaced dynamic title II" "</h2>" "</body>" "</html>")
user=> (map identical? *1 *2)
(true true true true false true true true)

Thus no string allocation except for the dynamic parts but the real raison d’être of the static nodes serializations cache is to reduce the time spent traversing the tree and the number of allocated Cons (since the strings are longer).

Even without this cache there’s no String allocation when serializing a tree (but many Cons instances are allocated):

user=> (let [x {:tag :html :content [{:tag :body :content [{:tag :h1 :content ["This is a static title"]} {:tag :h2 :content ["This is a dynamic title"]}]}]}]
         (every? (partial apply identical?) (map vector (emit* x) (emit* x))))
true

Are pipe dreams made of promises?

unsorted — cgrand, 18 November 2009 @ 19 h 32 min

follow-up: pipe dreams aren’t necessarily made of promises

(defn pipe
 "Returns a pair: a seq (the read end) and a function (the write end).
  The function can takes either no arguments to close the pipe 
  or one argument which is appended to the seq. Read is blocking."
 []
  (let [promises (atom (repeatedly promise))
        p (second @promises)]
    [(lazy-seq @p) 
     (fn 
       ([] ;close the pipe
         (let [[a] (swap! promises #(vector (second %)))]
           (if a
             (deliver a nil) 
             (throw (Exception. "Pipe already closed")))))
       ([x] ;enqueue x
         (let [[a b] (swap! promises next)]
           (if (and a b)
             (do 
               (deliver a (cons x (lazy-seq @b)))
               x)
             (throw (Exception. "Pipe already closed"))))))]))

Beware of not printing the seq while the pipe is still open!

(let [[q f] (pipe)]
  (future (doseq [x q] (println x)) (println "that's all folks!"))
  (doseq [x (range 10)]
    (f x))
  (f)) ;close the pipe

Enlive Google group

unsorted — cgrand, 15 October 2009 @ 21 h 36 min

I created a google group dedicated to Enlive — my HTML/XML CSS-based transformation and extraction library.

Clojure and Me has moved

unsorted — cgrand, 26 September 2009 @ 16 h 04 min

I’m sorry if my previous posts reappeared in your RSS reader. I just moved Clojure and me from blogger to cgrand.net.

About the subtitle

unsorted — cgrand, 13 August 2009 @ 10 h 54 min

The subtitle for this blog may be misleading. Actually it reads “When the pupil is ready to learn, a teacher will appear.”
I picked this Zen proverb because in early 2008 I was looking for a language with the following requirements: strong metaprogrammability, strong concurrency support, strong functional bias and bonus points for running on the JVM. I had prepared myself to not find the rare bird when I met Clojure (and it was nearly love at first sight).
So in that perspective I am the pupil and Clojure (the language, Rich Hickey and the whole community) the teacher.

Clojure as fast as Java!

unsorted — cgrand, 6 August 2009 @ 17 h 08 min

I wrote a prototypal implementation of IPersistentSet in Clojure written with new new and, surprisingly, on its first benchmark it’s already on par with Java-based implementations.

(dotimes [i 10]
  (let [n (int (Math/pow 2 (+ 10 i)))
        _ (println "n=" n)
        a (do
            (print "clojure\t")
            (time (doall (let [s (reduce conj empty-set (range 0 n 2))] (map #(get s %) (range n))))))
        b (do
            (print "java\t")
            (time (doall (let [s (reduce conj #{} (range 0 n 2))] (map #(get s %) (range n))))))]
    (println (= a b))))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

n= 1024
clojure "Elapsed time: 1.006064 msecs"
java "Elapsed time: 0.736197 msecs"
true
n= 2048
clojure "Elapsed time: 1.813009 msecs"
java "Elapsed time: 1.328102 msecs"
true
n= 4096
clojure "Elapsed time: 3.590191 msecs"
java "Elapsed time: 2.608153 msecs"
true
n= 8192
clojure "Elapsed time: 7.046566 msecs"
java "Elapsed time: 5.807302 msecs"
true
n= 16384
clojure "Elapsed time: 16.015862 msecs"
java "Elapsed time: 10.284897 msecs"
true
n= 32768
clojure "Elapsed time: 29.803928 msecs"
java "Elapsed time: 23.850378 msecs"
true
n= 65536
clojure "Elapsed time: 68.79778 msecs"
java "Elapsed time: 63.947582 msecs"
true
n= 131072
clojure "Elapsed time: 132.082499 msecs"
java "Elapsed time: 113.433411 msecs"
true
n= 262144
clojure "Elapsed time: 292.149631 msecs"
java "Elapsed time: 265.39197 msecs"
true
n= 524288
clojure "Elapsed time: 595.265321 msecs"
java "Elapsed time: 698.711009 msecs"
true

To tell the truth, this benchmark is a bit unfair for Java: no dummy map and slightly different algorithms (half-full bitmap nodes become array-nodes, no leaf-node etc.).
Follow me on Twitter, here

« Previous PageNext Page »
(c) 2018 Clojure and me | powered by WordPress with Barecity