Pipe dreams aren’t necessarily made of promises

unsorted — cgrand, 2 April 2010 @ 14 h 31 min

Because of the spinning nature of atoms, it’s kind of a hack (a fun hack but still a hack) to build queues on it. Here is the same pipe function built on Java queues:

(defn pipe []
  (let [q (java.util.concurrent.LinkedBlockingQueue.)
        EOQ (Object.)
        NIL (Object.)
        s (fn s [] (lazy-seq (let [x (.take q)] 
                               (when-not (= EOQ x) 
                                 (cons (when-not (= NIL x) x) (s))))))]
    [(s) (fn ([] (.put q EOQ)) ([x] (.put q (or x NIL))))]))

24 Comments »

  1. [...] follow-up: pipe dreams aren’t necessarily made of promises [...]

  2. Thanks again Christophe.

    Comment by Edmund — 2 April 2010 @ 16 h 10 min
  3. why did you decide to return a vector instead of a list?

    Comment by William Hidden — 2 April 2010 @ 16 h 38 min
  4. Nice. It took me a moment to understand the purpose of the NIL object, until I found that LinkedBlockingQueue.put won’t accept a null pointer.

    Comment by Stuart — 2 April 2010 @ 16 h 48 min
  5. J’y comprends toujours rien… mais la “spinning nature of atoms” ? Clojure sert à programmer des ordinateurs quantiques ?

    Comment by Olivier — 2 April 2010 @ 17 h 24 min
  6. @Olivier: un atome Clojure est une référence mutable synchrone non coordonnée (ne participe pas à une transaction) qui est basée sur une opération CAS. Le CAS pouvant échouer, la fonction de mise à jour peut être appliquée plusieurs fois : c’est ça la spinning nature of atoms ! Et oui en écrivant ceci je trouvais ça quantiquement drôle.

    Comment by cgrand — 3 April 2010 @ 20 h 08 min
  7. @William: In Clojure vectors are preferred over lists in such cases.

    Comment by Meikel — 8 April 2010 @ 10 h 03 min
  8. [...] input. I needed to convert input to a more sanitary thing, and so lifted a page directly from the CGrand Master himself. In that post he shows how to create a LinkedBlockingQueue (henceforth LBQ to save pixels), [...]

  9. Note that the implementation above will OutOfMemoryError if large sets of values and/or large values are written into the pipe. Here’s a version that caps the queue that seems to do a better job:

    (defn pipe2
    “Returns a pair: a seq (the read end) and a function (the write end).
    The function can take either no arguments to close the pipe
    or one argument which is appended to the seq. Reads and writes may block.
    The (optional) argument to pipe2 is the maximum size of the queue (beyond
    that size writes will block) – default value if not provided is 100.”
    ([] (pipe2 100))
    ([size]
    (let [q (java.util.concurrent.LinkedBlockingQueue. size)
    EOQ (Object.)
    NIL (Object.)
    s (fn s [] (lazy-seq (let [x (.take q)]
    (when-not (= EOQ x)
    (cons (when-not (= NIL x) x) (s))))))]
    [(s) (fn ([] (.put q EOQ)) ([x] (.put q (or x NIL))))])))

    Test code:

    (let [[q f] (pipe2)]
    (future (doseq [x q] (println x)) (println “that’s all folks!”))
    (doseq [x (range 1000000000)] (f x))
    (f))

    Note: I’m a Clojure n00b, so this may introduce other issues (e.g. deadlock if the queue is full and the readers are also writers).

    Comment by Peter Monks — 8 January 2013 @ 7 h 58 min
  10. [...] read Paul Ingles post “From Callbacks to Sequences” which led me to Christophe Grand’s handy pipe function. The pipe function gives you two things wrapped in a vector: a sequence to pass on for downstream [...]

    Pingback by Pithering About » Pmap styled pipes — 9 February 2013 @ 16 h 34 min
  11. stamps machine

    Clojure and me » Pipe dreams aren’t necessarily made of promises

    Trackback by stamps machine — 18 November 2014 @ 15 h 09 min
  12. Pure & Easy Slim Reviews

    Clojure and me » Pipe dreams aren’t necessarily made of promises

    Trackback by Pure & Easy Slim Reviews — 19 November 2014 @ 13 h 12 min
  13. sma pcb Sma Plug

    Clojure and me » Pipe dreams aren’t necessarily made of promises

    Trackback by sma pcb Sma Plug — 19 November 2014 @ 19 h 16 min
  14. Velour Skin Reviews

    Clojure and me » Pipe dreams aren’t necessarily made of promises

    Trackback by Velour Skin Reviews — 20 November 2014 @ 7 h 32 min
  15. muscle building pills Review

    Clojure and me » Pipe dreams aren’t necessarily made of promises

    Trackback by muscle building pills Review — 21 November 2014 @ 12 h 27 min
  16. Wonderful beat ! I wish to apprentice while you amend your
    website, how can i subscribe for a blog website?

    The account aided me a acceptable deal. I had been tiny
    bit acquainted of this your broadcast provided bright clear concept

  17. Hence, in the event of a simple error, we can say that most people do not come into direct contact with him.
    Tesla key generator could also be taken care of
    by attending green electric soltuions a trade school to ensure that a person who earned knowledge through
    various of assignments.

    Comment by Gforce san diego electric company — 6 December 2014 @ 12 h 20 min
  18. Nulexa

    Clojure and me » Pipe dreams aren’t necessarily made of promises

    Trackback by Nulexa — 17 December 2014 @ 9 h 52 min
  19. Hey there Your entire web site runs up honestly slow for my situation, I don’t know who’s issue is that however ,
    flickr opens up relatively quick. Nevertheless, I have to say thanks for including brilliant article.
    I think it really has already been incredibly useful to user who
    actually click here. This is definitely wonderful everything that you actually have implemented and wish to see a lot more
    content from your website. To obtain more information from
    posts that you post, I have added this website.

    Comment by free dota 2 keys 2015 — 6 February 2015 @ 4 h 48 min
  20. It is said that if a devotee recites the Durga Stuti with full faith ‘ will be
    blessed by Devi Durga certainly. community events, programs and projects in Draper, Utah.
    Check out the terms & conditions: Do read the
    terms and conditions of the site from which you
    have decide to shop, to be clear with the delivery, refund
    policies and warranty.

    Comment by online festival — 11 November 2015 @ 10 h 52 min
  21. Supreme Court said, so that’s the state of

    Comment by Optimum — 17 September 2023 @ 2 h 20 min
  22. The state has a mix of commercial and tribal casinos that are granted licenses.

    Comment by jaidensr.slypage.com — 25 October 2023 @ 6 h 26 min
  23. explicit expression of a person’s will, making it clear that silence or passivity do not

    Comment by Optimum\ — 2 March 2024 @ 2 h 44 min
  24. good vibes

    Comment by black demon — 16 April 2024 @ 13 h 14 min

RSS feed for comments on this post. TrackBack URI

Leave a comment

(c) 2024 Clojure and me | powered by WordPress with Barecity