I'm confused. Is Ferret a Clojure compiler? Because the code is Clojure, and the file extensions are CLJ, but it's able to compile Clojure into C++? Why is it called a language?
> Ferret is a functional, lazy language designed to be used in real time embedded control systems. It is heavily inspired by Clojure both syntactically and semantically. Functions / Macros that are present in both Ferret and Clojure should mimic their Clojure counter parts. If they don't it is considered a bug. (or not possible to implement with the current implementation.)
> This document is not intended to be a full lisp tutorial. It is a specification of the subset of lisp implemented by Ferret, and the particular workings of the Runtime and Core library. Any getting started guide for Clojure should get you upto speed on Ferret.
I remember reading about this years ago but I don't remember the "realtime" aspect. I remembered it just being a general Clojure-to-C++ compiler. I wonder if the authors "pivoted"?
I couldn't find any documentation in its runtime about how it plans to handle interrupts so I would wager it's probably "soft" real-time. Also not sure if the generated code can throw exceptions but if it can then hard realtime is out.
Well, it's an empirical observation that Haskell programmers, however experienced they may be, are constantly surprised by the memory footprint of their programs — IIRC sometimes even the GHC's core developers get blindsided.
That's only for long running programs generally with a GC. Ferret does not have a GC by default and furthermore can run without heap allocation if you want.
One of the ways to get lower latency is in fact to never deallocate and just ensure you won't run out of memory until the task is done.
Yes I am. And the ones I'm referencing clearly show that ballooning memory is not equal to higher latency, as you claimed. You might say there is a correlation but those are due to other factors and not amount of memory used.
Ballooning memory can be best for real time, if you can afford it: allocation becomes trivial, reclamation is absent.
I remember a story about a rocket which had a range limitation due to the amount if RAM onboard because it never freed any memory. When the engine of the rocket was upgraded, they simply doubled the RAM to let the firmware work for a longer time.
It's not "unpredictable" as in "stochastic", but rather as in "hard to reason about".
This may be compiler-specific. Say, Haskell does not give you much heads-up on laziness: whether stream fusion actually kicks in where you expect it to, or are you encountering quadratic behavior due to laziness [1].
If Ferret dives you decent visibility into issues like these, laziness may be usable even in hard-realtime setting, because it would become obvious and predictable for the developer.
I can talk a lot how disturbingly, distressingly wrong lazy IO is
theoretically, how it breaks all equational reasoning. Lazy IO entails
either incorrect results or poor optimizations. But I won’t talk about
theory. I stay on practical issues like resource management. We don’t
know when a handle will be closed and the corresponding file
descriptor, locks and other resources are disposed. We don’t know
exactly when and in which part of the code the lazy stream is fully
read: one can’t easily predict the evaluation order in a non-strict
language. If the stream is not fully read, we have to rely on unreliable
finalizers to close the handle. Running out of file handles or database
connections is the routine problem with Lazy IO. Lazy IO makes error
reporting impossible: any IO error counts as mere EOF.
It becomes worse when we read from sockets or pipes. We have to be
careful orchestrating reading and writing blocks to maintain
handshaking and avoid deadlocks. We have to be careful to drain the
pipe even if the processing finished before all input is consumed. Such
precision of IO actions is impossible with lazy IO. It is not possible to
mix Lazy IO with IO control, necessary in processing several HTTP
requests on the same incoming connection, with select in-between.
I have personally encountered all these problems. Leaking resources is
an especially egregious and persistent problem. All the above
problems frequently come up on Haskell mailing lists.
Clojure is evaluated eagerly but has lazy data structures. That means you can reason about performance, but you get to work with data in a more functional way than you otherwise could. I imagine ferret follows that example.
Neat, but ferret? It's going to run off with your variables and stash them in a cache in the corner. Every time I've moved, I've found little hoards of something or another from various ferrets I've had. In broadly chronological order:
- Eowyn stole computer parts in anti-static bags and kept them in the cage. I gave up trying to keep SATA cables anywhere but her stash, because she was better at finding them in boxes than I was. Wouldn't bother other stuff, but if it was in an anti-static bag, it was hers.
- Feromir and Eowyn just disagreed on where the scarf went - downstairs, or up under the couch. Unfortunately for Eowyn, mass matters for traction, so when the game got less fun, it went under the couch.
- Blanco... stole potpourri. Not even sure where he found some of it, but we found a couple little stashes of it when we moved.
- Frodo had a thing for stuffed animals, and would either relocate them under a corner table if they were small enough, or to a different corner if they were too large. It's really quite funny watching a ferret struggle with a stuffed animal several times their size.
They're wonderfully quirky little animals, but definitely not a name I'd use for a realtime language. If it involved finding and hiding things, though, totally a suitable name!
The problem with these alternative programming environments is that the developers push off everything other than compiling code. Then the community is left to struggle for years with:
1. How to debug in the language that they write in.
2. How to write unit tests.
3. How to debug unit tests.
4. Common IDE and code-editor integration for 1, 2 and 3.
5. What's the package management story?
For example, the newer programming environment that's released in most recent history that attempts to do an excellent job at this, Deno, still struggles with basics such as IDE integration for debugging unit tests. It's a struggle.
I've heard the creator of Clojure all but say he has put no thought into package management. Nice language, but it's not worth the cost of having to change how the rest of us work when there's existing environments that address the very basics that a developer needs.
> I've heard the creator of Clojure all but say he has put no thought into package management. Nice language, but it's not worth the cost of having to change how the rest of us work when there's existing environments that address the very basics that a developer needs.
Clojure relies on maven for this. Maven is a pretty accomplished package manager in the Java ecosystem. If Rich Hickey said that I assume he meant that he did not _have_ to invent a package manager as it comes for free from the underlying system.
To add, on Clojure you can also easily use git repos and local directories specified in deps.edn - these tend to get used mostly for private/internal deps.
ClojureScript side mostly uses the Maven mechanism for getting ClojureScript deps from Clojars (a Maven repo), but can also import Node modules for interop.
32 comments
[ 3.3 ms ] story [ 79.1 ms ] thread> This document is not intended to be a full lisp tutorial. It is a specification of the subset of lisp implemented by Ferret, and the particular workings of the Runtime and Core library. Any getting started guide for Clojure should get you upto speed on Ferret.
From the outline on the site: https://ferret-lang.org/#outline-container-sec-2
Last issue opened was over a year ago. It is still open, no replies.
www.ulisp.com
- https://github.com/nakkaya/ferret/blob/master/ferret.org
- https://en.wikipedia.org/wiki/Literate_programming
How does Ferret address that?
Also I'm pretty sure Ferret is strict, since it is not mentioned on their web page.
edit: it's mentioned in the "overview" section, probably the OP link pointed there[0].
[0] : https://ferret-lang.org/#outline-container-sec-2
Well, it's an empirical observation that Haskell programmers, however experienced they may be, are constantly surprised by the memory footprint of their programs — IIRC sometimes even the GHC's core developers get blindsided.
One of the ways to get lower latency is in fact to never deallocate and just ensure you won't run out of memory until the task is done.
I remember a story about a rocket which had a range limitation due to the amount if RAM onboard because it never freed any memory. When the engine of the rocket was upgraded, they simply doubled the RAM to let the firmware work for a longer time.
This may be compiler-specific. Say, Haskell does not give you much heads-up on laziness: whether stream fusion actually kicks in where you expect it to, or are you encountering quadratic behavior due to laziness [1].
If Ferret dives you decent visibility into issues like these, laziness may be usable even in hard-realtime setting, because it would become obvious and predictable for the developer.
[1]: https://accidentallyquadratic.tumblr.com/post/114149142587/r...
An aside, but I remembered seeing something that let you test for that, likely https://hackage.haskell.org/package/inspection-testing
My favorite slide:
I can talk a lot how disturbingly, distressingly wrong lazy IO is theoretically, how it breaks all equational reasoning. Lazy IO entails either incorrect results or poor optimizations. But I won’t talk about theory. I stay on practical issues like resource management. We don’t know when a handle will be closed and the corresponding file descriptor, locks and other resources are disposed. We don’t know exactly when and in which part of the code the lazy stream is fully read: one can’t easily predict the evaluation order in a non-strict language. If the stream is not fully read, we have to rely on unreliable finalizers to close the handle. Running out of file handles or database connections is the routine problem with Lazy IO. Lazy IO makes error reporting impossible: any IO error counts as mere EOF. It becomes worse when we read from sockets or pipes. We have to be careful orchestrating reading and writing blocks to maintain handshaking and avoid deadlocks. We have to be careful to drain the pipe even if the processing finished before all input is consumed. Such precision of IO actions is impossible with lazy IO. It is not possible to mix Lazy IO with IO control, necessary in processing several HTTP requests on the same incoming connection, with select in-between. I have personally encountered all these problems. Leaking resources is an especially egregious and persistent problem. All the above problems frequently come up on Haskell mailing lists.
- Eowyn stole computer parts in anti-static bags and kept them in the cage. I gave up trying to keep SATA cables anywhere but her stash, because she was better at finding them in boxes than I was. Wouldn't bother other stuff, but if it was in an anti-static bag, it was hers.
- Feromir and Eowyn just disagreed on where the scarf went - downstairs, or up under the couch. Unfortunately for Eowyn, mass matters for traction, so when the game got less fun, it went under the couch.
- Blanco... stole potpourri. Not even sure where he found some of it, but we found a couple little stashes of it when we moved.
- Frodo had a thing for stuffed animals, and would either relocate them under a corner table if they were small enough, or to a different corner if they were too large. It's really quite funny watching a ferret struggle with a stuffed animal several times their size.
They're wonderfully quirky little animals, but definitely not a name I'd use for a realtime language. If it involved finding and hiding things, though, totally a suitable name!
For example, the newer programming environment that's released in most recent history that attempts to do an excellent job at this, Deno, still struggles with basics such as IDE integration for debugging unit tests. It's a struggle.
I've heard the creator of Clojure all but say he has put no thought into package management. Nice language, but it's not worth the cost of having to change how the rest of us work when there's existing environments that address the very basics that a developer needs.
Clojure relies on maven for this. Maven is a pretty accomplished package manager in the Java ecosystem. If Rich Hickey said that I assume he meant that he did not _have_ to invent a package manager as it comes for free from the underlying system.
ClojureScript side mostly uses the Maven mechanism for getting ClojureScript deps from Clojars (a Maven repo), but can also import Node modules for interop.