I'm always a sucker for parallel functional programming, but I could not find any details on how specifically it works (and importantly: performs) in Clio.
Apparently it works by hosting HTTP or Websocket microservices and communicating between them via JSON.
The .clio files transpile to JavaScript, and the project optimistically targets full JS interoperability (e.g. compatible with any of your favorite npm modules).
Thanks to the portability of JSON, clio can import functions from any compatible microservice, including those written in other languages. For an example of this, see: https://github.com/clio-lang/python-microservice-example
I'd like to see a thorough performance evaluation. JS/NPM based untyped language together with communication via HTTP/JSON sounds painfully inefficient.
It seems Clio targets an AWS Lambda-style serverless infrastructure, having a network of short-lived, auto-scaling single-function instances, ideally leading to optimal resource utilization, hands-free administration, unlimited scalability, etc... but trading some performance.
Pure functions are memoized. They always return the same results so why not memoize? There's an option to disable this. It's currently being discussed in our telegram group: https://t.me/joinchat/B0kZo0kVldfXldTDqz95XA
Cheap pure functions should rarely, if ever, be memoized. With a large space of inputs, it is usually much faster to simply execute the function body than to look up the memoized return value. Not to mention the added space complexity of memoization.
This is actually quite difficult to do automatically. A sibling comment lays this out quite nicely.
Because memoization often can affect the asymptotic runtime of algorithms (and of course space usage), this can lead to very confusing behavior for a user, if you determine at runtime similarly to a JIT compiler whether to memoize. Imagine an algorithm's asymptotic complexity jumping back and forth between exponential and linear over the course of a program running as the runtime analyzer memoizes and unmemoizes a function call.
One issue I raised is that you don't want to memoize a function that itself is memoized. Well, if we can detect at runtime that a function is not reused, it does tend to mitigate that issue.
You need to be able to turn it on and off depending on demand.
A bloom filter (though there are more efficient variants out there) can watch the keys set and requested and calculate a predicted hit rate over a sample period. Most variants of bloom filters don't allow dropping keys, but that's not strictly necessary: you can simply sample for a given number of hits and then reset the filter.
If the hit ratio is less than a threshold after some number of hits, it can drop the table and reset the state back to sampling for potential memoization.
Another problem for long running processes is avoiding memory spikes. Ideally you just figure out how much you can allocate and stay within that. (You can also use weak references to mitigate low memory situations; the system can track how often it's being garbage collected and use that to self-suppress.)
A bloom filter naturally has very predictable memory usage, and you can limit the number of pairs a cache will hold. That suggests a global strategy whereby you have a parameter saying, "only memoize the top functions and ensure tables fit in X MB." Tracking the size of memoized values does mean walking the graph; though can always limit a graph walk and refuse to memoize very large objects.
It's adding a bunch of machinery to each function, so it would make sense for the compiler to use some heuristics to eliminate likely poor candidates. One heuristic is whether a function has loops or recursion. Another is whether a function has only a single caller.
It just seems like one of those ideas that really could work and cover a ton of use cases, but in prod you'd want a bunch of tunable parameters and compiler hints, and then all the tooling to make it usable. I think you'd want to sort those requirements back to something simple you can actually code so you can see what people do with it in practice.
> They always return the same results so why not memoize?
You only want to memoize when the lookup table is "cheap" compared to calculating the function. A lookup table is naturally expensive in terms of space, but can also be expensive in terms of computation; hashing is not free.
Any kind of function that is mostly a constructor, accessor or mutator is a bad candidate for memoization. Most of your primitives should probably not be memoized.
And you generally don't want to memoize memoized functions. Either memoize the constituents, if they're called from many places, or the top level function.
And most memoization evicts "old" entries, so it's possible to be waste all those resources putting things into and out of the table without ever getting hits.
But that problem can be far more insidious when it looks like the memoization works. You can get be getting great performance when there are few enough threads that everything fits in the table. Then you get too much load and misses start to increase, get some more load and memoization starts working against you and you get catastrophic failure.
As the siblings have said, it's an extremely situational decision whether to memoize a pure function, and only a tiny number of pure functions benefit from it. There's no way to do it automatically, it has to be programmer choice.
For instance, take the function that concatenates two strings into a third string. That's a pure function, but it would be madness to memoize it. It would mean saving every single string concatenation your program ever does in memory, for no real reason at all.
Or, you know: addition. Taking two numbers and returning the result of adding them together is a pure function. Should you memoize that? Literally every single addition you ever do would have to be saved, and every addition would have to do a hash-table look-up.
We want to provide an easy interface to quickly deploy decentralized services. Compared to alternative languages, the overhead to do so is much smaller with Clio
This is pretty much the first question I ask every time I see a new language pop up.
The fact is the “language” part of a programming language is the easy part. The hard part is the ecosystem, tooling, mindshare, etc.
If your differentiation is “it’s 35% easier to do task A with my new language vs the next best alternative” then I am afraid you’ve lost me and I suspect most potential users.
The effort involved in learning your new system isn’t going to be amortized out by the benefits unless the benefits are very substantial.
This is coming from someone who really lives to learn new languages and is spending time most everyday working through new problems in new languages out of personal interest. For your average “just wanna get the job done” type you have an even more uphill battle.
Having said all that, it can be a great learning experience to build a new language and it’s always possible you’ll influence changes in preexisting systems that do have an existing user base, so I don’t mean to sound too discouraging to aspiring language designers.
We discussed the problems with the ecosystems and I think you are right. Clio compiles to JavaScript, so it only makes sense to be 100% interoperable with the npm modules out there. We are currently working on that.
The fact is the “language” part of a programming language is the easy part. The hard part is the ecosystem, tooling, mindshare, etc.
This is certainly true, but on the other hand, any new language that will one day be successful needs to start somewhere, or we're going to be writing systems code in bug-ridden C and line-of-business applications in verbose Java forever.
A language where doing day-to-day things would be 35% easier than what I use right now is certainly significantly better, IMHO. It seems to me that to move our industry forward by adopting newer and significantly better languages then a new language probably needs some or all of:
1. easy compatibility with a major existing library ecosystem (a simple FFI to any library with a C interface, for example) or a very comprehensive set of standard libraries out of the box and some tolerable way to integrate with more specialised ones
2. compatibility with existing tools or a decent version of the essential tools of its own (where "essential" today probably means something like build system, package manager, debugger, and maybe profiler, plus usable support in editors and things like version control and diff tools)
3. a compelling use case where it is much better than anything out there today, to act as a starting point for everything else to attach to
4. a core group of initial developers who are interested enough to do things with the language and help develop the above.
Yes, any new language needs to start somewhere, but it’s interesting to think we need new languages for existing domains and that they have any chance of success to begin with.
Consider that by many accounts JavaScript, Java, and Python basically dominate in terms of number of developers and mindshare. These languages all arrived 25-30 years ago. I know some rankings claim C/C++ are far more popular than JavaScript but I just haven’t seen evidence of that in recent years.
Other widely used languages include C, Objective-C, C++, C# and Ruby. The first three are older than the aforementioned languages, and Ruby is from the same era as Java/JavaScript, and C# isn’t far behind that.
In terms of newer languages, the only ones I can think of offhand that seem to have serious traction are Swift and to a lesser extent Go. Some people might throw Kotlin in there, but I’m not sure how much actual traction it’s getting. Rust has been gaining traction slowly for ~8+ years and I’m hopeful it has a bright future but I don’t see a lot of actual projects using it today.
The point is that it’s really hard to succeed if your primary reason to exist is to make existing practice slightly better. It’s very hard to displace any of the existing languages in the space of general purpose programming.
You need to have libraries. You need to have at least syntax coloring support for several editors. You need language server support for VS Code. You need to generate Dwarf. You probably need a repl or website that allows interactively playing around with the language to get people started. You need enough mindshare that teams can actually hire from a pool of candidates once they get a project underway. You need a compiler with good diagnostics, books, StackOverflow answers for the top N questions people will run into, bloggers who are enthusiastic, etc.
Someone posted this on HN recently and I think the speaker does a good job of breaking down the challenges and examining why some languages have found success while others (notably functional languages) have languished on the sidelines:
Another way to look at it is that you need a reason for people to use your language, not features. C didn't have tooling to start, JS didn't have an ecosystem, VSCode is only a few years old, so there is the short view, and the long view. It's nice if you live long enough to see your work appreciated, and perhaps even benefit from it. Think true artist versus imitator. Engineers would say "Design for what people think they want, based on what they've seen before, or design for what people need, but they don't know it yet."
Familiarity helps, which is why so many langs follow historical syntactic and semantic rules; and what makes it hard on languages that are either different themselves, or target a niche domain. This includes languages whose paradigm is harder for mere mortals (the vast majority of us) to grasp.
There is also luck and timing. Backing from a big company doesn't hurt, but those langs (Swift and Go) were designed for their owners needs, not everybody else's.
There is a great irony here, which is that many lessons from the past have been forgotten, and ideas which would have help us as developers, and therefore the world, aren't widely used.
Clio doesn't look like my particular cup of tea, but they're trying to solve important problems, and I support them in that.
The point is that it’s really hard to succeed if your primary reason to exist is to make existing practice slightly better. It’s very hard to displace any of the existing languages in the space of general purpose programming.
I don't disagree (though FWIW I do disagree with some of the points you mentioned afterwards as "needs").
This is a real problem, though. If we can't manage to adopt a tool that would make us 50% more productive than what we use today, which was the example scenario under discussion, just because of momentum, then our entire industry is going to languish in substandard results forever. We'll continue churning out code that isn't as reliable as it could be and that doesn't perform as well as it could, at vast cost to our ever more technology-dependent society, all because we couldn't get our act together and learn something new.
I am somewhat more optimistic than you seem to be about the viability of doing this. New languages don't magically become popular overnight, but within the past decade or so, we've seen the likes of Go, Rust and Swift become somewhat established, certainly enough to use for real work and build a significant community and ecosystem. In the world of web development, we've seen incremental but very significant changes in JavaScript, but also serious traction for derivatives like TypeScript and significant interest in more specialised tools like Elm.
It's worth noting that in almost all of these cases, there was a "killer application" for the language that did set it apart from what had gone before. Some of them might have become general purpose languages with time, but usually there was some more specific focus at first while they were building up a critical mass of support.
The linked page doesn't say anything directly about decentralized systems (doesn't use the word "decentralized") and the only tutorial I find uses a centralized server + DB (the to-dos one). Could you explain more concretely? Like does it help with NAT traversal or peering? Very interested, thanks!
With Clio, you can develop modules and host them, so you can import them via network from another server. You can essentially build microservices as clio modules and build up decentralized systems. Feel free to join our telegram group to discuss about this. We love to hear new suggestions! :)
> It is made to take advantage of multiple CPUs and CPU cores (parallelism) by default
Taking advantage of CPUs and CPU cores oes not make a complete parallelism implementation. It should also take advantage of SIMD instructions (SSE, AVX, Neon etc).
Seems to be in the same ballpark as Unison which I got a bit hyped about when it was revealed earlier this autumn. The "memoize" and network foreign function parts is especially same looking, though Unison takes it a bit further. This seems like the more balanced approach to static typed Erlang-ish.
32 comments
[ 3.2 ms ] story [ 82.7 ms ] threadThe .clio files transpile to JavaScript, and the project optimistically targets full JS interoperability (e.g. compatible with any of your favorite npm modules).
Thanks to the portability of JSON, clio can import functions from any compatible microservice, including those written in other languages. For an example of this, see: https://github.com/clio-lang/python-microservice-example
Besides reading the tutorial, I found this source helpful to understanding how Clio knits the microservices together: https://github.com/clio-lang/clio/blob/develop/host/host.js
Because memoization often can affect the asymptotic runtime of algorithms (and of course space usage), this can lead to very confusing behavior for a user, if you determine at runtime similarly to a JIT compiler whether to memoize. Imagine an algorithm's asymptotic complexity jumping back and forth between exponential and linear over the course of a program running as the runtime analyzer memoizes and unmemoizes a function call.
Since you already have a lazy language, you can direct users to use lazy data structures to achieve memoization. http://jelv.is/blog/Lazy-Dynamic-Programming/
One issue I raised is that you don't want to memoize a function that itself is memoized. Well, if we can detect at runtime that a function is not reused, it does tend to mitigate that issue.
You need to be able to turn it on and off depending on demand.
A bloom filter (though there are more efficient variants out there) can watch the keys set and requested and calculate a predicted hit rate over a sample period. Most variants of bloom filters don't allow dropping keys, but that's not strictly necessary: you can simply sample for a given number of hits and then reset the filter.
If the hit ratio is less than a threshold after some number of hits, it can drop the table and reset the state back to sampling for potential memoization.
Another problem for long running processes is avoiding memory spikes. Ideally you just figure out how much you can allocate and stay within that. (You can also use weak references to mitigate low memory situations; the system can track how often it's being garbage collected and use that to self-suppress.)
A bloom filter naturally has very predictable memory usage, and you can limit the number of pairs a cache will hold. That suggests a global strategy whereby you have a parameter saying, "only memoize the top functions and ensure tables fit in X MB." Tracking the size of memoized values does mean walking the graph; though can always limit a graph walk and refuse to memoize very large objects.
It's adding a bunch of machinery to each function, so it would make sense for the compiler to use some heuristics to eliminate likely poor candidates. One heuristic is whether a function has loops or recursion. Another is whether a function has only a single caller.
It just seems like one of those ideas that really could work and cover a ton of use cases, but in prod you'd want a bunch of tunable parameters and compiler hints, and then all the tooling to make it usable. I think you'd want to sort those requirements back to something simple you can actually code so you can see what people do with it in practice.
You only want to memoize when the lookup table is "cheap" compared to calculating the function. A lookup table is naturally expensive in terms of space, but can also be expensive in terms of computation; hashing is not free.
Any kind of function that is mostly a constructor, accessor or mutator is a bad candidate for memoization. Most of your primitives should probably not be memoized.
And you generally don't want to memoize memoized functions. Either memoize the constituents, if they're called from many places, or the top level function.
And most memoization evicts "old" entries, so it's possible to be waste all those resources putting things into and out of the table without ever getting hits.
But that problem can be far more insidious when it looks like the memoization works. You can get be getting great performance when there are few enough threads that everything fits in the table. Then you get too much load and misses start to increase, get some more load and memoization starts working against you and you get catastrophic failure.
For instance, take the function that concatenates two strings into a third string. That's a pure function, but it would be madness to memoize it. It would mean saving every single string concatenation your program ever does in memory, for no real reason at all.
Or, you know: addition. Taking two numbers and returning the result of adding them together is a pure function. Should you memoize that? Literally every single addition you ever do would have to be saved, and every addition would have to do a hash-table look-up.
The fact is the “language” part of a programming language is the easy part. The hard part is the ecosystem, tooling, mindshare, etc.
If your differentiation is “it’s 35% easier to do task A with my new language vs the next best alternative” then I am afraid you’ve lost me and I suspect most potential users.
The effort involved in learning your new system isn’t going to be amortized out by the benefits unless the benefits are very substantial.
This is coming from someone who really lives to learn new languages and is spending time most everyday working through new problems in new languages out of personal interest. For your average “just wanna get the job done” type you have an even more uphill battle.
Having said all that, it can be a great learning experience to build a new language and it’s always possible you’ll influence changes in preexisting systems that do have an existing user base, so I don’t mean to sound too discouraging to aspiring language designers.
This is certainly true, but on the other hand, any new language that will one day be successful needs to start somewhere, or we're going to be writing systems code in bug-ridden C and line-of-business applications in verbose Java forever.
A language where doing day-to-day things would be 35% easier than what I use right now is certainly significantly better, IMHO. It seems to me that to move our industry forward by adopting newer and significantly better languages then a new language probably needs some or all of:
1. easy compatibility with a major existing library ecosystem (a simple FFI to any library with a C interface, for example) or a very comprehensive set of standard libraries out of the box and some tolerable way to integrate with more specialised ones
2. compatibility with existing tools or a decent version of the essential tools of its own (where "essential" today probably means something like build system, package manager, debugger, and maybe profiler, plus usable support in editors and things like version control and diff tools)
3. a compelling use case where it is much better than anything out there today, to act as a starting point for everything else to attach to
4. a core group of initial developers who are interested enough to do things with the language and help develop the above.
Consider that by many accounts JavaScript, Java, and Python basically dominate in terms of number of developers and mindshare. These languages all arrived 25-30 years ago. I know some rankings claim C/C++ are far more popular than JavaScript but I just haven’t seen evidence of that in recent years.
Other widely used languages include C, Objective-C, C++, C# and Ruby. The first three are older than the aforementioned languages, and Ruby is from the same era as Java/JavaScript, and C# isn’t far behind that.
In terms of newer languages, the only ones I can think of offhand that seem to have serious traction are Swift and to a lesser extent Go. Some people might throw Kotlin in there, but I’m not sure how much actual traction it’s getting. Rust has been gaining traction slowly for ~8+ years and I’m hopeful it has a bright future but I don’t see a lot of actual projects using it today.
The point is that it’s really hard to succeed if your primary reason to exist is to make existing practice slightly better. It’s very hard to displace any of the existing languages in the space of general purpose programming.
You need to have libraries. You need to have at least syntax coloring support for several editors. You need language server support for VS Code. You need to generate Dwarf. You probably need a repl or website that allows interactively playing around with the language to get people started. You need enough mindshare that teams can actually hire from a pool of candidates once they get a project underway. You need a compiler with good diagnostics, books, StackOverflow answers for the top N questions people will run into, bloggers who are enthusiastic, etc.
Someone posted this on HN recently and I think the speaker does a good job of breaking down the challenges and examining why some languages have found success while others (notably functional languages) have languished on the sidelines:
Why isn’t functional programming the norm? https://www.youtube.com/watch?v=QyJZzq0v7Z4
kotlin: 161/5224 | go: 148/5224 | swift: 97/5224 | rust: 6/5224
Familiarity helps, which is why so many langs follow historical syntactic and semantic rules; and what makes it hard on languages that are either different themselves, or target a niche domain. This includes languages whose paradigm is harder for mere mortals (the vast majority of us) to grasp.
There is also luck and timing. Backing from a big company doesn't hurt, but those langs (Swift and Go) were designed for their owners needs, not everybody else's.
There is a great irony here, which is that many lessons from the past have been forgotten, and ideas which would have help us as developers, and therefore the world, aren't widely used.
Clio doesn't look like my particular cup of tea, but they're trying to solve important problems, and I support them in that.
I don't disagree (though FWIW I do disagree with some of the points you mentioned afterwards as "needs").
This is a real problem, though. If we can't manage to adopt a tool that would make us 50% more productive than what we use today, which was the example scenario under discussion, just because of momentum, then our entire industry is going to languish in substandard results forever. We'll continue churning out code that isn't as reliable as it could be and that doesn't perform as well as it could, at vast cost to our ever more technology-dependent society, all because we couldn't get our act together and learn something new.
I am somewhat more optimistic than you seem to be about the viability of doing this. New languages don't magically become popular overnight, but within the past decade or so, we've seen the likes of Go, Rust and Swift become somewhat established, certainly enough to use for real work and build a significant community and ecosystem. In the world of web development, we've seen incremental but very significant changes in JavaScript, but also serious traction for derivatives like TypeScript and significant interest in more specialised tools like Elm.
It's worth noting that in almost all of these cases, there was a "killer application" for the language that did set it apart from what had gone before. Some of them might have become general purpose languages with time, but usually there was some more specific focus at first while they were building up a critical mass of support.
What does this even mean? It's non-obvious from the website.
Taking advantage of CPUs and CPU cores oes not make a complete parallelism implementation. It should also take advantage of SIMD instructions (SSE, AVX, Neon etc).