I want to use OCaml for professional development but have always been held back because of hearsay around tooling and libraries that cast doubt on the reasonableness of the choice. Care to advocate for that? (No need to sell the language itself; I like it.)
Not the original poster but I think the best answer to that would be from a talk DHH (the guy who made rails) gave about language choice.
He pointed out out that when he started his company there was no rails, or any great support for writing a server in ruby. He pointed out that it's actually surprisingly simple to get a barebones version of whatever library you like done in your preferred language. The original working version of rails was less than 2000 lines of code. Sure, there are no bells and whistles, but that's how rails started, and look where it is right now.
Compared to the ruby ecosystem in 2004, the Ocaml ecosystem actually pretty full of great libraries. Yeah, it's still tiny in comparison to the big server languages, but it's more than enough to build reliable systems on top of.
The ecosystem is fragmented between four competing "standard libraries" that overlap in functionality but each one has some feature the other doesn't have. So you can imagine the mess and frustration all this causes.
My dated impression is that you had to choose between Jane Street or the alternative. And the concern was that it didn't seem clear how to make this decision without first investing significantly in the language. So this possibility of 'choosing the wrong horse', in my mind, made it a more costly proposition.
The story isn't as dire as it sounds like. Yes there are multiple stdlib alternatives, but I wouldn't classify any of them as a "wrong" choice. They interoperate pretty well, and using one stdlib alternative will not block you from consuming libraries where the maintainer chooses another stdlib alternative.
I tend to look at this situation as there are multiple really high quality options to pick from. Base [1], and core [2] and containers [3] are the three options I typically recommend people to look into (if they are interested in options outside of the stdlib), and all three are really high quality. Your decision could end up being as easy as deciding which platforms are important for you, if you don't care about windows and need to work with unix apis then core or containers are excellent options. If you do care about windows, then starting with Base/Containers is an excellent choice.
* Core can be seen as an extension to base as it has a very similar API (It re-exports many modules from base), and it adds some unix related utilities and additional data structures.
* The built in Stdlib in OCaml has also been improving and if you are able to use a recent compiler version you might not feel the need for an stdlib alternative.
* Containers is an excellent "default" option as it tries to provide a similar API as the stdlib and works on all major platforms supported by OCaml.
With the caveat that I've only worked on toy projects, I've found the built-in stdlib to be perfectly adequate. I supplement with some well-known modules from dbuenzli@ and haven't felt like I was missing any basic functionality.
dune is quite weird compared to many other build tools, but once you learn how it works it's actually quite simple. opam is a pain in the ass but it's definitely being improved steadily and I've had better experiences with it recently. janestreet core seems more stable now than it has in the past but there are occasionally still some breaking changes. They aren't hard to keep up with if you update with each core release though (only like ~1 for my project in the past year or so). There's also a community movement to work a lot more on the ocaml platform and I'm super excited for this work [1] to get through the pipeline! Especially promising is the work on replacing ocamldoc (which sucks) with odoc (which looks great!). While I think work on the ocaml platform will take some time to come to fruition, the approach they are going for is super empathetic and I wholeheartedly agree with their approach for every facet of the ocaml ecosystem -- build tools, doc generation, lsp, editor integration, opam evolution, and all the stability guarantees they're gradually adding. Really exciting time to invest in ocaml!
Definitely don't use it if you need Unicode in any capacity. Also, there seem to be several standard libraries. Tooling is indeed quite confusing compared to other languages. The language itself seems good, though.
It's pretty close. There have been PRs landing to close the gap with Multicore for a couple of releases now.
You should see an OCaml 5.0 branch (which brings parallelism via Domains) in the next couple of months. The time between that branching and release depends a lot on how the review process for the Multicore patches go though.
I'm not aware of large standalone projects that depend solely on it. Multicore is backwards compatible with stock OCaml though and 4.12 ships with some stub Atomics (https://github.com/ocaml/ocaml/blob/4.12/stdlib/atomic.mli) which makes writing programs that can work on both easier.
We also maintain some projects on top of Multicore that are pretty interesting:
* eioio - very much a work in progress but enables high performance direct-style IO with fibers and effects. Backs on to io_uring on Linux and is designed to also work with IOCP on Windows: https://github.com/ocaml-multicore/eioio
I'm very excited for this! I remember seeing a presentation a few years ago that mentioned that OCaml could use effect handlers to implement preemption in fibers. I can't seem to find that presentation, but I believe it mentioned being able to interrupt a fiber with an effect. Is that something that is in the plans or was that just an early idea?
Effects that 'interrupt' a fiber are asynchronous effects. They can be used to implement schedulers and the like. We don't currently support them in Multicore, there's still a few open research questions as to how to reconile them with the effect system.
Cool! Glad to hear there is still work being done there. I was able to find the presentation I was thinking of: https://kcsrk.info/slides/handlers_edinburgh.pdf
Specifically the part about Preemptive Multithreading
It was my understanding that OCaml without a global runtime lock had existed for quite a while and could be optionally enabled, but in practice only made things slower as it significantly slowed down garbage collection. — was my understanding wrong?
There are also two functions on the Obj module which aren't supported because they're unsafe to do concurrently. These are clearly marked as "not for the casual user" so their use is relatively rare.
With regards to performance, you can find a fairly detailed analysis in our ICFP 20 paper: https://arxiv.org/abs/2004.11663 In short, the impact is 3-4% across those benchmarks.
I installed one of the experimental compilers a couple months ago and rewrote a shell tool with effects. It worked. And I loved it. I can't wait. I would have kept using the experimental compiler, but there was compatibility issues with the rest of the ecosystem. The most glaring one was I couldn't use any preprocessing. Nonetheless, inspiring stuff.
29 comments
[ 0.26 ms ] story [ 79.6 ms ] threadLibrary-wise we use bap and core as our main things. The documentation of both is rather hard to traverse through in my opinion, but I get by.
Otherwise depends on what your domain is. The owl ecosystem looks fun https://ocaml.xyz/book/ .
He pointed out out that when he started his company there was no rails, or any great support for writing a server in ruby. He pointed out that it's actually surprisingly simple to get a barebones version of whatever library you like done in your preferred language. The original working version of rails was less than 2000 lines of code. Sure, there are no bells and whistles, but that's how rails started, and look where it is right now.
Compared to the ruby ecosystem in 2004, the Ocaml ecosystem actually pretty full of great libraries. Yeah, it's still tiny in comparison to the big server languages, but it's more than enough to build reliable systems on top of.
My dated impression is that you had to choose between Jane Street or the alternative. And the concern was that it didn't seem clear how to make this decision without first investing significantly in the language. So this possibility of 'choosing the wrong horse', in my mind, made it a more costly proposition.
I tend to look at this situation as there are multiple really high quality options to pick from. Base [1], and core [2] and containers [3] are the three options I typically recommend people to look into (if they are interested in options outside of the stdlib), and all three are really high quality. Your decision could end up being as easy as deciding which platforms are important for you, if you don't care about windows and need to work with unix apis then core or containers are excellent options. If you do care about windows, then starting with Base/Containers is an excellent choice.
[1] https://opensource.janestreet.com/base/
[2] https://opensource.janestreet.com/core/
[3] https://c-cube.github.io/ocaml-containers/
Some additional notes:
* Core can be seen as an extension to base as it has a very similar API (It re-exports many modules from base), and it adds some unix related utilities and additional data structures.
* The built in Stdlib in OCaml has also been improving and if you are able to use a recent compiler version you might not feel the need for an stdlib alternative.
* Containers is an excellent "default" option as it tries to provide a similar API as the stdlib and works on all major platforms supported by OCaml.
[1] https://www.youtube.com/watch?v=E8T_4zqWmq8
You should see an OCaml 5.0 branch (which brings parallelism via Domains) in the next couple of months. The time between that branching and release depends a lot on how the review process for the Multicore patches go though.
We also maintain some projects on top of Multicore that are pretty interesting:
* Domainslib - channels and task queue that makes using parallelism via Domains much easier: https://github.com/ocaml-multicore/domainslib
* eioio - very much a work in progress but enables high performance direct-style IO with fibers and effects. Backs on to io_uring on Linux and is designed to also work with IOCP on Windows: https://github.com/ocaml-multicore/eioio
:) #ocam.el
thanks for answering (and for multicore)
Effects that 'interrupt' a fiber are asynchronous effects. They can be used to implement schedulers and the like. We don't currently support them in Multicore, there's still a few open research questions as to how to reconile them with the effect system.
I believe so. There hasn't been any ocaml release where you can opt in to removing the global runtime lock.
One issue is C libraries that use so-called naked pointers but these are discouraged anyway and 4.12 ships a detector to find these: https://discuss.ocaml.org/t/ann-a-dynamic-checker-for-detect...
There are also two functions on the Obj module which aren't supported because they're unsafe to do concurrently. These are clearly marked as "not for the casual user" so their use is relatively rare.
With regards to performance, you can find a fairly detailed analysis in our ICFP 20 paper: https://arxiv.org/abs/2004.11663 In short, the impact is 3-4% across those benchmarks.
You can grab the current multicore compiler on https://github.com/ocaml-multicore/ocaml-multicore and give it a spin to see, today.