Anyone cross-compiling OCaml with OpenEmbedded and Yocto? The security properties are attractive for IoT and embedded, but it's an ongoing challenge to build the OCaml toolchain from scratch and build apps without leakage of pre-compiled, native, bootstrap toolchain binaries.
With the rise of Arm and RISC-V, it would be nice to see the OCaml community contributing to an upstream OE/YP layer.
Cross architecture compilation is still not a breeze and not a first class feature (there are toolchains for android and ios, but you still can't build a generic toolchain easily as you could with Rust).
Though we are going this way slowly but steadily. Afaik some people are working on build system, so I hope soon we'll have a GCC/Rust-like experience.
I have some tools that you can use to cross compile pretty easily, not a first class yet but it works, you can build generic toolchains quite easily, just a new `sysroot` with the compiler + some flags
This is indeed on our development list for this winter to integrate more. Over in MirageOS (mirage.io), we've been working hard on getting RISC-V support in shape, so we can build unikernels for bare metal embedded devices with a simple "dune build @riscv-solo5".
As part of that work, a few things are going on:
- use "dune workspaces" (essentially a cross compilation context) to build all C bindings in OCaml packages with custom CFLAGS/LDFLAGS. Ongoing work here: https://github.com/mirage/mirage/pull/1153
- upstream in OCaml, the core developers have specced out the design for integrating cross-compilation more naturally into the compiler.
- OCaml 4.11.0 has just been released with a native code RISC-V backend (and of course has had ARM for years).
Mirage doesn't really use OpenEmbedded or Yocto, but I expect it should be straightforward to embed the workflow in there once it's stable upstream in Mirage itself. Any pointers to something I should read about how to get involved in Yocto?
There's a non-upstream OpenEmbedded layer for building OCaml components, but it jumps through hoops to coordinate OCaml toolchain with upstream OE releases, https://github.com/OpenXT/meta-openxt-ocaml-platform
This could be generalized to create an upstream OE layer that uses the upcoming cross-compiling improvements from upstream Ocaml, which would then enable MirageOS unikernels in OE meta-virtualization, https://layers.openembedded.org/layerindex/branch/master/lay.... RISC-V support is in early stages for meta-virt, Western Digital added some support.
Very handy, thanks. I notice that the openxt-ocaml-platform repo uses ocamlbuild/ocamlfind. Life should get a lot easier when we move that to use ocaml/dune directly -- cross-compilation really needs a lot of help from the build system, and dune is moving fast to support these workflows.
Hopefully there can be a way to use ocaml/dune directly without importing "everything" into each per-component build tree. OE best practices employ per-recipe sysroots for building each component, where a minimal set of dependencies is present for the build environment. This helps with reproducible builds and generation of manifests for supply chain integrity.
Yes that should be fine. For incremental installation, there is a `dune build -x` which builds using a particular cross-compilation toolchain. MirageOS builds everything in one go, but it can be broken up too (as opam itself does).
We ended up building an ARM image with the OCaml compiler installed, running it with qemu to build our OCaml code as a second step.
Basically from yocto have a "rootfs" which is your standard image + gcc, use it to build a docker container or chroot in it, add qemu-arm-static, build the OCaml compiler from opam and then use it to build binaries. Not the most straightforward, but it works... fine. And in the end it's just a docker image like any other that is used to build our binaries.
With that said this has been running for a few years, I didn't check if upstream OCaml cross-compiling made progress since then.
Anyone have experience using OCaml on Windows? I do some development for a cross-platform project where researchers wrote one of the components in OCaml. Installing this particular component on Windows is a giant pain requiring the user to first install Cygwin + a couple Cygwin tools. Is this typical for OCaml projects on Windows or do you think it'd be easy/possible to streamline this install process?
Started to learn Ocaml with the help of a friend with professional level OCaml, and from his own words, the experience on Windows is terrible and I can confirm!
OCaml can generate binaries that do not depend on Cygwin runtimes, and that's a common way to ship binaries. There are a number of development shops who have been doing that for years.
For development on Windows, Cygwin is commonly used for "casual" use, and can be tricky to get working if you're not familiar with the quirks of that toolchain. That's going to improve quite rapidly though: I just announced that our development focus from the OCaml Platform for opam (our package manager) is to get Windows support working end-to-end.
There aren't any real blockers to Windows support except for building a critical mass of developers to smooth out the thousands of details that get in the way right now: you can find more here: https://github.com/ocaml/opam/wiki/opam-2.2-slides.pdf
From a compiler developer's perspective, the number of variations of toolchains available on modern Windows is absolutely mindboggling. I'm looking forward to switching to Windows 10 from my OpenBSD desktop and getting familiar with all of this. Last time I seriously used Windows (95), the desktop eSheep app all the rage. I'm informed that it is now available for Windows 10: https://www.microsoft.com/en-us/p/esheep-64bit/9mx2v0tqt6rm
opam-repository-mingw does need Cygwin for development, but the resulting binaries won't depend on it. I was able to port a CLI application with a lot of dependencies without much trouble.
Alternatively, Esy supports Windows and might provide you with a smoother experience. Esy is an alternative to opam cli tool that feels more like npm and supports OCaml projects and opam packages.
On Windows the easiest is probably through reasonml. You get two runtimes out of the box but you are a few releases behind (depending on Bucklescript). The alternative is to use WSL. Ubuntu stock opam is out of date though. The distribution side of ocaml is definitely less than desirable, which is why I only use it to target js. Reasonml as js frontend is hard to beat experience wise.
The modern OCaml tooling (opam for package management, dune for building) is impressive.
It all feels very professional and industry-grade now. Really comprehensive and well-designed CLIs, proper man pages for all subcommands, and sweating the details [1]
[1] My litmus test for package managers is, when listing packages installed in an environment, is there a flag to only list packages that were explicitly installed? Many P-Ms fail this.
The workshop is hosted by the ICFP conference so you need to register there, which is probably too late. But all the videos are available online and you can post questions in the OP link to Discuss, we will try to forward it to the workshop and later post the Q&A session on our channel (so subscribe)
https://www.youtube.com/watch?v=E8T_4zqWmq8&list=PLKO_ZowsIO...
25 comments
[ 2.6 ms ] story [ 55.7 ms ] threadHe demonstrates Language Server Protocol LSP in VSCode
With the rise of Arm and RISC-V, it would be nice to see the OCaml community contributing to an upstream OE/YP layer.
Though we are going this way slowly but steadily. Afaik some people are working on build system, so I hope soon we'll have a GCC/Rust-like experience.
https://github.com/EduardoRFS/reason-mobile
As part of that work, a few things are going on:
- use "dune workspaces" (essentially a cross compilation context) to build all C bindings in OCaml packages with custom CFLAGS/LDFLAGS. Ongoing work here: https://github.com/mirage/mirage/pull/1153
- upstream in OCaml, the core developers have specced out the design for integrating cross-compilation more naturally into the compiler.
- OCaml 4.11.0 has just been released with a native code RISC-V backend (and of course has had ARM for years).
Mirage doesn't really use OpenEmbedded or Yocto, but I expect it should be straightforward to embed the workflow in there once it's stable upstream in Mirage itself. Any pointers to something I should read about how to get involved in Yocto?
This could be generalized to create an upstream OE layer that uses the upcoming cross-compiling improvements from upstream Ocaml, which would then enable MirageOS unikernels in OE meta-virtualization, https://layers.openembedded.org/layerindex/branch/master/lay.... RISC-V support is in early stages for meta-virt, Western Digital added some support.
Intro to meta-virt: https://youtube.com/watch?feature=emb_title&v=fIbWvrlWJYg
Intro to Yocto: https://elinux.org/images/a/a8/Getting_Started_with_Embedded...
Meta-virt mailing list would be a good place for discussion, https://lists.yoctoproject.org/g/meta-virtualization
Basically from yocto have a "rootfs" which is your standard image + gcc, use it to build a docker container or chroot in it, add qemu-arm-static, build the OCaml compiler from opam and then use it to build binaries. Not the most straightforward, but it works... fine. And in the end it's just a docker image like any other that is used to build our binaries.
With that said this has been running for a few years, I didn't check if upstream OCaml cross-compiling made progress since then.
However, I found this link very useful to have something working using WSL: https://www.cosmiccode.blog/blog/vscode-for-ocaml/
For development on Windows, Cygwin is commonly used for "casual" use, and can be tricky to get working if you're not familiar with the quirks of that toolchain. That's going to improve quite rapidly though: I just announced that our development focus from the OCaml Platform for opam (our package manager) is to get Windows support working end-to-end.
There aren't any real blockers to Windows support except for building a critical mass of developers to smooth out the thousands of details that get in the way right now: you can find more here: https://github.com/ocaml/opam/wiki/opam-2.2-slides.pdf
From a compiler developer's perspective, the number of variations of toolchains available on modern Windows is absolutely mindboggling. I'm looking forward to switching to Windows 10 from my OpenBSD desktop and getting familiar with all of this. Last time I seriously used Windows (95), the desktop eSheep app all the rage. I'm informed that it is now available for Windows 10: https://www.microsoft.com/en-us/p/esheep-64bit/9mx2v0tqt6rm
Alternatively, Esy supports Windows and might provide you with a smoother experience. Esy is an alternative to opam cli tool that feels more like npm and supports OCaml projects and opam packages.
https://fdopen.github.io/opam-repository-mingw/
https://esy.sh/en/
https://github.com/esy-ocaml/hello-ocaml
I have a bash shell, launch VSCode from it (with `code`), and all my extensions (ocp-indent, merlin) run in the linux subsystem and talk to VSCode.
[1] https://docs.microsoft.com/en-us/windows/wsl/install-win10
It all feels very professional and industry-grade now. Really comprehensive and well-designed CLIs, proper man pages for all subcommands, and sweating the details [1]
[1] My litmus test for package managers is, when listing packages installed in an environment, is there a flag to only list packages that were explicitly installed? Many P-Ms fail this.
https://forum.rescript-lang.org/t/rescript-frequently-asked-...