Author here — I work on Tesseract at Pasteur Labs, and I wrote this up because the "what if this was possible" was bugging me for way too long :)
I was surprised by how well this worked, the LFortran + Enzyme stack seems to be a very clean way to get gradients through Fortran code via LLVM IR transformations. Pretty cool to see a 220-line Fortran heat solver turn into ~6,900-line reverse pass automatically if I dare say so.
Would be awesome to see this applied to a real scientific codebase, and I hope that the demo is enough to convince people that it’s worth trying.
Lots of scientific code in Fortran has sparse arrays, so a NxN array that only has values on 5 diagonals will store that as 5xN array to save memory allowing you to run a larger problem.
I would also like to know this. Fortran itself is column-major, so I would guess the internal layout isn't same for multi-dimensional arrays when compared to row-major C? I'm not sure how LFortran represents arrays internally though.
LFortran internally uses column-major, so interchanging data with C should be done carefully for multi-dimensional arrays.
If row-major representation is highly needed feature, We can introduce a flag to do that. I'm not totally sure about that but it's doable under some conditions for sure.
We can internally do both row-major and column-major arrays. For Fortran we enable the column-major flag by default, but the door is open to have both at the same time.
Very interesting stuff. How would I get GPU offload working? I have a rather complex scientific code I'm working on with JAX. Most of it can be expressed well with JAX's programming model, but the last 10% really sucks. It's still worth it so I don't have to mess around with offload onto whatever XPU flavor of the week. But going to C++ would really make my life easier, as long as I could use e.g. Kokkos.
Really nice work. I love Enzyme, and used it in my project about differentiable atomic descriptors. Idea was that I can quickly gobble up existing C++ and fortran codes alike for atomic descriptors and create a encompassing library what differentiate against hyper-params as well! But at time Enzyme was very early ~0.0.50 version or so. In our observations also Enzyme was fast enough that performance wise it matched the analytical gradients (when embedded inside entire pipeline)  .
Great work, I am glad LFortran worked well for this work. We are very close to beta quality now, many codes just work, pretty much all Fortran features are implemented now, but some codes still don't compile due to various small bugs, so we've been fixing them all in the last couple months.
15 comments
[ 0.25 ms ] story [ 34.2 ms ] threadI was surprised by how well this worked, the LFortran + Enzyme stack seems to be a very clean way to get gradients through Fortran code via LLVM IR transformations. Pretty cool to see a 220-line Fortran heat solver turn into ~6,900-line reverse pass automatically if I dare say so.
Would be awesome to see this applied to a real scientific codebase, and I hope that the demo is enough to convince people that it’s worth trying.
A shared layout and a shared calling convention would be very nice.
Sorry about my naive question. Haven't touched Fortran directly in three decades I think.