Covariant and contravariant indices would be the formal terms. I'm not really sure whether I've seen "upstairs" written down. Sub/superscript... strike me as the typographical terms, not the meaning? Like…
This doesn't really help with programming, but in physics it's traditional to use up- and down-stairs indices, which makes the distinction you want very clear. If input x has components xⁿ, and output f(x) components…
That's what Reactant.jl is aiming to do: Take the LLVM code from Julia and pipe to to XLA, where it can benefit from all the investment which make Jax fast. Same author as the more mature Enzyme.jl, which is AD done at…
This is also how it works in Julia, where macros digest notation for einsum-like operations before compile-time. In fact the linked file's explanatory comment: (einsum (A i j) (B i k) (C k j)) results in the the updates…
using Unitful: m; foo(2.0m) With the above definition, this will give DimensionError: 2.0 m and 1.0 are not dimensionally compatible. Probably this means you should define `foo(x::Number) = x + oneunit(x)` to respect…
Yes, it obeys enough algebraic laws that calling it a derivative is useful. But I don't think there's any underlying notion of small changes to something continuous. It is not the slope of some smooth function.
Or perhaps better "foreigner", legal alien, someone from over the border. The alien derivative is an operation which moves you from one sector to another. A bit like crossing a branch cut.
I presume so. Note that this pseudo-code is very nearly valid Julia code (with my package): @tullio (min) C[i,j] := A[i,k] + B[k,j]
Yes, although that seems like the easy half of this, making sure `struct NewNum <: AbstractFloat` defines everything. There aren't yet tools for this but they are easy to imagine. And missing methods do give errors. The…
The unicode epsilon isn't in the public API, it's describing the 3rd positional argument. This was added recently, and for some reason the PR (1840) didn't fix the docs, which is bad. The Optimisers.jl version has an…
The struct's field name is `eta`, but this is an internal detail. Its constructor takes a positional-only argument, no public name. The greek letter is used in the documentation. And the reason is that every optimiser's…
> > Learning rate (η): Amount by which gradients are discounted before updating the weights. > so this is already explicit to anyone who reads the documentation. The quibble in the post is about the named parameter. As…
On the GPU, operations which look like matrix multiplication are indeed quite slow. As you say there are many tricks, and Tullio doesn't (right now) know them. For operations which are more like broadcasting it does…
There is a discussion here: https://discourse.julialang.org/t/rust-julia-comparison-post... So far, a bug fix improves it 4700 min -> 150 min, changing the type stored -> 20 min, further optimisations -> 15 min.
Thanks for the references. We're trying to write up a paper about all this work, and (to us) it seems obvious that differential geometry is the right framework. What reverse mode AD talks informally about as "gradients"…
Some points have simple answers: * Support for using @copied s.x = 1 where s is an immutable struct. I believe the standard package for this is this: https://github.com/jw3126/Setfield.jl#usage * Currently macros have…
The big Alcoa press is 50_000 tons, not lbs. There are slightly larger ones now, 60, 75ktons. (22 tons is, of course, force that any rail yard could produce a century ago.)
The Julia translations of these might look as follows: using OffsetArrays cash_flows = zeros(0:max_time) altitude = zeros(Bool, -100:100) for I in eachindex(cash_flows) some_function(cash_flows[I]) end # or in this case…
Sure! The desire for fancier notation is from more complicated examples, the kind where you write elaborate comments to explain why you're permuting 3rd & 4th dim of B to line up with C and part of D.
> until next time you have to read that But how will this tool help? If you put the sample you used in a comment, then your confusion is wondering if that's in sync... and whether the input is like what you wrote. IMO…
Right, NamedDims.jl is the one closest what the article describes. Very simple and light-weight, plays well with others. There is a small zoo of packages attaching also labels along the indices, alla python's xarray.…
You can do this: @_ images |> filter(size(_,1)>100, __) .|> resize(__, 100,100) .|> crop(__, 25,25,75,75) with Underscores.jl. Not exactly `images.first.resize(100, 100).crop(25, 25, 75, 75)` anymore, to point out that…
At 2:39 it's saying that "doing good" and "doing well" are both correct, but mean different things: superman is doing good deeds, and scrooge is doing very well. But what to make of "I am good" isn't addressed (unless I…
Those are all good synonyms for "got" in the context of shooting at things. But none of the others already has a strong meaning in the context of aircraft, and this other meaning does create some confusion, which is why…
That seems strange, if I'm reading it correctly. In the author's packages, integers which label dimensions etc. have gradient `nothing`, but arrays which happen to contain integers do not signal anything: julia>…
Covariant and contravariant indices would be the formal terms. I'm not really sure whether I've seen "upstairs" written down. Sub/superscript... strike me as the typographical terms, not the meaning? Like…
This doesn't really help with programming, but in physics it's traditional to use up- and down-stairs indices, which makes the distinction you want very clear. If input x has components xⁿ, and output f(x) components…
That's what Reactant.jl is aiming to do: Take the LLVM code from Julia and pipe to to XLA, where it can benefit from all the investment which make Jax fast. Same author as the more mature Enzyme.jl, which is AD done at…
This is also how it works in Julia, where macros digest notation for einsum-like operations before compile-time. In fact the linked file's explanatory comment: (einsum (A i j) (B i k) (C k j)) results in the the updates…
using Unitful: m; foo(2.0m) With the above definition, this will give DimensionError: 2.0 m and 1.0 are not dimensionally compatible. Probably this means you should define `foo(x::Number) = x + oneunit(x)` to respect…
Yes, it obeys enough algebraic laws that calling it a derivative is useful. But I don't think there's any underlying notion of small changes to something continuous. It is not the slope of some smooth function.
Or perhaps better "foreigner", legal alien, someone from over the border. The alien derivative is an operation which moves you from one sector to another. A bit like crossing a branch cut.
I presume so. Note that this pseudo-code is very nearly valid Julia code (with my package): @tullio (min) C[i,j] := A[i,k] + B[k,j]
Yes, although that seems like the easy half of this, making sure `struct NewNum <: AbstractFloat` defines everything. There aren't yet tools for this but they are easy to imagine. And missing methods do give errors. The…
The unicode epsilon isn't in the public API, it's describing the 3rd positional argument. This was added recently, and for some reason the PR (1840) didn't fix the docs, which is bad. The Optimisers.jl version has an…
The struct's field name is `eta`, but this is an internal detail. Its constructor takes a positional-only argument, no public name. The greek letter is used in the documentation. And the reason is that every optimiser's…
> > Learning rate (η): Amount by which gradients are discounted before updating the weights. > so this is already explicit to anyone who reads the documentation. The quibble in the post is about the named parameter. As…
On the GPU, operations which look like matrix multiplication are indeed quite slow. As you say there are many tricks, and Tullio doesn't (right now) know them. For operations which are more like broadcasting it does…
There is a discussion here: https://discourse.julialang.org/t/rust-julia-comparison-post... So far, a bug fix improves it 4700 min -> 150 min, changing the type stored -> 20 min, further optimisations -> 15 min.
Thanks for the references. We're trying to write up a paper about all this work, and (to us) it seems obvious that differential geometry is the right framework. What reverse mode AD talks informally about as "gradients"…
Some points have simple answers: * Support for using @copied s.x = 1 where s is an immutable struct. I believe the standard package for this is this: https://github.com/jw3126/Setfield.jl#usage * Currently macros have…
The big Alcoa press is 50_000 tons, not lbs. There are slightly larger ones now, 60, 75ktons. (22 tons is, of course, force that any rail yard could produce a century ago.)
The Julia translations of these might look as follows: using OffsetArrays cash_flows = zeros(0:max_time) altitude = zeros(Bool, -100:100) for I in eachindex(cash_flows) some_function(cash_flows[I]) end # or in this case…
Sure! The desire for fancier notation is from more complicated examples, the kind where you write elaborate comments to explain why you're permuting 3rd & 4th dim of B to line up with C and part of D.
> until next time you have to read that But how will this tool help? If you put the sample you used in a comment, then your confusion is wondering if that's in sync... and whether the input is like what you wrote. IMO…
Right, NamedDims.jl is the one closest what the article describes. Very simple and light-weight, plays well with others. There is a small zoo of packages attaching also labels along the indices, alla python's xarray.…
You can do this: @_ images |> filter(size(_,1)>100, __) .|> resize(__, 100,100) .|> crop(__, 25,25,75,75) with Underscores.jl. Not exactly `images.first.resize(100, 100).crop(25, 25, 75, 75)` anymore, to point out that…
At 2:39 it's saying that "doing good" and "doing well" are both correct, but mean different things: superman is doing good deeds, and scrooge is doing very well. But what to make of "I am good" isn't addressed (unless I…
Those are all good synonyms for "got" in the context of shooting at things. But none of the others already has a strong meaning in the context of aircraft, and this other meaning does create some confusion, which is why…
That seems strange, if I'm reading it correctly. In the author's packages, integers which label dimensions etc. have gradient `nothing`, but arrays which happen to contain integers do not signal anything: julia>…