If I understand the helper code part correctly, libraries are banned? That makes the challenge quite pointless, as it doesn’t even pretend to be equivalent to “real world” usage. All it can show is the julia language has it by default... but not whether the language’s design actually leads to something more clean, readable and sufficiently fast than what would “normally” be done in other languages.
It doesn’t really matter if python doesnt have as powerful an array syntax as julia, if I can pull in numpy (a standard, battle-tested library) and produce equivalent (and good) code.
But if I pull in numpy, and still can’t produce code as nice as julia (because of whatever limitations of python as a language), then there’s an argument to be made for switching to julia.
But going purely off what the language gives by its initial installation... its just an arbitrary limitation to give the question one answer.
The article claims that it proves the point ... "This can be unfair since Julia has lots of functionality for array algorithms built in, but also illustrates the point" ... but assuming good intentions, I'm not sure what point that is? At best, you might be comparing APL languages to one another, but nothing broader than that.
The way I have experienced it (with Javascript, Perl, Python and Matlab) is that many casual users give up whenever you mention packaging. Any language that lets people go very far without involving external modules will get significant usage from amateur programmers.
Hell, so do I; I absolutely hate learning new package ecosystems & infrastructure (but I absolutely do not have interest in learning a language without a decent package infrastructure). Probably a bigger blocker for me than learning the language itself!
But it's not clear to me that this "challenge" is intended to say that array operations are easy, for a newcomer to the language. As I understood it, it's an argument for Julia being better in the long run, for the code that requires "developer effort, performance and extensibility"; that is, elements of the language beyond the first few days of using it, for packages larger than what would be written by "amateur programmers"
If depicting usage for the amateur was the main point of the article, it totally went over my head
elsewhere it suggests “Python + Numba” and “Python + C” as possibilities. i have to assume that numpy would be okay. i mean, it’s close to being part of the standard library at this point.
that's exactly why I made that point debatable ;) Feel free to use Numpy, I'd consider that as the Python base array library anyways;)
Note though, that this challenge is aimed at the people, that need to implement libraries like Numpy ;) But if Numpy is a good enough basis to actually create such a new flexible implementation, consider me impressed!
Well, it imports from Base - Anyways, too many people got hung up on this rule, so I deleted it... It wasn't very important anyways, as long as the core implementation is somewhat from scratch.
Then a question for you, as I don't know Julia the Language, how much is required from Base. In other words, if you didn't have the import from Base, what can you not do?
In a language like Unicon/Icon, I can "$include" a source file, I can "link" an ucode (compiled source, unlinked) file, I can "import" a ucode class file, I can put all the source in the same file or I can get the compiler to bring all the files together on the command line and it will basically end up with the same result.
Does Julia provide Base as source to allow you to do anything like the above?
> if you didn't have the import from Base, what can you not do?
So the part that gets loaded when you load the language/compiler is what is referred to as Base. So, it's kind of impossible to not have it and it is part of the language. So it's somewhat like asking, what can you do without "importing" integers? (ok, that's a hyperbole, but you get the idea ;) )
But, the source code does get shipped, so you might actually go in there and change stuff if you want, and Julia will pick up those changes - even, to get back to that hyperbole, how integers are defined, since they're implemented in pure Julia.
I'm not a 100% sure what you're aiming at, but does that answer your question? ;)
Sorry for the tardiness of my response, I have been doing other things including sleeping.
It does answer the question in that it appears that Julia, as a language, doesn't provide you with much to start with, not even the bare bones. I have seen commentaries back in the 1980's that discussed such language designs. It makes for some uniformity but does require much work to be done to create the basic infrastructure that would allow and enable others to use these kinds of languages out of the box. Interesting in some ways, I suppose.
Guess I wrote my answer more ambiguous than I thought: the exact opposite is the case...
Julia Base is huge and includes much more than most other languages I know - and that's what gets loaded on start, whether you like it or not ;)
I like Python for the aesthetic reasons pg summarized so well in The Python Paradox essay.
Despite it has great libraries, I think it's the wrong language for scientific computing. In a nutshell, it's much harder than it should be to write concise and performant code. There's a huge stack of things used to achieve this, which often makes it complex to reason about performance and obscures the underlying mathematics: cpython, NumPy, pandas, Cython, Numba, PyTorch...
Julia is really appealing because it's very close to executable mathematics. And thanks to a few language design decisions, it's very easy to write performant code and reason about it.
For example, Julia ML libraries like Flux have full composability with Julia code. That's a game changer in terms of simplicity. Tensorflow is getting ported to Swift to mimic that feature.
In the beginning the article talks about contestants being things such as "python + numba, Python + C" and later it says:
> "Types & Functions From Different Packages
It's mandatory to use an existing package without changing it. This can also be emulated by putting it in an isolated module not using any constructs defined in our implementation."
I don't know if the article has been revised, but this I read the rules about libraries completely opposite from you.
The rule is about the implementation itself, but the other one is part of the benchmarking code. You'll need to be able to use the implementation, that doesn't rely much on external libraries, with types from other packages. If you read the article, you'll find Julia is all about composing functionalities from different packages. Its a major driving factor behind the ecosystem - and usually not possible if your language relies on c libraries to do the heavy lifting.
Am I the only one who sees a big PR push for Julia? Up until a few weeks ago I'd never heard of it. Not I see it all over, frequently touted as the "one language to rule them all."
It hit 1.0 about a month ago. Its always been a pretty exciting language, but it was hard to seriously use it without stability for an ecosystem to build around.
I think a lot of people had prepared articles, packages etc for the 1.0 release - and now they're slowly getting released ;)
If you're a Julia developer, the release definitely motivates you to put in the extra work and try to release your material around the same time!
Looks like it. Section 4.1 eliminates lower level languages on code size, and 4.2 eliminates higher level languages on performance. The challenge in C form would be:
as i said, if it makes sense, i'd totally allow using external libraries! maybe i should make that clearer since quite a few people get hung up on that
A few of us perceive it as unrealistic, because it's too restrictive. C/C++ are general programming languages and I feel I am flat out eliminated before even starting. So if you didn't target the general programming languages, who did you want to target with the challenge?
Also, heck, it's a challenge, we all want in, please make it so :)
Makes sense. I will try to update the article!
Feel free to use whatever library that helps you to compose your program nicely. It would be nice though, if those libraries were in a set of standard libraries ;) If you need to implement all helpers yourself, my point "in Julia, you can just sit down and start implementing a pretty difficult problem" would become a bit watered down... First of all, Julia is also pretty multi-purpose - and secondly, if you put it to the extreme, you could just write a DSL that you parse in C++ that is perfectly tailored to the problem, and consider the challenge as won :P
Yes but, then we can draw some scientific conclusion.
If I can outperform Julia by 50% in C++, but I have to write 10x code. I won't bother with C++.
If you manage to have that in the end, that's a very valuable promotional information.
For one of my master courses, I had to write 200 lines in python. C++ solution would have been in the thousands. My colleague solved it in 10 R lines. R is on my list now :)
I think the logic is that strlen is virtually free for short strings, and that memcpy can be a lot faster for long ones (reading https://sourceware.org/git/?p=glibc.git;a=blob;f=string/memc..., it even copies “whole pages from SRCP to DSTP by virtual address manipulation, as much as possible”. strlen doesn’t do things the K&R way either anymore. It checks one byte at a time until things are word aligned, then checks words for zero bytes. That can read from past the string, so it requires the platform it runs on to make that OK. See https://sourceware.org/git/?p=glibc.git;a=blob;f=string/strl...)
There is something that puts me off about all these languages that want to show off their own strengths by pointing at other languages weaknesses. X in Y is another such pattern, who cares what 'X' is written in, the question is does it perform well, does it serve a need and is it maintainable and secure if it is network facing or otherwise employed in a situation that would require the program to be locked down properly.
Setting up an artificial challenge with rules that favor 'your' language up front is a great way to get the result that you want, but not a great way to learn. Each language has its strengths and its weaknesses, but none of those strengths and weaknesses matter all that much from the point of view of an end user until they have a direct impact on the quality of that which is produced. And programmers have shown time and again they can write crappy code in just about any language (and good code too...) with the language features having a much smaller impact on the outcome than the quality of the people writing the code.
I must admit, that I find your sentiment very understandable! It is an egoistic challenge, after all.
For me the problems I describe are highly relevant, and I'm very curious about any solution that isn't written in Julia. And I do think, that those challenges are also highly relevant for a large part of the scientific computing community. So whatever language excels at those should get proper recognition;) And I do believe, that if you can implement a hard problem efficiently with little effort and not that much code and still have it composable, that it will very quickly influence the quality and user experience for the end user.
Ok fair enough - maybe this is just my bad english, let me know if you have a better way to phrase it.
I do think, that it is quite showing, that google + tensorflow don't believe in the mix of python + c/c++.
I've been working quite closely on the challenges that faces machine learning libraries at the moment, and I can totally see where they're coming from when they're unhappy with the current solution and want to rewrite it in swift. How should I express this in a better way?
As someone who has used both languages extensively over the last 6 years (Julia for grad school research and Python for my work as a data scientist), I'm always surprised at how quickly people rush to defend Python. "Yes, you can do that in Python too using such-and-such library or with this hack..." Don't get me wrong — the numerous Python libraries are impressive, but at the end of the day, for me personally the experience of using Julia is just far more pleasant than using Python. It seems as though I'm always fighting against Python to accomplish something, whereas with Julia it feels like the language just gets out of the way and lets me do what I want.
Yes, so much this. I want to be able to write simple idiomatic code without having an insanely large (10x-1000x) overhead. That just doesn't happen in Python.
This seems very finely tuned to the Julia language (i.e. relying heavily on meta-programming). It's not clear to me that strong support for meta-programming is actually that necessary for every day use cases.
I'd be curious to see what kinds of implementations someone could come up with in CUDA using templates, though. It could easily blow the Julia performance out of the water.
How could it blow the Julia performance out of the water? Julia is just manipulating the generated LLVM IR, and LLVM generates the .ptx. If C++ and Julia are both using LLVM for this compilation, you'll get to pretty much the same place.
And this is tuned towards package development, specifically generic package development. This is quite a hot topic in scientific computing. The challenge isn't for people who use packages, it's for people who develop packages and are looking to make them generally useful (i.e. not just single project).
You can certainly use a GPU package for Julia with macros for generating ptx code. That's not what the OP did though. He merely optimized for using SIMD, which provides Julia a distinct advantage over other languages with strong metaprogramming, such as C++. But if you care enough about performance that you focus on the performance benefits of SIMD over ordinary execution, you probably should be using the GPU instead. By using CUDA (not C++) as the language, one can get far faster performance right out of the box.
Thus, a much fairer comparison, if we care so much about performance that non-SIMD execution is a non-starter, would be CUDA vs. Julia targeting the GPU.
Note that I absolutely love Julia because I find it far easier to write than a language like C++, but idiomatic Julia code still has low overhead, unlike, say, Python, which introduces orders of magnitude of overhead compared to low level languages, and you have to move away from simple idiomatic code to achieve reasonable performance. However, I find it unreasonable to focus on SIMD performance - the performance gain over non-vectorized code isn't that big, and if you do want to eek out that much extra performance, you should focus on using the GPU.
>But if you care enough about performance that you focus on the performance benefits of SIMD over ordinary execution, you probably should be using the GPU instead.
That's not true. There's a lot of heavily serial algorithms which do not always work well on the GPU unless they are of a very large size, larger than most applications. This happens a lot in optimization and differential equations, two of the pillars of scientific computing.
>Thus, a much fairer comparison, if we care so much about performance that non-SIMD execution is a non-starter, would be CUDA vs. Julia targeting the GPU.
>That's not true. There's a lot of heavily serial algorithms which do not always work well on the GPU unless they are of a very large size, larger than most applications. This happens a lot in optimization and differential equations, two of the pillars of scientific computing.
And those situations tend to get a very minimal speedup from SIMD.
Yes, that's great, and would have made for an interesting challenge, rather than focusing on SIMD instead of serial or GPU performance.
>Where?
I didn't word it well, but I meant to say that Julia doesn't have much overhead compared to C/C++, unlike Python (what overhead it does have comes from garbage collection).
Well, that'd be comparing apples and oranges...
But fun fact: Julia actually has excellent GPU support!
I will soon update the challenge with a GPU accelerated version. Spoiler: it won't need much more than 5 lines of code to get it to the GPU. Good luck trying to beat that with pure CUDA C++ ;)
I think the point is to be able demonstrate
the ability to implement high-level abstractions,
in a very composable way,
that doesn't cost on performance.
> Use Python + Numba, Python + C, Swift, Cython, Go, Lua-Jit, Rust, D - surprise us!
I'm familiar with Python scientific ecosystems and currently learning Julia and Rust. As far as I see it, these languages likely do not compete well with Julia on all three categories --- "developer effort, performance and extensibility" for the problem you picked, because none of them was designed for native array arithmetic with broadcasting in the first place (except Numba maybe).
The current dominance of Python in scientific computing is rather a beneficent coincidence, because most people were pretty fed up with C++ and legacy Fortran back then, and efforts to improve tools of the trade tended to converge on Python. Fast forward 18 years after Python was born, Julia came to light and it was designed to do numerical tasks fast and elegantly and to avoid some pitfalls of "scripting languages". Does the success of Julia in these tasks signify the incompetence of Python? I don't think so. It served well what it was designed for. And if you tell me Julia is the right track of the history, you'd have to wait for it to happen. (Don't get me wrong, I like the Julia language and use it for a side project. I just don't see bashing other languages to prove that Julia is the one language to rule them all would be fruitful.)
On the other hand, I'd say you forget to count in Fortran 2008! For broadcasting with multithreading, Fortran + OpenMP works pretty well.
So the actual challenge is to provide a means, in your language of choice, to write an arbitrary expression like:
user_defined(x) + y + z
Which may mix matrices (x), vectors (y), scalars (z), and user_defined or built-in (+) functions applied elementwise; has automatic extension of dimensions; and must operate in a single pass, performing only one traversal.
This problem statement is rather tailored to Julia’s way of doing things—I don’t believe any other language can really “succeed” as a result, and not because Julia is any better (nor worse).
In Haskell I might simply rely on laziness and rewrite rules for fusion, or reach for a package like repa¹ or folds², depending on the actual problem at hand. There wouldn’t need to be a library providing this “broadcast” functionality, because it can be expressed already with other composable tools.
If you work with the language and type system, there are lots of tools available for giving the compiler enough information about the structure of your code to guarantee good performance, abstracted behind a reasonable API. You might end up writing something more explicit, like:
zipWith3 (+) (userDefined <$> x) (extend y) (extend z)
-- With MonadComprehensions + ParallelListComp
[ userDefined a + b + c
| a <- x | b <- extend y | c <- extend z
]
But I’d be perfectly happy with that, since it uses abstractions like functors and monads that I already understand. I probably wouldn’t reach for the direct equivalent of this Julia solution, which would be to write some Template Haskell macro that rewrites an expression to insert dimensional conversions (“extend” in the pseudocode above) and fusion of operations—that approach feels too brittle and difficult to extend, and not because of any deficiency in the language.
>There wouldn’t need to be a library providing this “broadcast” functionality
Actually, your sample "solution" is (was?) also a much used pattern in Julia - especially before we had such a powerful broadcast implementation.
So in Julia there is no need either, since it is already implemented much better in Base.
I think you're overlooking the actual point of my challenge: it's not that we need a broadcast package, but that a lot of other scientific libraries will need to use similar patterns, to write new packages with new algorithms!
The point is also, that I can go into the Julia repository, and without a lot of time & effort, can improve the current implementation of our broadcast - because it is not that much code and pretty easy to follow.
So, an equivalent in your example would be, if you can quickly implement something like repa or folds in a similar minimal example like my Julia implementation.
If you don't like the details of my challenge, feel free to make a minimal folds implementation from scratch, and show me that the result is fast and composable ;)
>Actually, your sample "solution" is (was?) also a much used pattern in Julia - especially before we had such a powerful broadcast implementation.
To add some more historical facts to this: before broadcast, we mostly used maps & zips in Julia. Which is not very elegant if you want to write down your math. Even with broadcast, some math nuts still complain that it isn't close enough to the actual math :D So consider, that those people are Julia's user base ;)
And then, someone pretty much implemented broadcast in their freetime and added it as a PR to the Julia language. This is how the language grows, and a large part of that is possible, because it takes relatively little time to come up with a performant implementation of such a difficult problem.
And now we have broadcasting in the language, and maps + zips are indeed less idiomatic... Because anyone who started using the new broadcast immediately thought it was an improvement to "the old ways".
53 comments
[ 3.2 ms ] story [ 117 ms ] threadIt doesn’t really matter if python doesnt have as powerful an array syntax as julia, if I can pull in numpy (a standard, battle-tested library) and produce equivalent (and good) code.
But if I pull in numpy, and still can’t produce code as nice as julia (because of whatever limitations of python as a language), then there’s an argument to be made for switching to julia.
But going purely off what the language gives by its initial installation... its just an arbitrary limitation to give the question one answer.
The article claims that it proves the point ... "This can be unfair since Julia has lots of functionality for array algorithms built in, but also illustrates the point" ... but assuming good intentions, I'm not sure what point that is? At best, you might be comparing APL languages to one another, but nothing broader than that.
But it's not clear to me that this "challenge" is intended to say that array operations are easy, for a newcomer to the language. As I understood it, it's an argument for Julia being better in the long run, for the code that requires "developer effort, performance and extensibility"; that is, elements of the language beyond the first few days of using it, for packages larger than what would be written by "amateur programmers"
If depicting usage for the amateur was the main point of the article, it totally went over my head
You understood that right, it is not :) That's why I wrote, that I'd probably need years to create the same implementation in any other language.
I do think it does have a widespread implication for the ecosystem though, and therefore at some point will also be relevant to newcomers.
In a language like Unicon/Icon, I can "$include" a source file, I can "link" an ucode (compiled source, unlinked) file, I can "import" a ucode class file, I can put all the source in the same file or I can get the compiler to bring all the files together on the command line and it will basically end up with the same result.
Does Julia provide Base as source to allow you to do anything like the above?
So the part that gets loaded when you load the language/compiler is what is referred to as Base. So, it's kind of impossible to not have it and it is part of the language. So it's somewhat like asking, what can you do without "importing" integers? (ok, that's a hyperbole, but you get the idea ;) ) But, the source code does get shipped, so you might actually go in there and change stuff if you want, and Julia will pick up those changes - even, to get back to that hyperbole, how integers are defined, since they're implemented in pure Julia. I'm not a 100% sure what you're aiming at, but does that answer your question? ;)
It does answer the question in that it appears that Julia, as a language, doesn't provide you with much to start with, not even the bare bones. I have seen commentaries back in the 1980's that discussed such language designs. It makes for some uniformity but does require much work to be done to create the basic infrastructure that would allow and enable others to use these kinds of languages out of the box. Interesting in some ways, I suppose.
Despite it has great libraries, I think it's the wrong language for scientific computing. In a nutshell, it's much harder than it should be to write concise and performant code. There's a huge stack of things used to achieve this, which often makes it complex to reason about performance and obscures the underlying mathematics: cpython, NumPy, pandas, Cython, Numba, PyTorch...
Julia is really appealing because it's very close to executable mathematics. And thanks to a few language design decisions, it's very easy to write performant code and reason about it.
For example, Julia ML libraries like Flux have full composability with Julia code. That's a game changer in terms of simplicity. Tensorflow is getting ported to Swift to mimic that feature.
> "Types & Functions From Different Packages It's mandatory to use an existing package without changing it. This can also be emulated by putting it in an isolated module not using any constructs defined in our implementation."
I don't know if the article has been revised, but this I read the rules about libraries completely opposite from you.
It is a general purpose language, but I don't think it is that much better than python in the scripting space (probably less good for now).
https://en.wikipedia.org/wiki/List_of_cognitive_biases#Frequ...
The vast Julia PR machine is certainly earning its million dollar bonuses this month...
(But you're right, of course; with a stable release out, people are now able to use and promote the language more effectively.)
Create a better string copy way:
Also, heck, it's a challenge, we all want in, please make it so :)
If I can outperform Julia by 50% in C++, but I have to write 10x code. I won't bother with C++.
If you manage to have that in the end, that's a very valuable promotional information.
For one of my master courses, I had to write 200 lines in python. C++ solution would have been in the thousands. My colleague solved it in 10 R lines. R is on my list now :)
d := s
Strings are immutable, so it internally just copies a descriptor.
Setting up an artificial challenge with rules that favor 'your' language up front is a great way to get the result that you want, but not a great way to learn. Each language has its strengths and its weaknesses, but none of those strengths and weaknesses matter all that much from the point of view of an end user until they have a direct impact on the quality of that which is produced. And programmers have shown time and again they can write crappy code in just about any language (and good code too...) with the language features having a much smaller impact on the outcome than the quality of the people writing the code.
Come on.
I do think, that it is quite showing, that google + tensorflow don't believe in the mix of python + c/c++. I've been working quite closely on the challenges that faces machine learning libraries at the moment, and I can totally see where they're coming from when they're unhappy with the current solution and want to rewrite it in swift. How should I express this in a better way?
I'd be curious to see what kinds of implementations someone could come up with in CUDA using templates, though. It could easily blow the Julia performance out of the water.
And this is tuned towards package development, specifically generic package development. This is quite a hot topic in scientific computing. The challenge isn't for people who use packages, it's for people who develop packages and are looking to make them generally useful (i.e. not just single project).
Thus, a much fairer comparison, if we care so much about performance that non-SIMD execution is a non-starter, would be CUDA vs. Julia targeting the GPU.
Note that I absolutely love Julia because I find it far easier to write than a language like C++, but idiomatic Julia code still has low overhead, unlike, say, Python, which introduces orders of magnitude of overhead compared to low level languages, and you have to move away from simple idiomatic code to achieve reasonable performance. However, I find it unreasonable to focus on SIMD performance - the performance gain over non-vectorized code isn't that big, and if you do want to eek out that much extra performance, you should focus on using the GPU.
That's not true. There's a lot of heavily serial algorithms which do not always work well on the GPU unless they are of a very large size, larger than most applications. This happens a lot in optimization and differential equations, two of the pillars of scientific computing.
>Thus, a much fairer comparison, if we care so much about performance that non-SIMD execution is a non-starter, would be CUDA vs. Julia targeting the GPU.
Tim already wrote that article where it shows how Julia code generation can be faster in some instances for GPUs. https://devblogs.nvidia.com/gpu-computing-julia-programming-...
>but idiomatic Julia code still has low overhead
Where?
And those situations tend to get a very minimal speedup from SIMD.
>Tim already wrote that article where it shows how Julia code generation can be faster in some instances for GPUs. https://devblogs.nvidia.com/gpu-computing-julia-programming-....
Yes, that's great, and would have made for an interesting challenge, rather than focusing on SIMD instead of serial or GPU performance.
>Where?
I didn't word it well, but I meant to say that Julia doesn't have much overhead compared to C/C++, unlike Python (what overhead it does have comes from garbage collection).
I'm familiar with Python scientific ecosystems and currently learning Julia and Rust. As far as I see it, these languages likely do not compete well with Julia on all three categories --- "developer effort, performance and extensibility" for the problem you picked, because none of them was designed for native array arithmetic with broadcasting in the first place (except Numba maybe).
The current dominance of Python in scientific computing is rather a beneficent coincidence, because most people were pretty fed up with C++ and legacy Fortran back then, and efforts to improve tools of the trade tended to converge on Python. Fast forward 18 years after Python was born, Julia came to light and it was designed to do numerical tasks fast and elegantly and to avoid some pitfalls of "scripting languages". Does the success of Julia in these tasks signify the incompetence of Python? I don't think so. It served well what it was designed for. And if you tell me Julia is the right track of the history, you'd have to wait for it to happen. (Don't get me wrong, I like the Julia language and use it for a side project. I just don't see bashing other languages to prove that Julia is the one language to rule them all would be fruitful.)
On the other hand, I'd say you forget to count in Fortran 2008! For broadcasting with multithreading, Fortran + OpenMP works pretty well.
This problem statement is rather tailored to Julia’s way of doing things—I don’t believe any other language can really “succeed” as a result, and not because Julia is any better (nor worse).
In Haskell I might simply rely on laziness and rewrite rules for fusion, or reach for a package like repa¹ or folds², depending on the actual problem at hand. There wouldn’t need to be a library providing this “broadcast” functionality, because it can be expressed already with other composable tools.
If you work with the language and type system, there are lots of tools available for giving the compiler enough information about the structure of your code to guarantee good performance, abstracted behind a reasonable API. You might end up writing something more explicit, like:
But I’d be perfectly happy with that, since it uses abstractions like functors and monads that I already understand. I probably wouldn’t reach for the direct equivalent of this Julia solution, which would be to write some Template Haskell macro that rewrites an expression to insert dimensional conversions (“extend” in the pseudocode above) and fusion of operations—that approach feels too brittle and difficult to extend, and not because of any deficiency in the language.¹ http://hackage.haskell.org/package/repa
² https://hackage.haskell.org/package/folds
Actually, your sample "solution" is (was?) also a much used pattern in Julia - especially before we had such a powerful broadcast implementation.
So in Julia there is no need either, since it is already implemented much better in Base. I think you're overlooking the actual point of my challenge: it's not that we need a broadcast package, but that a lot of other scientific libraries will need to use similar patterns, to write new packages with new algorithms!
The point is also, that I can go into the Julia repository, and without a lot of time & effort, can improve the current implementation of our broadcast - because it is not that much code and pretty easy to follow.
So, an equivalent in your example would be, if you can quickly implement something like repa or folds in a similar minimal example like my Julia implementation.
If you don't like the details of my challenge, feel free to make a minimal folds implementation from scratch, and show me that the result is fast and composable ;)
To add some more historical facts to this: before broadcast, we mostly used maps & zips in Julia. Which is not very elegant if you want to write down your math. Even with broadcast, some math nuts still complain that it isn't close enough to the actual math :D So consider, that those people are Julia's user base ;)
And then, someone pretty much implemented broadcast in their freetime and added it as a PR to the Julia language. This is how the language grows, and a large part of that is possible, because it takes relatively little time to come up with a performant implementation of such a difficult problem.
And now we have broadcasting in the language, and maps + zips are indeed less idiomatic... Because anyone who started using the new broadcast immediately thought it was an improvement to "the old ways".