It's a fun observation, but the causality in this feels off. There are tons of esoteric programming languages that are difficult to write in and no one talks about because they have no other value. BF is not very useful…
I think, maybe, the stronger argument is that the area of Generative Programming (which covers just about anything that generates another program) itself is too broad to properly address in a semester course. This…
There are different forms of argumentation and they have a different tone that maybe doesn't come through so well on the internet without context. I dislike the debate club mentality where arguments are more like a…
You're not wrong. You can think of internet arguments as a method of developing a thought and collecting counterarguments in a real environment to refine it. The choice of forum affects the quality of the…
I was fortunate to learn this from pointless internet flamewars early in my life. Understand why you're in an argument and what you hope to accomplish by being in an argument. On the internet, it is usually very clear…
Parallelization and GPUs were the hot story 10-5 years ago, and require(d) a pretty substantial shift in the software stack for less-general gains. You're still hoping the cost-per-transistor goes down. I think recent…
I want to believe that there's some world where developers start putting actual craftsmanship into their software (again?), but I think this will only be true for SaaS or related environments where the developer is also…
I don't usually respond to old comments, so I don't know if you'll read this, but I hope I can encourage you to think more broadly about what "differentiable programming" means. Different fields have a different…
There's no contradiction: autodiff is a method of implementing differentiable programming. In this example, it is implemented as a type that handles a trace of a program, but everything else is left to the programmer.…
This "discretizes then differentiates" to borrow terminology from [1] which is one of the more accessible presentations and papers. The program might evaluate correctly, but equational reasoning (like you might want for…
The existence of a different kind of CPU isn't a meaningful distinction at the level of discussing paradigms. The semantics are different, so the abstract machine is different. The fact that I need a different set of…
I don't need OOP to do a hash table lookup and then an indirect function call with the receiver as the first argument either but that ignores that there's more to a paradigm than the algorithm I use for facilitating it.…
Autodiff does not work with for loops or if statements. The current solutions effectively pick a few promising traces through the program and then assume that nothing else exists. To handle it more elegantly (for things…
Object oriented programming, for example, doesn't let me have a variable hold half of one object and half of another or let the language derive the code that gave me that object at runtime, but object oriented +…
At the time the comment was made the link was https://harvard-iacs.github.io/2019-CS109A/pages/materials.h... where neural networks were mentioned. See https://news.ycombinator.com/item?id=32295656
The average consumer is more willing to pay for hardware than software, even if all the value is created by the software. It's the reason why Apple no longer charges for OS updates. Nvidia has essentially the same…
Apple's ethos of giving more to creators was because they were focused on the niche markets that would spend good money for good products. By the late 90s, creators were their primary users so the survival of Apple…
The difference is much more nuanced than this. A modern GPU can (and probably does) do most of what you've listed for a CPU. Speculative execution and branch prediction are a bit less likely to be invested in (because…
Most people just want an allocator that works reasonably well, and maintaining that expectation means not exposing too many details that you might be held accountable to. If you care deeply, there are usually…
Depends on how you define innovation. The hard truth is that there is no free lunch. We like to pretend we're using a Turing machine, but the moment you start caring about performance or memory limits, the abstraction…
C and C++ are some of the few languages where the spec goes out of its way to not depend on an allocator, for good reason, and this is well after you've accounted for the majority of the code that, hopefully, doesn't…
It's unreasonable to assume that an stdlib must be designed around performance to any capacity. For most software, the priorities for the stdlib are 1) existing, 2) being bug/vulnerability free, and likely, in the…
This misses the point of my comment. When you put faith in malloc, you're putting hope in a lot of heuristics that may or may not degenerate for your particular workload. Windows is an outlier with how bad it is, but…
If you're depending on the performance of malloc, you're either using the language incorrectly or using the wrong language. There is no such thing as a general purpose anything when you care about performance, there's…
The semiconductor world is very small by tech standards, and so there are correspondingly many fewer people at the management or upper IC levels that can coordinate large, consequential decisions and even fewer that…
It's a fun observation, but the causality in this feels off. There are tons of esoteric programming languages that are difficult to write in and no one talks about because they have no other value. BF is not very useful…
I think, maybe, the stronger argument is that the area of Generative Programming (which covers just about anything that generates another program) itself is too broad to properly address in a semester course. This…
There are different forms of argumentation and they have a different tone that maybe doesn't come through so well on the internet without context. I dislike the debate club mentality where arguments are more like a…
You're not wrong. You can think of internet arguments as a method of developing a thought and collecting counterarguments in a real environment to refine it. The choice of forum affects the quality of the…
I was fortunate to learn this from pointless internet flamewars early in my life. Understand why you're in an argument and what you hope to accomplish by being in an argument. On the internet, it is usually very clear…
Parallelization and GPUs were the hot story 10-5 years ago, and require(d) a pretty substantial shift in the software stack for less-general gains. You're still hoping the cost-per-transistor goes down. I think recent…
I want to believe that there's some world where developers start putting actual craftsmanship into their software (again?), but I think this will only be true for SaaS or related environments where the developer is also…
I don't usually respond to old comments, so I don't know if you'll read this, but I hope I can encourage you to think more broadly about what "differentiable programming" means. Different fields have a different…
There's no contradiction: autodiff is a method of implementing differentiable programming. In this example, it is implemented as a type that handles a trace of a program, but everything else is left to the programmer.…
This "discretizes then differentiates" to borrow terminology from [1] which is one of the more accessible presentations and papers. The program might evaluate correctly, but equational reasoning (like you might want for…
The existence of a different kind of CPU isn't a meaningful distinction at the level of discussing paradigms. The semantics are different, so the abstract machine is different. The fact that I need a different set of…
I don't need OOP to do a hash table lookup and then an indirect function call with the receiver as the first argument either but that ignores that there's more to a paradigm than the algorithm I use for facilitating it.…
Autodiff does not work with for loops or if statements. The current solutions effectively pick a few promising traces through the program and then assume that nothing else exists. To handle it more elegantly (for things…
Object oriented programming, for example, doesn't let me have a variable hold half of one object and half of another or let the language derive the code that gave me that object at runtime, but object oriented +…
At the time the comment was made the link was https://harvard-iacs.github.io/2019-CS109A/pages/materials.h... where neural networks were mentioned. See https://news.ycombinator.com/item?id=32295656
The average consumer is more willing to pay for hardware than software, even if all the value is created by the software. It's the reason why Apple no longer charges for OS updates. Nvidia has essentially the same…
Apple's ethos of giving more to creators was because they were focused on the niche markets that would spend good money for good products. By the late 90s, creators were their primary users so the survival of Apple…
The difference is much more nuanced than this. A modern GPU can (and probably does) do most of what you've listed for a CPU. Speculative execution and branch prediction are a bit less likely to be invested in (because…
Most people just want an allocator that works reasonably well, and maintaining that expectation means not exposing too many details that you might be held accountable to. If you care deeply, there are usually…
Depends on how you define innovation. The hard truth is that there is no free lunch. We like to pretend we're using a Turing machine, but the moment you start caring about performance or memory limits, the abstraction…
C and C++ are some of the few languages where the spec goes out of its way to not depend on an allocator, for good reason, and this is well after you've accounted for the majority of the code that, hopefully, doesn't…
It's unreasonable to assume that an stdlib must be designed around performance to any capacity. For most software, the priorities for the stdlib are 1) existing, 2) being bug/vulnerability free, and likely, in the…
This misses the point of my comment. When you put faith in malloc, you're putting hope in a lot of heuristics that may or may not degenerate for your particular workload. Windows is an outlier with how bad it is, but…
If you're depending on the performance of malloc, you're either using the language incorrectly or using the wrong language. There is no such thing as a general purpose anything when you care about performance, there's…
The semiconductor world is very small by tech standards, and so there are correspondingly many fewer people at the management or upper IC levels that can coordinate large, consequential decisions and even fewer that…