Since the dawn of computing, hardware (computer & memory) has been getting cheaper. And software has been filling the gap.
So if throwing effort at hardware no longer improves things, it must be time to put real effort into improving software. No, not making it bigger! Not by adding features. We could start by making it smaller, at reducing/removing/replacing. By investigating alternate models, instead of adding minor tweaks or very-specific-purpose features to our existing software.
Fixing "software" won't be easy, and won't be quick, or cheap. But we've put enormous amounts of effort (money) into hardware. It must be time to put serious effort into some of the alternative software architectures that have been proposed.
I want to believe that there's some world where developers start putting actual craftsmanship into their software (again?), but I think this will only be true for SaaS or related environments where the developer is also the one spending money on the compute. Everyone else will probably outsource the optimization effort to a mixed hardware-software vendor that amortizes the cost (like hyperscale cloud providers) via middleware, instead of just amortizing the cost via hardware.
People (including on HN!) like to say that cycles are cheap but developer time isn't. There's a reason for this that will be invariant of the actual costs: it's the user that pays for the hardware and the developer that pays for development time. End-users are willing to spend money on hardware, but not software, and this has remained true even as companies like Apple and NVIDIA have blurred the boundary. In the current environment of $0 software, how do you (as a developer) fund more efficient software? I think the likely answer will be that developers will happily lock themselves to whatever vendor offers to solve this problem for them. We've seen this in ML already.
Parallelization and GPUs were the hot story 10-5 years ago, and require(d) a pretty substantial shift in the software stack for less-general gains. You're still hoping the cost-per-transistor goes down. I think recent 400W+ GPUs have shown that we're coming close to the end of this particular S-curve. The big question is whether any of the tricks we have left are broad enough and strong enough to address the economic problem.
8 comments
[ 2.8 ms ] story [ 31.6 ms ] threadSo if throwing effort at hardware no longer improves things, it must be time to put real effort into improving software. No, not making it bigger! Not by adding features. We could start by making it smaller, at reducing/removing/replacing. By investigating alternate models, instead of adding minor tweaks or very-specific-purpose features to our existing software.
Fixing "software" won't be easy, and won't be quick, or cheap. But we've put enormous amounts of effort (money) into hardware. It must be time to put serious effort into some of the alternative software architectures that have been proposed.
People (including on HN!) like to say that cycles are cheap but developer time isn't. There's a reason for this that will be invariant of the actual costs: it's the user that pays for the hardware and the developer that pays for development time. End-users are willing to spend money on hardware, but not software, and this has remained true even as companies like Apple and NVIDIA have blurred the boundary. In the current environment of $0 software, how do you (as a developer) fund more efficient software? I think the likely answer will be that developers will happily lock themselves to whatever vendor offers to solve this problem for them. We've seen this in ML already.
Write software that runs on bare metal, rather than on top of inefficient stacks.