I thinks this type of paradigm shift is what will be needed for the next level of performance for AI. But it also seems like for that to be competitive with current methods, they need to reduce the size by a factor of 2-3 and also make bigger chips.
I just feel like IBM has a track record recently of doing great research but not actually manufacturing anything. Kind of like Google.
Usually IBM doesn't sell to the kind of HN crowd, rather people happy with Aix, IBM i and z/OS, DB2, cloud for mainframes, own JVM implementations, Java EE servers, offshoring services,...
Some of the research does end in such products, that are outside HN radar, because IBM isn't cool, yet many HN startups won't achieve a legacy like it and still be pumping money like IBM, even for boring business.
I would bet a large portion of software engineers work in Big Companies. FANG employees comment regularly here and they work for the biggest companies in Tech.
No one will let you play with a mainframe. I’ve worked for finical services companies and toured a data center with large IBM mainframes I wasn’t allowed to get close to much less touch the software.
I’m sure if I wanted to move to the team it would’ve been possible to play with z/OS.
I feel like the irony here is that for all the fences most companies put around their mainframes... at the end of the day and inside the fence it's usually 1-2 people running everything who still know how to hack JCL et al.
Yes, IBM reached a peak in the past from which a gradual fall still leaves them raking in money from largely legacy enterprise clients for a very long time.
That doesn't make them competitive in new markets, though.
Your repeated discount of "it's shrinking and has been shrinking" with "we just don't know" is like saying that because you won't be around in 100 years you also can't make claims about whether the sun will come up.
I won't be around in 100 years either but I am sure IBM won't survive for the same reason I'm sure the sun will rise: all rigorous analysis today indicates it.
Yea I'm just saying it's not like all hardware projects that Google does are "great research but not actually manufacturing anything." as claimed by OP.
Just a laymen here, but if the organic brain is something we're trying to model I always ached for some kind of analog processor for things like neural networks. Op-amps, not logic gates.
We're not really trying to model the organic brain; while some structures are somewhat inspired by some structures in the brain, the current learning structures are fundamentally quite different from biological brains.
While there is a bunch of research for modeling organic brains, IMHO it is primarily driven by neuroscience trying to understand how humans work, and not directly applicable to making computation more efficient.
But wouldn't some computations be more efficient it they could run on brain-like synapses/clusters or hardware analogs thereof? Pattern recognition, multi-modal perception and such.
Maybe? This presupposes we know how the brain works (we don't).
Then you can kind of walk it to "but what if it kinda worked like [my understanding of how the brain works], would it be more efficient?", but it's not really clear what conversation we're having at that point.
I wonder how many years (if ever) it'll be before it's available for purchase, and I wonder what architecture Nvidia will be on by then - it won't be Hopper.
Compact, low-power chips to run ML inference on stuff like vision, image generation, high-quality voice synthesis, maybe even translation and LLM stuff, would be very welcome.
But unless it can directly or indirectly support current transformer architectures (with their huge parameter counts), it's going to critically struggle with adoption.
We absolutely can and will in time, but for better or worse LLMs and large transformer models are highly effective and are entrenching themselves. Any competing hardware architecture that cannot port LLMs will have to do a lot of bootstrapping, and will have to have some extremely compelling reasons for people to choose it.
Cutting energy costs by an order of magnitude could be enough motivation, but you'll still need to demonstrate competitive model performance.
There is a book called The Silicon Eye about the history, and not coincidentally Carver Mead also wrote a book on neuromorphic analog neural network implementation back in the 80s.
From what I recall there are analog feedback stages between adjacent cells, and that was a ongoing theme throughout the work of the teams around Mead. His neuromorphic book is almost entirely about doing that to sound and images.
As an aside the Foveon cameras are worth experiencing. They are amazingly slow, the colours go wrong in less than perfect lighting (the stacked filter) but the edges on objects in resulting images have a definition that you only realize bayer filters completely destroy when they are gone.
Do you have an example or can expand on what you mean about the last point? Isn't sharpness mostly limited by the lens systems in modern cameras rather than the Bayer sensor resolution?
The use of a bayer filter means that the actual edges for different colour areas are created in software processing afterwards. This causes everything to be slightly misaligned and fuzzy in a way the foveon was not, and no amount of reconstructive processing is going to help you.
Of course today the actual resolution of more conventional sensors dramatically exceeds the best foveons to the point where this advantage is nullified.
I work on camera systems, I know how a Bayer filter works, it doesn't make everything misaligned and fuzzy. Edge directed upscaling works great. I'm curious how the foveon looked different or if you had an example.
If you only mean for old digital cameras, then sure. But you're not seeing bayer artifacts in any modern camera. Rolling shutter artifacts, on the other hand..
I can send you some example images of Foveon Merrill sensor if you want. The cameras are slow by today’s standards, but the images are delightfully detailed. Sigma is still working on a new full frame Foveon camera, I am waiting patiently.
Back to topic, I believe this type of tech (analog/physical computing, processing and memory in the same unit) will become very significant on a 10+ year horizon. Hinton has been hinting at some collaborations recently
> I work on camera systems, I know how a Bayer filter works, it doesn't make everything misaligned and fuzzy. Edge directed upscaling works great.
The point is you might think that if you were blissfully unaware of anything else. Even if you're not you shouldn't, because edge directed upscaling is imagining information into the sensor data that isn't actually there.
It's not like Foveons are objectively better for satellite imageries and military "electro-optical" cameras, it's bit of a digital camera equivalent to reversal films and DSD audio.
The two main gotchas of Foveon sensors in its days(so far) were while SIGMA maintained that Foveon pixels are worth 3x Bayer, the sensor resolutions were consistently 1/3rd or less than competitors to begin with, and also that the sensitivity was atrocious. ISO800 was pushing it for early models and recognizable colors at ISO3200 in later models was an achievement in its own(they lose chroma before luma). Fujifilm did better than that in films!
SIGMA Corporation in Japan was virtually the only user of Foveon sensor, in their DP and SD series cameras. Photos taken with e.g. dp2 Quattro in 50mm, DP1x in 28mm are easily found on Flickr. To me a Foveon image looks like it's taken through a piece of radiation shielding glass and later color corrected, which I'm guessing to be close to literal description as the color components are indeed captured through preceding luma and first chroma layers.
Foveon is apparently "more of YUV", because it's not yet possible to band-stop just the desired wavelength range for each layers and let others pass. Instead the three layers gets RGB, RG and R channels respectively, and the camera computes the delta.
The "Quattro" generation Foveon used 2x2 subpixels for the top layer. My impression then was it was a last-ditch effort to mitigate poor sensitivity, but it's interesting how it was arguably an only analog convolutional network processor in a consumer product, from a different perspective.
Rule of thumb: IBM tends to spoof results to get published, then use the published results to trick tech-illiterate clients into using their stack.
At conferences I used to go to, they'd just phack the hyperparameters of an ensemble model on the benchmark sets, and put out a press release saying they were state of the art. They were mostly ignored by the academics who went to the talks on the actual novel work.
I have become pretty wary of all headlines touting how a new chip is going to dethrone Nvidia. The problem with every hardware company is that they ridiculously underestimate software. I think the only way a company is going to start competing is by being software first. Make the software so good and easy to use that every one wants to use it. Only then is the hardware going to compete.
IBM has a long history of competent software development, hardware development, and product development. This isn't a kickstarter. It's backed by published research as was the case for relational databases.
Modern IBM is at the forefront of modern chip design. They regularly publish some of the most advanced research in the world, and are part of the industry consortium driving those changes.
I am sorry for sounding harsh but this and other comments in the thread sound like IBM paid promotion. The headline is claiming to head up against Nvidia. First show me something that can compete with AMD which is miles behind Nvidia. Otherwise it's just click bait.
I do with that at least _tech journalism_ outlets would develop a habit of asking at least one followup question when a PR says that something is like the human brain. Having analog values between zero and one is not at all particular to the human brain, or any brain, where we have spikes occurring at varying rates, not scalar activation levels. This chip could equally well be described as having components which mimic a dimmer switch, or perhaps a gas gauge that varies between empty and full. Yes, that's cool. No, it's not really brain-like.
This does sound very intriguing. Even if it would only take away some inference load when it becomes viable. Has a non-zero chance of spawning the next leap in computing?
Considering IBM likely isn't going to crater in stock like some riskier bet might, and even if it's just to benefit from initial hype, I don't see a negative in moving chips I have laying around to big blue even though I very rarely do trades nowadays.
70 comments
[ 2.8 ms ] story [ 135 ms ] threadI just feel like IBM has a track record recently of doing great research but not actually manufacturing anything. Kind of like Google.
Some of the research does end in such products, that are outside HN radar, because IBM isn't cool, yet many HN startups won't achieve a legacy like it and still be pumping money like IBM, even for boring business.
More like outside people's radar because IBM only sells to big companies.
I’m sure if I wanted to move to the team it would’ve been possible to play with z/OS.
It is a matter of being on the right team.
That doesn't make them competitive in new markets, though.
I won't be around in 100 years either but I am sure IBM won't survive for the same reason I'm sure the sun will rise: all rigorous analysis today indicates it.
They just haven't done a whole lot lately, and most of what they have done is support people locked into their legacy solutions.
The chip in the article gains its advantages from doing the compute directly in the memory cells. It saves power by moving information around less.
(Quick google found this fun hobbyist article of a perceptron built with op-amps: https://www.nutsvolts.com/magazine/article/the_perceptron_ci...)
While there is a bunch of research for modeling organic brains, IMHO it is primarily driven by neuroscience trying to understand how humans work, and not directly applicable to making computation more efficient.
Then you can kind of walk it to "but what if it kinda worked like [my understanding of how the brain works], would it be more efficient?", but it's not really clear what conversation we're having at that point.
IBM blog post: https://research.ibm.com/blog/analog-ai-chip-low-power
Also I'd like to mention that those chips are for inference only. You still need GPUs for training.
Imagine a good inference engine right in a phone.
So about about 1000x away from being useful? Most current models are billions not millions.
It's not as if we said wow, all you need is attention, and then Nvidia built some GPUs for it.
Cutting energy costs by an order of magnitude could be enough motivation, but you'll still need to demonstrate competitive model performance.
https://en.m.wikipedia.org/wiki/Foveon_X3_sensor
There is a book called The Silicon Eye about the history, and not coincidentally Carver Mead also wrote a book on neuromorphic analog neural network implementation back in the 80s.
From what I recall there are analog feedback stages between adjacent cells, and that was a ongoing theme throughout the work of the teams around Mead. His neuromorphic book is almost entirely about doing that to sound and images.
As an aside the Foveon cameras are worth experiencing. They are amazingly slow, the colours go wrong in less than perfect lighting (the stacked filter) but the edges on objects in resulting images have a definition that you only realize bayer filters completely destroy when they are gone.
Of course today the actual resolution of more conventional sensors dramatically exceeds the best foveons to the point where this advantage is nullified.
If you only mean for old digital cameras, then sure. But you're not seeing bayer artifacts in any modern camera. Rolling shutter artifacts, on the other hand..
https://www.digitalcameraworld.com/news/sigma-will-never-giv...
Back to topic, I believe this type of tech (analog/physical computing, processing and memory in the same unit) will become very significant on a 10+ year horizon. Hinton has been hinting at some collaborations recently
The point is you might think that if you were blissfully unaware of anything else. Even if you're not you shouldn't, because edge directed upscaling is imagining information into the sensor data that isn't actually there.
The two main gotchas of Foveon sensors in its days(so far) were while SIGMA maintained that Foveon pixels are worth 3x Bayer, the sensor resolutions were consistently 1/3rd or less than competitors to begin with, and also that the sensitivity was atrocious. ISO800 was pushing it for early models and recognizable colors at ISO3200 in later models was an achievement in its own(they lose chroma before luma). Fujifilm did better than that in films!
SIGMA Corporation in Japan was virtually the only user of Foveon sensor, in their DP and SD series cameras. Photos taken with e.g. dp2 Quattro in 50mm, DP1x in 28mm are easily found on Flickr. To me a Foveon image looks like it's taken through a piece of radiation shielding glass and later color corrected, which I'm guessing to be close to literal description as the color components are indeed captured through preceding luma and first chroma layers.
The "Quattro" generation Foveon used 2x2 subpixels for the top layer. My impression then was it was a last-ditch effort to mitigate poor sensitivity, but it's interesting how it was arguably an only analog convolutional network processor in a consumer product, from a different perspective.
At conferences I used to go to, they'd just phack the hyperparameters of an ensemble model on the benchmark sets, and put out a press release saying they were state of the art. They were mostly ignored by the academics who went to the talks on the actual novel work.
Modern IBM isn't what it once was.
Edit: to clarify i mean on the software side. I don't know enough about the hardware side to comment.
https://youtu.be/DZ0yfEnwipo
essentially they develop and license technologies around the chip making process
I do with that at least _tech journalism_ outlets would develop a habit of asking at least one followup question when a PR says that something is like the human brain. Having analog values between zero and one is not at all particular to the human brain, or any brain, where we have spikes occurring at varying rates, not scalar activation levels. This chip could equally well be described as having components which mimic a dimmer switch, or perhaps a gas gauge that varies between empty and full. Yes, that's cool. No, it's not really brain-like.
Considering IBM likely isn't going to crater in stock like some riskier bet might, and even if it's just to benefit from initial hype, I don't see a negative in moving chips I have laying around to big blue even though I very rarely do trades nowadays.