The article mentions "See Longbottom’s extensive tests and comparisons article here." and [1]. This was already mentioned in a snapshot of 18 Jan 2024 [2] so it wasn't added after your criticism.
This is actually mildly surprising to me that the Raspberry Pi is only 4.5x faster. I would have bet 10-20x faster just because of how much time has passed and all the talk about: "your cellphone is 1000x faster than the Apollo computers" that I've been hearing since the time of the t-mobile sidekick.
I've always taken them with a grain of salt, but even if they were only an order of magnitude off, a Pi is loads faster than a sidekick. And sure the Cray is loads faster than the Apollo computers, but I wouldn't have thought it was THAT much faster.
The Cray 1 was way ahead of its time. It implemented SIMD vector instructions to speed up numerical computation, was liquid cooled, and ran at 80 MHz (in 1976). The chief architect, Seymour Cray, was kind of a genius, and is responsible for designing many other pioneering machines.
Also, fun fact, it didn't have a CPU. It used all discrete logic chips, and was wired by hand, with lots and lots of wire. IIRC Seymour Cray liked to hire women to do the wiring job, because they had an easier time fitting inside the computer core to wire it, and doing detailed work because they had smaller hands.
Lots and lots of wire, yes, with each one cut to a specific length so that all of each gate's input signals arrive at the same time. That's why the backplane looks like a mess of wires hanging down due to seemingly being way too long.
Take a look at all the PCB traces near RAM and CPU. You can see them 'squiggle' looking for matching-space at either end of the RAM/CPU connection.
PCBs make things like consistent length matching easier. All circuit boards have near identical lengths, controlled impedances and other features that support ... Well... Effectively 6-billion baudrates for DDR5.
6000Mhz DDR5 with 64-bits per half-clock (3000Mhz clock so 6000 transfers per second) has a bit every 5 centimeters or so.
And that kinda makes it unfair to the Raspberry, maybe not the RPI1 but I'm pretty sure that the RPi4 (rated at 78x the Cray) running code in compute-mode would thrash the Cray 1 even harder (yes, not entirely comparable but since you probably didn't get full throughput on the Cray with regular C code then using the GPU on the RPi should be fair game).
That term is used to describe modern vector instructions that are NOT vector instructions in the classical Cray-sense. Modern CPUs use wide registers that can be regarded as vectors of 2/4/8/... values. Cray used memory-to-memory variable-length vectors.
I'm not saying that "SIMD vector instructions" is an incorrect description of what the Cray machines did, I'm saying that the term usually means something else today.
The fluorinert coolant was really neat stuff too. Inert, so you could submerge electronics in it to cool them. I was able to find recycled coolant and used that to do the liquid cooled build for my dual celeron system where coolant flowed over the board/cpu. Knowing what I do now... I should have been far less cavalier with the stuff.
I guess it was THAT much faster. That is an insane leap in performance in a decade. I know they are apples and oranges. One weighed about 150x more than the other, but still such a surprise to me.
They're not just apples and oranges, but apples vs differential equations. They had completely different requirements. Like ROM being sown for Apollo.
You get something completely different if failure is not an option.
But also yes, it is a big leap. NASA bootstrapped the semiconductor industry by buying up most of the world's supply. Without the Apollo program we may only just now have gotten smartphones. (And some people still think it's a waste of money. pff!)
And probably most of those people don't realize they wouldn't be posting anything at all if not for 'whitey on the moon', the Apollo program era started with sliderules in 1962 and ended in 1972 with functional microcomputers (8008, 1972, shortly followed by the 6502, 1975) and a very short while later we had programmable pocket calculators (Ti59, 1977). The whole semi conductor industry was jumpstarted in those years.
When I first started out with electronics transistors were an absolute rarity and tubes were normal, in the space of a decade that changed completely.
> NASA bootstrapped the semiconductor industry by buying up most of the world's supply. Without the Apollo program we may only just now have gotten smartphones.
I strongly disagree with that. The semiconductor industry was thriving before NASA even existed, none of the central enabling inventions (MOSFET transistors, semiconductor manufacturing techniques) were made there or even related to the Apollo program, and "computers" as in turing complete machines already existed before NASA and had plenty of applications apart from space travel.
NASA/Apollo giving todays smartphones a 10 year technology boost is just pure fiction and not even remotely supported by facts.
We can't do an A/B with history, but I'm basing it on things like:
> By the mid-1960s, according to the PBS documentary, NASA was buying 60 percent of the integrated circuits produced in the United States. Fairchild was a major supplier, shipping about 100,000 devices for the Apollo space program in 1964 alone.
Not only did the Cray weigh far more than the Apollo computer, and not only did the Cray supercomputer use far more power, but the Apollo computers had to be designed to be highly resistant to interference from radiation.
Radiation hardened electronics are fascinating to me. To this day, electronics that function in outer space are much slower than datacenter or consumer-oriented components.
You’re not wrong, since the 4.5x number comes from a comparison with the rPi 1.
The rPi 4 is over 10x faster than the 1.
The original article points out that the raspberry pi 400, with the benchmark targeting 64 bits, was about 80x Cray for the same benchmark which was 4.5x for the rPi 1.
Raspberry Pi 5 advertises a 2-3x performance gain over earlier versions, though this wasn’t benchmarked in these tests.
Aren't these also in FLOPS? The rpi isn't exactly a floating-point monster. Hell, until the Pentium, Intel CPU's weren't guaranteed to even have an FPU.
The 80486SX was introduced two years after the 80486 and did not have an FPU (unless it was purchased as an option).
I think the meaning of my comment was clear: the 80486 was the first Intel CPU guaranteed to have an FPU. I didn't say all the Intel CPUs which came afterward were guaranteed to have them...
I guess that's true. I never really thought of them as the same thing.
Looking over some old info about the "i486" (I swear I had completely forgotten about the i) was a treat. It's only barely believable but I think the sheer annoyingness of Intel's marketing and market segmentation might have actually peaked three decades ago.
It looks like the Cray gets a bit of a boost from the linpack scores (the pi is only 1.6x faster!), which is a good test for the Cray (understatement!).
My dad worked in Facilities at the Minnesota Supercomputer Center (or maybe it was Cray, or maybe it was Control Data, or maybe it was NetASPx, or maybe it was something else, I was never clear to be honest) here on Washington Ave in Minneapolis. Sometimes as a kid in the 90s, he'd take me in to follow him around for a day. I definitely remember looking at the server racks and supercomputers they had on their main floor, watching automated tape backup machines move tapes around, all these crazy water cooling setups with colored lights, stuff like that. I even got to see the big diesel backup generators in the basement once. But the standout was the decommissioned Cray-1 they had as a seating bench in the front lobby. Could walk inside the unit and look at all the wires and circuit boards connected together, or just sit on the nice padded bench and have a chat. If you really knew what you were doing, you could lift up the seat panels and have a look at more boards and wires. Fun stuff.
> “The Raspberry Pi ... is more than 4.5 times faster than the Cray 1.”
The applications have grown as well.
The Cray 1 was used for mundane tasks like "large-scale scientific applications, such as simulating complex physical phenomena, and was sold to government and university laboratories." [1] But the power of the Raspberry Pi allows for cutting edge computing tasks like "watering plants, monitoring the birds in your yard, or for a smart doorbell!" [2]
Am I the only one that is shocked by the fact that a 1978 computer, even if a supercomputer (but still using the technology of the time) was 1/4 the speed of a Raspberry? The Pi, if you look at the big picture of computing, is a very fast computer. For comparison: you can run a 1 billion parameters LLM on a Raspberry pi at decent speed. This means that the Cray could run it, even if slowly. That's incredible.
But seriously, while we could have run it maybe speedwise, it definitely lacked the memory, not? And if one tried to train it he wouldn't be finished today.
But would make a fun backwards sci-fi story imagining a time traveller that brought the 80ies an LLM from today, what would the world say and do with that slow oracle?
Does that mean just a multi gig file? What is INSIDE the LLM that would be of value? How does one speak to an LLM WRT 80's tech, and what could one glean from it....
In a way that someone in the 80s could understand?
An LLM is a very highly compressed store of knowledge combined with an advanced parser than understands questions in plain English. A consequence of the compression is that sometimes the answers lose some accuracy, which is a deliberate trade-off to make it work at all.
My story plot would sure include LLM (coefficients file as today) + code to run it. So 80s humans could run it on the Cray, ask it questions, and get answers (after some time :D).
LLM could explain itself what it is.. (if there are not more important questions to ask, contention would ensue).
I don’t think they’d have any trouble with the math, it’s just a bunch of regressions and matvecs, right?
I think the process of collecting and storing all the data would be more mind blowing to them—of course they were at the beginning of Moore’s law, so they could see the trajectory if they looked for it, but it is one thing to stand on the coast with waves lapping at your ankles and imagine how the ocean gets deeper as you keep going and another to get chucked out of a helicopter in the middle of the Pacific.
LLM is a lossy compression of the internet.
We can provide it in a form that is directly executable on 80s computers though gpt4 tries to convince me that it is practically impossible the reduced model would be much weaker (somebody doesn't want to be sent to 80s ;)
I can't find it now but I wrote a bit of fanfic where Ken Thompson sent an LLM back in time (referencing his "Love, Ken" UNIX tapes that he would send out) to save humanity. He was always a bit head of his time.
This guy Time Travels. (check his hands, he likely has extra fingers)
But... lets look at the availability of DATA in the 80s..
Frankly, this is how hacking/phreaking was invented.
Dumpster-diving for line-printer discards in dumpsters to understand what their systems did.
(This is an actual story; people were bin dipping (at&t?) dumpsters and finding exploits (social or electronic) in the discarded line-printer outputs....
Brian Roemmele says they've been dumpster diving for decades salvaging huge collections of microfilm/microfiche that's been thrown out by libraries, research institutions, etc.
The supercomputers of the time were very heavily designed to run floating-point operations (IIRC) and so while the FPU performance might be comparable, I'm not sure a Cray could be used as a "1/4 speed Pi" for general computing things like running Linux.
I used a Cray around 1998 (from the Pittsburgh Supercomputing Center IIRC) and it was super fast on very particular tasks. Specifically, there was some type of processing pipeline that once you had it set up, it would produce a stream of calculations very quickly.
I wonder if the Raspberry Pi is faster on all tasks, or is there some type of computation the old Cray is still competitive?
I suspect the Cray is "competitive" for some value of "doesn't absolutely stink" for things that are designed for it.
But you can emulate a Cray on an FPGA: https://www.chrisfenton.com/homebrew-cray-1a/ so I suspect that while it could still do "real work" you can also beat the pants off it if you setup your code as designed to run on modern GPUs.
Remarkable, yes. Shocking, no. Exponential growth was something experience in the computer industry for decades, and people where quite normalized to it.
That was on a 700 MHz Raspberry Pi 1. On an 1800 MHz Raspberry Pi 400 NEON SIMD the difference was another order of magnitude.
[QUOTE]
Comparison - The three 700 MHz Pi 1 main measurements (Loops, Linpack and Whetstone) were 55, 42 and 94 MFLOPS, with the four gains over Cray 1 being 8.8 times for MHz and 4.6, 1.6, 15.7 times for MFLOPS.
The 2020 1800 MHz Pi 400 provided 819, 1147 and 498 MFLOPS, with MHz speed gains of 23 times and 69, 42 and 83 times for MFLOPS. With more advanced SIMD options, the 64 bit compilation produced Cray 1 MFLOPS gains of 78.8, 49.5 and 95.5 times.[/QUOTE]
"1/4 the speed of a Pi" applies to the original (slow) 2012 Pi which is unable to run LLM as fast as you think. However the 2020 Pi 400 (equivalent to Pi 4), which can run the LLM workload, is about 100 times faster than the Cray 1:
"Raspberry Pi ARM CPUs - The comment above was for the 2012 Pi 1. In 2020, the Pi 400 average Livermore Loops, Linpack and Whetstone MFLOPS reached 78.8, 49.5 and 95.5 times faster than the Cray 1." http://www.roylongbottom.org.uk/Cray%201%20Supercomputer%20P...
A Pi 4 can infer ~0.8 tokens/sec with some of the more optimized configs (as per https://www.dfrobot.com/blog-13498.html). So the Cray would have needed ~2 minutes per token, so ~2.5 hours to generate one sentence... if hypothetically it had enough RAM (it didn't).
In 1978 RAM cost about $25k per megabyte (https://jcmit.net/memoryprice.htm). Assuming you needed 4GB for inference, RAM would have cost $100M in 1978 dollars, or $470M in today's dollars.
For comparison, the Cray cost $7M in 1978 which is $32M in today's dollars. So once you buy a Cray you would have had to spend 14 times that amount on building a custom RAM device extension of 4GB, somehow hooked to the Cray, to finally be able to generate one sentence every 2.5 hours...
But in 1978, even if RAM was available to do LLM inference, it would have been impossible to train the model, as vastly more compute power is needed than for inference.
The shocking thing is that every contemporary PC and handheld device would place on the TOP500 list in the 90's yet they're still burdened with slow software when doing basic operations.
Another really fun Cray comparison: Turner Whitted (“father of ray tracing”) is rumored to have speculated some time back when he first published on ray tracing that in order to do real time ray tracing, you could put one Cray supercomputer per pixel out in the desert, each one with a single colored light, and view it from an airplane, and that would be roughly enough compute to achieve real time.
A 4090 today is roughly 500,000 times faster, which means we now have achieved one Cray per pixel (!) for an 800x600 image (smaller than images today, but maybe a bit larger than the average image size in the late 70s).
Sure, to get Cyberpunk 2077 at 4K 60FPS, we cheat like hell. But, a scene from 70s ray tracing research at 800x600x24 fps on a 4090 doesn’t need to cheat much.
In a way, yes, partly because we’re rendering typically larger images now (1080p is 4x larger than 800x600), but also because the image quality standards are very high, and because scenes are enormous compared to a pair of spheres and a checkerboard plane with a single light, because games don’t get to spend the entire frame budget on rays, and because we’re using a lot more stochastic sampling now using path tracing than what Turner Whitted did. We do get the 500,000x relative to Cray, and denoising effectively adds another x-factor on top of that, and that’s what it takes to get today’s games up to passable real time ray tracing. On the other hand, genuine real-time ray tracing without denoising has been possible for a long time for toy scenes outside of games, it all depends on what goal posts we’re talking about exactly, right?
I have not... heard that idea before. Instead, I heard "reality is just 100 million polygons per second" (Jim Clark?), implying that if you can do ray tracing at a high resolution and frame rate, you can fool somebody's optical nerves into thinking they are looking at reality (ignoring the difference between pixel screens and the physics of how human vision actually works).
Does anybody have a reliable link to the Whitted quote?
BTW, mostly unrelated, but scrub that video to 5:47:10 for an amazing talk by Ivan Sutherland (“father of computer graphics”) that is not about graphics (he politely refuses to talk about graphics anymore :P), but about the active research he’s been doing (at 85 years old) into Single Quantum Flux circuits (an alternative to CMOS).
Also- Jim Clark at 4:48:40
Jim’s idea you mention is still correct & compatible with Turner’s idea. Jim’s point is that we only need to render finite pixels. You might need an x-factor more triangles than pixels because of sampling and depth complexity and secondary lighting, so 100M polys is probably in the ball park, as long as we can quickly pick the right 100M polys in real time…
In a way, Turner was talking about a lower bound, while Jim is talking about an upper bound, albeit slightly different things but they are similar, both relate to how much compute is needed for real time rendering.
Thanks for the timestamps. Great to see Ivan breaking ground in another area, his contribution to early graphics and interactive computing is still amazing today.
Offtopic but FYI youtube share links now come equipped with a tracking parameter (si=). You can safely leave the querystring behind if you don't want Google to associate your google account with other places you post online.
There was a comment on .. slashdot or similar, where someone explained that the massive L0 cache of the CRAY (8MB ?) meant it could sustain it's "limited" throughput, whereas an intel quad core would peak above then plummet due to spilling.
Cray didn't believe in caches or virtual memory. He felt that data placement and movement should be 100% in the programmer's control.
8MB was the main memory, but it was made from really high speed (and super expensive!) static RAM chips instead of the dynamic RAMs most other machines used. Other machines used SRAMs for caches, so I guess you could consider the Cray 1's main memory to be a cache.
The Cray had 8 "A" address registers but also 64 "B" registers. In the same way you had 8 "S" scalar registers plus 64 "T" registers. The main memory was highly interleaved so you could very quickly load and save blocks of B and T registers as well as the vector registers. You can think of B as a sort of cache for A and T as a sort of cache for S, but you had to explicitly handle this in your program.
I probably distorted the original comment I read. The idea being having massive bandwidth so your processor could iterate over a very large dataset (for the era) constantly. Unlike mainstream processors, with way faster clock speeds but more bottlenecks, so for personal computing workload they were perfectly suited, but anything larger would cause performance drops.
I’m pretty sure it was here that I read how cray would drive on a family trip and insist the kids stay silent while he drove and designed much the cray in his head on the drive… if anyone has a link would love to re read that story
By the time the Cray-1 was designed, Seymour Crays children where in college. This might have applied to when he was designing the CDC 6600 when the children were younger. I know this because the Cray kids were a couple years ahead of me in high school. There are a lot of stories like this put out by John Rollwagen to build up Seymour's design creds.
Some advantages of Cray also included the fact that your purchase/commission included a cadre of support guys who would pretty much immediately show up with replacement parts and slide logic boards in/out until your system was repaired. This sort of service speaks to both the kind of modularity that we've lost with SBCs, as well as the enterprise service levels available with high-end equipment like that.
99 comments
[ 2.9 ms ] story [ 151 ms ] thread[1] http://www.roylongbottom.org.uk/Cray%201%20Supercomputer%20P...
Cray-1 vs Raspberry Pi - https://news.ycombinator.com/item?id=38758355 - Dec 2023 (180 comments)
[1] http://www.roylongbottom.org.uk/Cray%201%20Supercomputer%20P...
[2] https://archive.is/a99i3
I've always taken them with a grain of salt, but even if they were only an order of magnitude off, a Pi is loads faster than a sidekick. And sure the Cray is loads faster than the Apollo computers, but I wouldn't have thought it was THAT much faster.
I am amazed.
Also, fun fact, it didn't have a CPU. It used all discrete logic chips, and was wired by hand, with lots and lots of wire. IIRC Seymour Cray liked to hire women to do the wiring job, because they had an easier time fitting inside the computer core to wire it, and doing detailed work because they had smaller hands.
That's how CPUs were built back then. What it didn't have was a single-chip CPU, or a microprocessor.
Next up: "The Model T Ford didn't have an engine, it had this gasoline-burning device to provide motive power."
Take a look at all the PCB traces near RAM and CPU. You can see them 'squiggle' looking for matching-space at either end of the RAM/CPU connection.
PCBs make things like consistent length matching easier. All circuit boards have near identical lengths, controlled impedances and other features that support ... Well... Effectively 6-billion baudrates for DDR5.
6000Mhz DDR5 with 64-bits per half-clock (3000Mhz clock so 6000 transfers per second) has a bit every 5 centimeters or so.
Err...
That term is used to describe modern vector instructions that are NOT vector instructions in the classical Cray-sense. Modern CPUs use wide registers that can be regarded as vectors of 2/4/8/... values. Cray used memory-to-memory variable-length vectors.
I'm not saying that "SIMD vector instructions" is an incorrect description of what the Cray machines did, I'm saying that the term usually means something else today.
edit: some searching later:
https://www.goodreads.com/en/book/show/116196
You get something completely different if failure is not an option.
But also yes, it is a big leap. NASA bootstrapped the semiconductor industry by buying up most of the world's supply. Without the Apollo program we may only just now have gotten smartphones. (And some people still think it's a waste of money. pff!)
The cray was 10,000 lb and 115 kilowatts. Not payload friendly.
I'm imagining Kevin Bacon doing his Apollo 13 power budgeting scene, but needs another ~114kw.
Some of those people are right here. The number of times I've seen the 'whitey on the moon' nonsense is quite large.
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
And probably most of those people don't realize they wouldn't be posting anything at all if not for 'whitey on the moon', the Apollo program era started with sliderules in 1962 and ended in 1972 with functional microcomputers (8008, 1972, shortly followed by the 6502, 1975) and a very short while later we had programmable pocket calculators (Ti59, 1977). The whole semi conductor industry was jumpstarted in those years.
When I first started out with electronics transistors were an absolute rarity and tubes were normal, in the space of a decade that changed completely.
I strongly disagree with that. The semiconductor industry was thriving before NASA even existed, none of the central enabling inventions (MOSFET transistors, semiconductor manufacturing techniques) were made there or even related to the Apollo program, and "computers" as in turing complete machines already existed before NASA and had plenty of applications apart from space travel.
NASA/Apollo giving todays smartphones a 10 year technology boost is just pure fiction and not even remotely supported by facts.
> By the mid-1960s, according to the PBS documentary, NASA was buying 60 percent of the integrated circuits produced in the United States. Fairchild was a major supplier, shipping about 100,000 devices for the Apollo space program in 1964 alone.
Radiation hardened electronics are fascinating to me. To this day, electronics that function in outer space are much slower than datacenter or consumer-oriented components.
The rPi 4 is over 10x faster than the 1.
The original article points out that the raspberry pi 400, with the benchmark targeting 64 bits, was about 80x Cray for the same benchmark which was 4.5x for the rPi 1.
Raspberry Pi 5 advertises a 2-3x performance gain over earlier versions, though this wasn’t benchmarked in these tests.
As to ARM in general, here's a post about how ARM-1 compared to the 387: https://retrocomputing.stackexchange.com/questions/24826/did...
80486
I think the meaning of my comment was clear: the 80486 was the first Intel CPU guaranteed to have an FPU. I didn't say all the Intel CPUs which came afterward were guaranteed to have them...
Looking over some old info about the "i486" (I swear I had completely forgotten about the i) was a treat. It's only barely believable but I think the sheer annoyingness of Intel's marketing and market segmentation might have actually peaked three decades ago.
It looks like the Cray gets a bit of a boost from the linpack scores (the pi is only 1.6x faster!), which is a good test for the Cray (understatement!).
Well, alright, you can sit on one once.
http://www.clustered-pi.com/blog/clustered-pi-zero.html
The applications have grown as well.
The Cray 1 was used for mundane tasks like "large-scale scientific applications, such as simulating complex physical phenomena, and was sold to government and university laboratories." [1] But the power of the Raspberry Pi allows for cutting edge computing tasks like "watering plants, monitoring the birds in your yard, or for a smart doorbell!" [2]
[1] https://www.britannica.com/topic/Cray-1
[2] https://picockpit.com/raspberry-pi/the-7-most-common-uses-fo...
> This means that the Cray could run it
Gladly, we had better things to do than that :D
But seriously, while we could have run it maybe speedwise, it definitely lacked the memory, not? And if one tried to train it he wouldn't be finished today. But would make a fun backwards sci-fi story imagining a time traveller that brought the 80ies an LLM from today, what would the world say and do with that slow oracle?
What value does an LLM hold intrinsically.
Lets say "brought an LLM from today"
Does that mean just a multi gig file? What is INSIDE the LLM that would be of value? How does one speak to an LLM WRT 80's tech, and what could one glean from it....
ELI5 an LLM;
BARD: https://i.imgur.com/ahRVECz.png
OpenAI: https://i.imgur.com/Rbk5BD6.png
Bing: https://i.imgur.com/zVJ1tu6.png
--
So, how would one explain 80s folks what even an LLM is when we cant even ELI5 2024?
An LLM is a very highly compressed store of knowledge combined with an advanced parser than understands questions in plain English. A consequence of the compression is that sometimes the answers lose some accuracy, which is a deliberate trade-off to make it work at all.
LLM could explain itself what it is.. (if there are not more important questions to ask, contention would ensue).
I think the process of collecting and storing all the data would be more mind blowing to them—of course they were at the beginning of Moore’s law, so they could see the trajectory if they looked for it, but it is one thing to stand on the coast with waves lapping at your ankles and imagine how the ocean gets deeper as you keep going and another to get chucked out of a helicopter in the middle of the Pacific.
The hard part of LLMs (and current AI in general) is training, which is orders of magnitude harder than inference.
If somehow we had a way to travel to the future in the 70s, train the models and then come back, we would be in Star Trek right now
But... lets look at the availability of DATA in the 80s..
Frankly, this is how hacking/phreaking was invented.
Dumpster-diving for line-printer discards in dumpsters to understand what their systems did.
(This is an actual story; people were bin dipping (at&t?) dumpsters and finding exploits (social or electronic) in the discarded line-printer outputs....
Can someone validate that comment?
--
Brian Roemmele says they've been dumpster diving for decades salvaging huge collections of microfilm/microfiche that's been thrown out by libraries, research institutions, etc.
Now that LLMs are here, they're taking that collection and training an LLM against it (instead of the internet): https://twitter.com/BrianRoemmele/status/1746945969533665422
I wonder if the Raspberry Pi is faster on all tasks, or is there some type of computation the old Cray is still competitive?
But you can emulate a Cray on an FPGA: https://www.chrisfenton.com/homebrew-cray-1a/ so I suspect that while it could still do "real work" you can also beat the pants off it if you setup your code as designed to run on modern GPUs.
[QUOTE] Comparison - The three 700 MHz Pi 1 main measurements (Loops, Linpack and Whetstone) were 55, 42 and 94 MFLOPS, with the four gains over Cray 1 being 8.8 times for MHz and 4.6, 1.6, 15.7 times for MFLOPS.
The 2020 1800 MHz Pi 400 provided 819, 1147 and 498 MFLOPS, with MHz speed gains of 23 times and 69, 42 and 83 times for MFLOPS. With more advanced SIMD options, the 64 bit compilation produced Cray 1 MFLOPS gains of 78.8, 49.5 and 95.5 times.[/QUOTE]
"Raspberry Pi ARM CPUs - The comment above was for the 2012 Pi 1. In 2020, the Pi 400 average Livermore Loops, Linpack and Whetstone MFLOPS reached 78.8, 49.5 and 95.5 times faster than the Cray 1." http://www.roylongbottom.org.uk/Cray%201%20Supercomputer%20P...
A Pi 4 can infer ~0.8 tokens/sec with some of the more optimized configs (as per https://www.dfrobot.com/blog-13498.html). So the Cray would have needed ~2 minutes per token, so ~2.5 hours to generate one sentence... if hypothetically it had enough RAM (it didn't).
In 1978 RAM cost about $25k per megabyte (https://jcmit.net/memoryprice.htm). Assuming you needed 4GB for inference, RAM would have cost $100M in 1978 dollars, or $470M in today's dollars.
For comparison, the Cray cost $7M in 1978 which is $32M in today's dollars. So once you buy a Cray you would have had to spend 14 times that amount on building a custom RAM device extension of 4GB, somehow hooked to the Cray, to finally be able to generate one sentence every 2.5 hours...
But in 1978, even if RAM was available to do LLM inference, it would have been impossible to train the model, as vastly more compute power is needed than for inference.
Besides, it's way further behind in basically every respect but compute.
A 4090 today is roughly 500,000 times faster, which means we now have achieved one Cray per pixel (!) for an 800x600 image (smaller than images today, but maybe a bit larger than the average image size in the late 70s).
Does anybody have a reliable link to the Whitted quote?
https://www.youtube.com/live/LUFp6sjKbkE?si=8vcxo-Vp8oeRUnob
Scrub to 3:52:19 for the Turner Whitted story.
BTW, mostly unrelated, but scrub that video to 5:47:10 for an amazing talk by Ivan Sutherland (“father of computer graphics”) that is not about graphics (he politely refuses to talk about graphics anymore :P), but about the active research he’s been doing (at 85 years old) into Single Quantum Flux circuits (an alternative to CMOS).
Also- Jim Clark at 4:48:40
Jim’s idea you mention is still correct & compatible with Turner’s idea. Jim’s point is that we only need to render finite pixels. You might need an x-factor more triangles than pixels because of sampling and depth complexity and secondary lighting, so 100M polys is probably in the ball park, as long as we can quickly pick the right 100M polys in real time…
In a way, Turner was talking about a lower bound, while Jim is talking about an upper bound, albeit slightly different things but they are similar, both relate to how much compute is needed for real time rendering.
8MB was the main memory, but it was made from really high speed (and super expensive!) static RAM chips instead of the dynamic RAMs most other machines used. Other machines used SRAMs for caches, so I guess you could consider the Cray 1's main memory to be a cache.
The Cray had 8 "A" address registers but also 64 "B" registers. In the same way you had 8 "S" scalar registers plus 64 "T" registers. The main memory was highly interleaved so you could very quickly load and save blocks of B and T registers as well as the vector registers. You can think of B as a sort of cache for A and T as a sort of cache for S, but you had to explicitly handle this in your program.
Thanks for all your informations.
https://imgflip.com/i/8cwqf1
Seriously though, I think the fact that the PI is general purpose makes it even more impressive.