What I think is more amazing is that with the exception of the 2 Windows systems, EVERYTHING is either AIX UNIX or Linux, or a combination. UNIX started as a "microcomputer" OS, but it really has proven to be incredibly flexible. Personally, I think the ideas that have come out of the Solaris and BSD communities are invaluable and I always like to see them included in projects - I'm a little disappointed they have lost favor in the HPC community. They are similar enough to Linux and other UNIX-like OSs that targeting all of them is relatively practical in most cases, and users will find similar interfaces everywhere, but their ideas, communities and approaches are different enough that I think they would prevent it from being a monoculture.
One of the things that imo has pushed some of the other Unixes out of HPC is the move to using cluster file systems. Any Unix supports a traditional NFS-mounted NAS solution for storage, as used to be typical in HPC clusters (and still is fairly common in smaller installations at universities). But only AIX and Linux really have solid cluster filesystems; there's nothing in the BSD world analogous to IBM GPFS, or to Lustre, and porting is nontrivial.
I think you're right. DragonflyBSD probably isn't widely used enough to be trusted by high-budget HPC projects anyway, but I was under the impression their HAMMER filesystem was a cluster file system. Looking at the website now I can't seem to find support for that, though...
One very minor nit...the PDP-7 where Unix started out cost US$72k+ in 1969 dollars. Nothing compared to the big iron of the day, but very far from a microcomputer (and, really, that market didn't exist until 1972 or so).
And as far as capabilities go, the PDP-7 could address 64K of 18 bit words, more than double 1st generation microcomputers (those prior to the IBM PC and Apple Macintosh). Then again, I don't think it's generally known how much (very expensive, hand made) core memory was on that first UNIX(TM) PDP-7.
Maybe not that much, it next moved to the first model of the PDP-11, which had a max of 56 KiB memory of 8 bit bytes. Then to an 11/45, which could do 64 KiB of code, 56 KiB of data and 8 KiB of stack.
The 11/45 has a 16-bit address space optionally expanded to 18-bits with an MMU so could do 256KB of memory. With the MMU, you could take advantage of split instruction and data memory, which gave a pair of 64k address spaces mapped into the 18-bit space.
No PDP-11 AFAIK have a separate, dedicated stack space. You're thinking of the 8k dedicated I/O page.
Ah, yes, for the 11/45 I was referring to what a single program could do.
However, as I recall (from the very early '80s when I was using both a 11/44 and 11/45), the MMU had a 8 KiB page size, so the natural thing to do was to divide the data as described, so a stack overflow would trigger a trap. Although after a bit of time with Google, I note the 11/40 manual says all PDP-11s have a location in which to put a stack boundary, so maybe that was a convention which matched what the MMU could do.
About 99% of all devices "just work" out of the box, the other 1% either are very obscure/specialty hardware, or are new hardware. Linux doesn't need a vendor to support Linux like it used to... most generic drivers are built in and work OK (just like the generic drivers built into Windows now-a-days).
> About 99% of all devices "just work" out of the box, the other 1% either are very obscure/specialty hardware, or are new hardware
Well if you count most GPUs under speciality hardware then I guess you're right, but most people don't. And most linux GPU drivers still pale in comparison to their Windows versions.
Actually I would think GPUs are one of the biggest counterexamples where driver support has been noticeably improving recently (mostly thanks to Valve and Steam). Even the open source drivers work well enough to let you get work done (although the performance might not be where you want it to be).
Compare that to driver support for things like the fingerprint reader in a Lenovo Thinkpad. I wouldn't want to bet on getting that working at all, unless Lenovo had preinstalled Linux for you. And unlike GPUs, this is likely to be all or nothing, so if you don't get the drivers you can't use that hardware feature at all.
I was given a free Chromebox (Samsung "Series 3"), and I tried to install Linux on it. It didn't work at all, apparently because the Chromebox doesn't support "legacy boot," whatever that is. I tried Crouton and Chrubuntu in turn, and found that sound didn't work in the first case, and that the kernel was not upgradeable.
I gave up and I bought a PC (Intel NUC). I was able to install Linux on it, but it failed to boot. I then found a BIOS update, and managed to get it installed (not a trivial task). After that my NUC worked well, except for the IR port, which required additional driver hunting.
Sadly, we're still a long ways from "it just works." The NUC didn't support Linux out of the box, even though it ships without an OS!
You are describing customized devices which are build for a specific OS. Windows and MacOS don't work well on Chromeboxs either.
Also to my knowledge all modern Chrome devices have Coreboot support which enable "legacy" bootloaders and will boot any distro like a normal laptop. Older models like the C710 have unofficial Coreboot builds which allow the same (and linux driver support on modern distros).
For some users, of course, that 1% is all that matters (see Chromebook complaints). I don't find that a very compelling reason to seek out patent encumbered OSes but to each their own.
I still can't fully utilize the nvidia Optimus gfx card under Linux. I've tried the bumblebee project and other suggestions I've found through google, but I'm still stuck with minimal capability. I hear that the latest stable release of ubuntu (14.04?) has added more support for it, so maybe that will work for me.
my optimus card in my w520 works perfectly in 14.04 - and there are task bar widgets for switching now (logout required). After installing 14.04 my external monitors and display switching work perfectly! I was really happy as I was stuck with a non functional gfx card in linux for a few years and now it finally works.
I upgraded to ubuntu 14.04 on my reasonably late-model desktop from a major name and now my nvidia graphics card doesn't cause the x server to segfault and send me back to a login prompt when rendering certain fonts, but now my keyboard stops working after screen lock about 10% of the time and sometimes half the usb ports will shut down for no reason at all.
This is a little better then when I was hand-editing x configs back in the 90s but overall I'm disappointed.
Gosh, I see so many people having issues like this... and get really confused, because I've ran Linux on my laptops for the past 4 years, and they've all worked completely out of the box with whatever version of Ubuntu was the latest at the time! And I don't go out of my way to pick a laptop that supports Linux, I just buy whichever fit my requirements the best.
Of course, now I'm just using a rMBP, but yeah, I wonder if I've just been very lucky?
While to say that all Linux drivers are crappy is false, it's as much as false the (almost) opposite.
In Linux, I always have problems with devices, in one way or another (specifically Ubuntu, but I think it's fairly representing the concept), both for new things, old things, not-so-new things, obscure things, etc. etc.
To complicate the problem, drivers fit in stacks that may interact in unexpectedly broken ways. This means that say, a wireless audio adapter, may work only 50% of the times it's connected. I was using Windows long ago, and I don't remember this category of things happening (I guess because there are only a few Windows).
So, certainly they work "OK", but one needs to be quite flexible on what OK means.
I've got two machines which are not so arcane, and I have at least 4, hardware-related (not stack-related), open bugs.
That is true in my experience, sooner or later you are going to run into a silly driver related problem with linux. Right now I'm having problems with the integrated Intel soundcard in my laptop. Using Xubuntu 10.10/Fedora 16 there never was any problem with it, ever since then it sometimes won't output to the speakers but I always get sound on the headphone jack. But it's mostly those peripheral things at least for me, the system as whole usually works pretty well.
Thing is, Linux is much more enjoyable to use for software development and deployment, development on Windows makes me feel like a second class citizen, the OS to me feels much more oriented towards the exclusive consumer, especially now with the new (metro) tablet like user interface.
Well, Valve already has ToGL, a DirectX2OpenGL abstraction layer... so that is already happening. Also, SteamOS (and Steam on Linux) now has low latency Streaming from your Windows pc to your Linux PC, SteamOS/SteamBox on your TV, etc.
So, in large part, just about all games are now playable on Linux, and more are coming as developers re-release games on Linux, or as many big AAA studios have committed to releasing all future games on linux as well (Crytek, Valve, Unreal, Firaxis Game, etc etc... there are a lot and more coming!)
More routers run linux than you'd think. There's very little reason to reinvent basic networking and routing components and a kernel, when all the common SoCs are supported by a free OS. For the small price of stashing a source code bundle somewhere on their website (if even!) companies can get most of what makes their product tick off the shelf.
What significant networking vendor runs OpenBSD? And I don't mean the low-end, low-support, OpenBSD on an embedded board from China stuff (so, no, Soekris isn't a significant vendor in the router space). I can't think of a single business or enterprise class one that does.
Cisco doesn't use it (IOS is home grown, IOS XR is QNX, NX-OS is Linux), Juniper uses FreeBSD, Arista uses Linux, HP uses (I think) Comware (linux), Intel & Broadcom are pushing Cumulus (linux) on their reference networking platforms, Dell/Force10 is NetBSD based (and older Dell is VXWorks, I think), Pretty sure Brocade is Linux, but they've also used VXWorks.
Don't know what Hauwei uses (that wasn't stolen from Cisco, or course).
Not that this is authoritative, but I've talked to a couple of execs at companies who were using OSS platforms for their networking devices (Nokia, Juniper, Force10, Arista) and all of them basically said OpenBSD wasn't a consideration because of the open hostility the people leading the project had towards commercial enterprises. No one was interested in betting the company with the threat of a "DARPA funding" moment hanging over them, much less someone who was more interested in dictating terms and making "statements" than cooperation on business relationships.
I believe the OpenBSD community of developpers is not too concerned with popularity or integration with vendor's hardware. They are more concerned with doing things right (or, some would say, doing things their own way).
Nevertheless, there are some small companies selling things based on OpenBSD - even though, I'll admit, they are mostly companies started by OpenBSD developpers:
And, of course, there is always the strong possibility that some people are using OpenBSD while keeping it hush-hush and "under the radar" -- maybe even under the radar of their own management.
I would certainly say "doing their own way" rather than "doing it right", and I would never say "wrong", for most values of wrong. I'm merely pointing out to the OP that other than the "side projects" (e.g. SSH), there is for all practical purposes zero uptake of OpenBSD in the commercial networking space. Yeah, there are some very niche players and lots of DIY folks, but that's not what was being alleged.
Are all companies which have such stores traded publicly? I don't see a direct connection between these two aspects. For instance, in China there are various Android markets and I'm not sure all of them belong to publicly traded companies.
Publicly traded companies are required to disclose revenue, but not required to break it down by source.
Neither Apple or Google break out the revenue from the *Stores (unless they have started doing it very recently). There are third party analytics firms that provide reasonable estimates though[1].
Mobile is interesting, because while the kernel is theoretically GPL you can't normally replace it with your own kernel without using a security exploit against your own device to get root. And almost all the userland is closed as well. It's far harder to run a properly Free operating system on a phone that nominally already runs Linux than it is to install Linux on a PC shipped with Windows.
On mobile, Linux is a thin shim of Free software in a proprietary stack.
A lot of broadcast TV is all windows/mac. Even for things like playout servers, automation, and real time graphics engines mostly run on windows or mac.
It's worth pointing out that this is what most people think of as the traditional "Linux Operating System" but sans the Linux kernel.
Also, there are a few components on these supercomputers that you won't find on a typical workstation or cluster machine.
First, the kernel itself is frequently very small, lightweight, and much closer to what you would find on an embedded system than on a traditional desktop computer. That's because once the program on a supercomputer is loaded, the kernel's job has been mostly to get out of the way. This isn't to say that you don't find the Linux kernel on these supercomputers, it just isn't as common as you would think from reading this piece.
Second, many of these computers primarily only run code in C, C++, Fortran, and Python. These tend to be the only major languages in play on the HPC machines, with acceleration frameworks such as OpenMP, OpenCL, and CUDA playing major roles.
Finally, everything is glued together with MPI, a high-level (at least it was in the 90s) abstraction for scientific programming that maps down to very high-performance networks designed to help scientific codes "scale", that is, run effectively when millions of cores are simultaneously engaged.
These are beautiful machines producing important science, and the GNU/Linux operating system plays an incredibly important role in both their implementation and culture.
This is not exactly true, most of the ones I know are running straight up Linux. Titan, for example, runs basically SLES on the login nodes and Compute Node Linux on the compute nodes. CNL is a pared down version of Linux, but it definitely is the "real" Linux kernel with the functionality you'd expect to be there.
Most are just slightly spruced up commodity server hardware running Linux. I'm not sure if this is what you're suggesting, but they don't run C/Fortran/whatever on bare metal. They're run by the OS on the compute node just like a normal OS process, except that tasks are dispatched to compute nodes by a central cluster manager. Processes running in a gang communicate via MPI to share data, though coprocessors are also pretty popular as well so you see a lot of communication between the host processor and a coprocessor too. Titan and Tianhe both actually have most of their compute power in the coprocessors (Xeon Phi and Nvidia Tesla, respectively), but they're still arranged in a master-slave arrangement just like if you bought a Phi or Tesla and stuck it in a spare PCI-E slot. They use plain old PCI-E, too. The Cray XT/XE series (a popular model of which Titan is an example) is basically just really nice blades with integrated cooling and a network backbone in a custom cabinet and possibly coprocessors attached to each blade. You could just as easily run Windows XP and play Minesweeper on each blade if you really wanted to, except maybe for some driver issues. The most foreign thing is probably the network backbones, where fabric architectures like Infiniband are popular.
They're also not limited to specific programming languages. In truth, you can run whatever you want if someone has paid the bill for your resource allocation. I watch people run MATLAB on large clusters all the time, which hurts me because it's so damned inefficient. That said, Fortran and C++ comprise the overwhelming majority of large and computationally taxing codes. Just because all that power is there doesn't mean that all of the users take proper advantage of it. One of the larger calculations run on Titan that I know of (Denovo, a nuclear reactor simulation code) didn't even use the GPUs, only the CPUs. Making codes that can take advantage of GPU processing ergo Titan and its predecessor Jaguar has been a major project at the DOE, with libraries like Trilinos being developed to make it easier on scientists, many of whom are only computer programmers as a secondary concern.
The setup you're describing used to be how it was until up to maybe 10 years ago and there are still systems in the top 500 that work like that. Probably some new ones being built, too. But what I've described is what seems to be in fashion these days mostly and the machines I use are all like that. I've heard mumblings about FPGA coprocessors being the Next Big Thing, but we will see.
Yup, thanks for the comment. I think FPGA has interesting potential for sequencing (and we need more sequencing compute), I'm not sure what's ahead for the simulation machines.
FPGA has potential for simulation too from what I know, but I honestly don't know enough about them to comment intelligently about them. I expect most people will use them via an intermediate abstraction layer, as from what I know they can be tough to program for.
Yes, and also a fairly basic research fail, referring to the ECMWF's old supercomputer at position 60 as the UK's weather predicting system. They have couple of new systems at positions 19 and 20. ECMWF is for Europe, not UK, even if it is based in Reading. The key is in the name.
Another way to put it is that specialized supercomputers no longer exist, and the top500 is now populated with super-branded racks of PCs running Linux with souped up network cards.
For the young ones in the audience, the supercomputer term has historically been used to describe machines with largely custom hardware from the CPUs up - or at least from the CPUs onwards. Cray vector machines are the canonical example. These were shared-memory / single-system-image machines, software really saw it as one computer instead of a networked cluster.
I don't think this description is entirely fair. What is expensive with these computers is the fast interconnection buses between processors and modules, which is needed for many scientific problems that can't easily be divided into independent subproblems. Just having a large cluseter of computers connected with normal Ethernet would not cut it. OK, you say souped out network cards, and if you had said souped up switches as well, I could have given you some points, but this is the major cost of modern supercomputers. You can get a non souped-up cluster with the same number of processors for a one digit percentage of the price of the supercomputer.
Pretty sure everything in top500 is using infiniband by now.
I was briefly involved with the supercomputing world about four years ago. "Should we use Infiniband?" was no longer considered a non-rhetorical question even then.
It's either Infiniband or some other high speed interconnect. It's not exclusively Infiniband, for instance the Chinese developed their own, but it's usually something similar.
> Another way to put it is that specialized supercomputers no longer exist, and the top500 is now populated with super-branded racks of PCs running Linux with souped up network cards.
While there's a grain of truth in this, in reality, a supercomputer still looks like a room full of refrigerators and not at all like a typical computer or even a server farm.
Some of the computer hardware in there is just commodity hardware but for example the interconnects can't really be described as "souped up network cards", they're in a category of their own.
Yes, the era of the Cray boxes is gone but a supercomputer is still a supercomputer.
They're not exactly "super-branded" racks of "PC"s the way we tend to think about them, a bunch of white/beige boxes. They are server-grade computers, with all that comes there (special interconnect features, ECC memory and all that), but there is indeed no question as to their distributed nature.
This is probably a technological response to the rapid pace of development. It's more efficient this way, instead of developing custom chips and boards for two years and deploying them when the high pace of advance makes them half-obsolete.
In addition to just the networking hardware, the networking stacks and topologies are very different. I would consider something a supercomputer if it was using a high-dimension torus topology, and some MPI variant, and a different internet / transport protocol (I forget the name, but as I understand it's a very different approach where there's much less decision making in the routers). On the other hand, what I would consider to be a cluster is almost going to use off-the-shelf networking hardware (as you mention), TCP/IP, and a simpler hierarchical topology with a switch or two per rack.
In the article it was mentioned that ECMWF's supercomputer was the fastest one of those running not on Linux. It's worth mentioning that ECMWF is in the process of migrating to an XC30 Cray system which runs on Linux.
85 comments
[ 3.0 ms ] story [ 144 ms ] threadAfter years of being a Windows fanboy -- I look back now and am very glad I made the switch. With Linux -- it's never a case of "you can't do that".
One very minor nit...the PDP-7 where Unix started out cost US$72k+ in 1969 dollars. Nothing compared to the big iron of the day, but very far from a microcomputer (and, really, that market didn't exist until 1972 or so).
Maybe not that much, it next moved to the first model of the PDP-11, which had a max of 56 KiB memory of 8 bit bytes. Then to an 11/45, which could do 64 KiB of code, 56 KiB of data and 8 KiB of stack.
No PDP-11 AFAIK have a separate, dedicated stack space. You're thinking of the 8k dedicated I/O page.
However, as I recall (from the very early '80s when I was using both a 11/44 and 11/45), the MMU had a 8 KiB page size, so the natural thing to do was to divide the data as described, so a stack overflow would trigger a trap. Although after a bit of time with Google, I note the 11/40 manual says all PDP-11s have a location in which to put a stack boundary, so maybe that was a convention which matched what the MMU could do.
About 99% of all devices "just work" out of the box, the other 1% either are very obscure/specialty hardware, or are new hardware. Linux doesn't need a vendor to support Linux like it used to... most generic drivers are built in and work OK (just like the generic drivers built into Windows now-a-days).
Well if you count most GPUs under speciality hardware then I guess you're right, but most people don't. And most linux GPU drivers still pale in comparison to their Windows versions.
Compare that to driver support for things like the fingerprint reader in a Lenovo Thinkpad. I wouldn't want to bet on getting that working at all, unless Lenovo had preinstalled Linux for you. And unlike GPUs, this is likely to be all or nothing, so if you don't get the drivers you can't use that hardware feature at all.
https://launchpad.net/~fingerprint/+archive/fingerprint-gui
I gave up and I bought a PC (Intel NUC). I was able to install Linux on it, but it failed to boot. I then found a BIOS update, and managed to get it installed (not a trivial task). After that my NUC worked well, except for the IR port, which required additional driver hunting.
Sadly, we're still a long ways from "it just works." The NUC didn't support Linux out of the box, even though it ships without an OS!
Also to my knowledge all modern Chrome devices have Coreboot support which enable "legacy" bootloaders and will boot any distro like a normal laptop. Older models like the C710 have unofficial Coreboot builds which allow the same (and linux driver support on modern distros).
This is a little better then when I was hand-editing x configs back in the 90s but overall I'm disappointed.
Of course, now I'm just using a rMBP, but yeah, I wonder if I've just been very lucky?
In Linux, I always have problems with devices, in one way or another (specifically Ubuntu, but I think it's fairly representing the concept), both for new things, old things, not-so-new things, obscure things, etc. etc.
To complicate the problem, drivers fit in stacks that may interact in unexpectedly broken ways. This means that say, a wireless audio adapter, may work only 50% of the times it's connected. I was using Windows long ago, and I don't remember this category of things happening (I guess because there are only a few Windows).
So, certainly they work "OK", but one needs to be quite flexible on what OK means. I've got two machines which are not so arcane, and I have at least 4, hardware-related (not stack-related), open bugs.
Thing is, Linux is much more enjoyable to use for software development and deployment, development on Windows makes me feel like a second class citizen, the OS to me feels much more oriented towards the exclusive consumer, especially now with the new (metro) tablet like user interface.
This high number is due to kernel programmers writing device drivers, since not so many manufacturers write linux drivers themselves.
If you are thinking of video driver it is a totally different story.
So, in large part, just about all games are now playable on Linux, and more are coming as developers re-release games on Linux, or as many big AAA studios have committed to releasing all future games on linux as well (Crytek, Valve, Unreal, Firaxis Game, etc etc... there are a lot and more coming!)
I can think of only a few categories: desktops, ATM machines, real time, SCADA.
What are the remaining big sectors Linux doesn't yet dominate?
Cisco doesn't use it (IOS is home grown, IOS XR is QNX, NX-OS is Linux), Juniper uses FreeBSD, Arista uses Linux, HP uses (I think) Comware (linux), Intel & Broadcom are pushing Cumulus (linux) on their reference networking platforms, Dell/Force10 is NetBSD based (and older Dell is VXWorks, I think), Pretty sure Brocade is Linux, but they've also used VXWorks. Don't know what Hauwei uses (that wasn't stolen from Cisco, or course).
Not that this is authoritative, but I've talked to a couple of execs at companies who were using OSS platforms for their networking devices (Nokia, Juniper, Force10, Arista) and all of them basically said OpenBSD wasn't a consideration because of the open hostility the people leading the project had towards commercial enterprises. No one was interested in betting the company with the threat of a "DARPA funding" moment hanging over them, much less someone who was more interested in dictating terms and making "statements" than cooperation on business relationships.
[edit for punctuation]
Nevertheless, there are some small companies selling things based on OpenBSD - even though, I'll admit, they are mostly companies started by OpenBSD developpers:
http://www.openbsd.org/products.html
You can also get some consulting & support:
http://www.openbsd.org/support.html
And, of course, there is always the strong possibility that some people are using OpenBSD while keeping it hush-hush and "under the radar" -- maybe even under the radar of their own management.
Make of this what you will.
Because Linux does dominate mobile? Android is Linux and iOS is a *nux. (iOS is a BSD derivative.)
In revenue (app-sales etc) iOS dominates.
Who made such research? I'm not sure if it's even possible to know exactly, unless all vendors who run their own stores publish sales statistics.
They do, thats one of those things that companies that are traded publicly are required to do.
Publicly traded companies are required to disclose revenue, but not required to break it down by source.
Neither Apple or Google break out the revenue from the *Stores (unless they have started doing it very recently). There are third party analytics firms that provide reasonable estimates though[1].
[1] eg http://techcrunch.com/2014/06/23/google-play-quarterly-app-r...
On mobile, Linux is a thin shim of Free software in a proprietary stack.
(Android barely counts as Linux for reasons discussed in my other comment)
Also, there are a few components on these supercomputers that you won't find on a typical workstation or cluster machine.
First, the kernel itself is frequently very small, lightweight, and much closer to what you would find on an embedded system than on a traditional desktop computer. That's because once the program on a supercomputer is loaded, the kernel's job has been mostly to get out of the way. This isn't to say that you don't find the Linux kernel on these supercomputers, it just isn't as common as you would think from reading this piece.
Fear not, many pieces of these operating systems are still open source. Here's IBM's fusedOS prototype:https://github.com/ibm-research/fusedos.
Second, many of these computers primarily only run code in C, C++, Fortran, and Python. These tend to be the only major languages in play on the HPC machines, with acceleration frameworks such as OpenMP, OpenCL, and CUDA playing major roles.
Finally, everything is glued together with MPI, a high-level (at least it was in the 90s) abstraction for scientific programming that maps down to very high-performance networks designed to help scientific codes "scale", that is, run effectively when millions of cores are simultaneously engaged.
These are beautiful machines producing important science, and the GNU/Linux operating system plays an incredibly important role in both their implementation and culture.
Most are just slightly spruced up commodity server hardware running Linux. I'm not sure if this is what you're suggesting, but they don't run C/Fortran/whatever on bare metal. They're run by the OS on the compute node just like a normal OS process, except that tasks are dispatched to compute nodes by a central cluster manager. Processes running in a gang communicate via MPI to share data, though coprocessors are also pretty popular as well so you see a lot of communication between the host processor and a coprocessor too. Titan and Tianhe both actually have most of their compute power in the coprocessors (Xeon Phi and Nvidia Tesla, respectively), but they're still arranged in a master-slave arrangement just like if you bought a Phi or Tesla and stuck it in a spare PCI-E slot. They use plain old PCI-E, too. The Cray XT/XE series (a popular model of which Titan is an example) is basically just really nice blades with integrated cooling and a network backbone in a custom cabinet and possibly coprocessors attached to each blade. You could just as easily run Windows XP and play Minesweeper on each blade if you really wanted to, except maybe for some driver issues. The most foreign thing is probably the network backbones, where fabric architectures like Infiniband are popular.
They're also not limited to specific programming languages. In truth, you can run whatever you want if someone has paid the bill for your resource allocation. I watch people run MATLAB on large clusters all the time, which hurts me because it's so damned inefficient. That said, Fortran and C++ comprise the overwhelming majority of large and computationally taxing codes. Just because all that power is there doesn't mean that all of the users take proper advantage of it. One of the larger calculations run on Titan that I know of (Denovo, a nuclear reactor simulation code) didn't even use the GPUs, only the CPUs. Making codes that can take advantage of GPU processing ergo Titan and its predecessor Jaguar has been a major project at the DOE, with libraries like Trilinos being developed to make it easier on scientists, many of whom are only computer programmers as a secondary concern.
The setup you're describing used to be how it was until up to maybe 10 years ago and there are still systems in the top 500 that work like that. Probably some new ones being built, too. But what I've described is what seems to be in fashion these days mostly and the machines I use are all like that. I've heard mumblings about FPGA coprocessors being the Next Big Thing, but we will see.
For the young ones in the audience, the supercomputer term has historically been used to describe machines with largely custom hardware from the CPUs up - or at least from the CPUs onwards. Cray vector machines are the canonical example. These were shared-memory / single-system-image machines, software really saw it as one computer instead of a networked cluster.
I was briefly involved with the supercomputing world about four years ago. "Should we use Infiniband?" was no longer considered a non-rhetorical question even then.
While there's a grain of truth in this, in reality, a supercomputer still looks like a room full of refrigerators and not at all like a typical computer or even a server farm.
Some of the computer hardware in there is just commodity hardware but for example the interconnects can't really be described as "souped up network cards", they're in a category of their own.
Yes, the era of the Cray boxes is gone but a supercomputer is still a supercomputer.
This is probably a technological response to the rapid pace of development. It's more efficient this way, instead of developing custom chips and boards for two years and deploying them when the high pace of advance makes them half-obsolete.
Edit: Here is one of the articles: http://phys.org/news/2010-12-air-playstation-3s-supercompute...