The Itanium is a noteworthy example of what happens when one designs an architecture exclusively for parallelism to the exclusion of all else, and leaves all instruction scheduling to the compiler. The performance was great when software could take advantage of the parallelism, but horrible otherwise, since the processor would still be fetching bundles of 3 instructions (16 bytes each!) but only 1/3 of them would be anything other than NOPs, effectively making each instruction a ridiculously-cache-bloating 16 bytes long.
I suppose a similar analogy would be doing everything in x86 with SIMD instructions and not using the scalar set at all.
> I suppose a similar analogy would be doing everything in x86 with SIMD instructions and not using the scalar set at all.
With ever wider SIMD units this is actually happening to some extent. With AVX512 (e.g. on KNL) you can do 64 operations (single precision FLOPs) in vector units in the same amount of time as you can do 1 scalar operation. Combined with the low clock speed of the KNL, you really don't want to be doing scalar operations if you care at all about performance. Of course SIMD isn't the same as VLIW, since the operation you're doing has to be the same in all vector lanes, but with masking support in AVX512, it is getting a little closer to that.
Also compilers have gotten quite a bit smarter and with hardware die shrinks taking longer, I wouldn't be at all surprised if VLIW ISAs were to make a comeback. I guess the Itanium was a bit ahead of its time.
This, recent compiler advances in the past two decades have actually made VLIW somewhat tractable, but since the memory hierarchy and data movement wall consumes most of the power nowadays, the benefits of VLIW are mitigated.
Even if that were so, things are moving away from VLIW. E.g. AMD switched from VLIW to a more standard load-store SIMT approach a couple years ago, and even though it's more complex for the GPU, it increased system efficiency (esp. for workloads that their compiler was not well tuned for).
Trace scheduling started with Fisher 81 [1] and Ellis 85 [2]. Trace scheduling is what made VLIW even possible if not exactly tractable. That was 30+ years ago.
What recent advances have made VLIW any more tractable than trace scheduling already did? BTW, trace scheduling works for scheduling superscalar processors as well.
VLIW never went away. It's used in embedded, the TriMedia processors. It's used in the REX Computing NEO chip. The Mill CPU is a VLIW of sorts.
I'm not anti-VLIW but I don't know of any recent breakthroughs that make it any more tractable now for non-embedded, non-HPC general purpose computing.
[1] Trace Scheduling: A Technique for Global
Microcode Compaction
Designer of the Neo here, and owner of the California "VLIW" license plate. As you can guess, I am a die hard VLIW advocate, and a strong believer that the original promises of VLIW (Drastically simpler decode logic, implicit instruction level parallelism, virtually no control/data hazards on chip).
VLIW has gotten an extremely bad rap outside of the embedded space due to Itanium, which I strongly contend was not a VLIW in spirit (which you cans see me talk in more depth on here: https://youtu.be/ki6jVXZM2XU?t=441). Itanium introduced a ton of indeterminism by having static scheduling unfriendly things like branch prediction, variable latency caches, and still trying to have some level of support for x86. The core problem for Itanium is that it was impossible for the compiler to make good decisions in the vast majority of applications since the compiler could not know exactly where and when things in memory would be. I believe the underlying failure point has been computer architects for failed VLIW architectures not realizing that fancy bells and whistles and "cleverly complex" hardware designs is the exact opposite of what you want when the compiler needs to be able to do purely static allocation. Adding fancy dynamic pieces in hardware to account from this (seen in a lot of places with Itanium) just makes the problem worse.
The main improvement we have made with the Neo architecture is we have hard (exact cycle count) guarantees on all memory movements on and off the chip, which actually gives the compiler the information it needs to make good decisions and very condensed code. I think the jury is out on the applicability for "general purpose" code on VLIWs... it is not something we at REX really care about at the moment, and I think RISC-V is a great solution with improvements over x86 and ARM.
The two best application cases of instruction level parallelism that we have at the moment are our FFT and DGEMM, where our 64 bit instruction word contains the operations for doing a double precision fused multiply add, two 128 bit load/stores, and an ALU op for address calculation. The inner loops of these programs are able to get up to 4 instructions per cycle (5 if you count the FMA as 2 ops, which is what everyone else does but I think is cheating), and compared to a superscalar architecture, uses very little space in local memory (fitting 4 operations in only 64 bits is awesome!). Compared to the latest 14/16nm offerings from Intel and NVIDIA, we have a 25x energy efficiency advantage for our FFT while our test chip is still on 28nm.
These are the same reasons why DSPs have been VLIW since the 90s, and our initial applications overlap a lot. It has been sad to see the trend of TI and other DSP makers going down the path of adding hardware caching (that reduces determinism and energy efficiency) to try to "make it easier for programmers" when they already have a decent enough architecture, but a lack of good compiler tools to take advantage of it.
Not a fan of RISC-V although Chisel is pretty cool. Still I was forced to admit that its free as in beer licensing has won some nontrivial design wins (NVidia). I think however that that was more Arm shooting itself in the foot.
My biggest complaint about RISC-V is its lack of condition codes. This is just too puritanical. I think ARMv8 strikes a nice balance here. Leaving something out that both ARM and Intel provide seems pointlessly austere.
As was pointed out in the A History of Modern 64-bit Computing cite, both RISC and Itanium show an academic bias. Yes, RISC-V is elegant and simple and that helps in a one semester upper div architecture class. But that doesn't help you much outside of a one semester upper div architecture class.
So what do you tangibly get with that elegance and simplicity? You aren't going to write RISC-V assembly; compilers are better especially for such a regular and spare ISA. This is 2017 and LLVM has been a thing for quite some time. Compilers are not a big deal.
Lower power? Chip area? How much lower power and chip area are we talking about?
I'm not too sure about the "academic bias" point, since modern "CISC" CPUs are essentially "CISC on RISC" and essentially RISC already.
That being said, greater efficiency is the main point, it's not too difficult to get a RISC-V core that beats (albeit marginally) an equivalent ARM core.
Optimizing compilers can often be difficult though, especially if you want to use novel modifications to the backend to make things more efficient.
No, Skylake is not a RISC internally. Find something from the Intel Optimization Manual or their patents saying that. I been over this already. μops are hundreds of bits wide. Ain't nothing reduced about that.
Read up on horizontal microprogramming. A RISC ISA is an encoding with an opcode. A μop is really just a long set of fields with some bits clocking some FUs on and others off. There is no decoding; there is no opcode.
Original microprogrammed CPUs had a writeable control store (WCS). Modern processors translate CISC instructions into μops and cache this translation. The difference is the μop cache (μstore) vs WCS. FWIW, Skylake also has a WCS for obscure instructions.
Microinstructions, μops, do not have an opcode. They are not decoded. Bits in a horizontal μop gate logic on/off directly.
How far away are you from getting to the point where you can accelerate workloads such as blender? Also I see that your backend isn't in the trunk -- is there plan to push to upstream and share the details? I'd love to learn about your isa.
It isn't something we have been looking at, as our current customers and target applications would have our chip as a standalone embedded processor. We are working on a prototype PCIe card with an FPGA to be able to convert our SerDes to something a PC can talk to, but it isn't really a priority unless we have someone come to us directly asking for it.
As for upstreaming, it is something that we want to do, but are not sure if it will be accepted as our chip is not generally available at the moment. It would probably be done around the same time as our next tape out (which would be in larger quantity than our first one) when we would have generally available development kits this coming winter.
I have been working on public documentation in my spare time... originally supposed to go out in March but we have had other priorities. If you watch my Stanford talk I show the basic ISA overview as well as the WIP documentation made using doxygen. I'm hoping to get stuff out on our website before our next media push in ~June.
> The main improvement we have made with the Neo architecture is we have hard (exact cycle count) guarantees on all memory movements on and off the chip ...
This is a fascinating idea, but I have a question.
What happens when the next process generation lets you improve cache timings? What about newer, faster DRAM (or whatever) timings?
Do I have to recompile the world in that case, to see any improvement in performance?
The assumption by us is that you would be recompiling from at least the LLVM IR level for each different version of the chip. This isn't a real problem for the markets we care about, as they have the source code for all of their applications. In the case of not wanting to distribute your source code, I am interested in being able to distribute emcrypted LLVM IR as a pseudo-binary, similar to how silicon IP companies deliver encrypted verilog and vhdl for integration into other chips.
Interesting. The traditional problem in this space is that the source code is lost or otherwise not available. It is not clear that some encrypted source analog to encrypted verilog would be more available.
First, your silicon will be smaller - no need for pipelines, even double precision floating point multiply can be done in one clock cycle. Less registers, less silicon area. You can internally do that operation at higher frequencies than main core, timesharing die area for SIMD operations - saving silicon in other way.
Second, your compiler will be simpler - no need for NP-hard algorithms to schedule operations in the basic block, no need for fancy software pipelining for loop code to be warmed up and/or cooled down.
Third, your CPU won't be faster than memory. Your code won't get big hit from missed cache access. And cache will be used to save energy, not the time.
That's exactly the reasons behind one of exascale CPU architectures. It was long ago - circa 2012. I remember they managed to get FLOP/W ratios much higher than Intel's offering at the time.
With ever wider SIMD units this is actually happening to some extent. With AVX512 (e.g. on KNL) you can do 64 operations (single precision FLOPs) in vector units in the same amount of time as you can do 1 scalar operation
I was talking about code which simply can't be parallelised because it is branchy or has long and unavoidable dependency chains. In those cases using SIMD instructions will only make the code larger (thus more cache-consuming) and arguably more power-consuming too, since 63/64 of those operations aren't contributing anything to the computation.
Yes, I understand. What I meant was that you don't want to be running those kinds of codes on a KNL, because the performance is absolutely terrible. Nevertheless, Intel is producing the KNL, so there's now a market for this kind of thing, which may not have been there 15 years ago.
Actually it is 16 floating point operations in one AVX512 instruction and each core has two units, making 32 flops if you are running two or more threads (each core can do 4 way SMT but can also only decode two instructions per cycle).
That's Intel marketing and is somewhere between deceptive and wrong. Intel uses that to claim inflated maximum flops, with the huge aside that you would have to be doing nothing but fused multiply add, which is absurd.
AFAICS everybody has been doing that long before Intel, so in their defense they're just following precedent.
But yes, outside of carefully tuned kernels (e.g. GEMM and FFT), you're not going to see a 2x speedup. But that's no different from many other ISA extensions.
ILP specifically which, as it turned out was largely giving way to TLP in any case. I started writing a lesson from Itanium piece a while back. Maybe I should dust it off. (I followed Itanium pretty closely for a long time while I was an analyst.)
Your sequential CPUs are also exploiting parallelism to go faster, OoO execution, VLIW just wanted to do that in the compiler. There was nothing inherently wrong with the idea, the folly of Itanium was not the focus on parallelism, it was that compiler technology was not sufficiently advanced.
The overhead of padding 2/3 possible instructions with NOPs doesn't happen with "sequential CPUs". They extract parallelism only when and if it occurs.
Even the most "sufficiently advanced" compiler can't parallelise everything.
I think there's a strong argument that it wasn't just that the compiler technology was insufficiently god-like but also that they over-estimated the degree to which instruction-level parallelism mattered to the general market. The quote below really fits my memory of the initial experience — the claims were really ambitious but even if it'd on-time (i.e. 2+ years earlier) and at the originally promised clock rates the results wouldn't have been especially impressive outside of a few fields like scientific computing. Even in HPC it was a question of how much time you were willing to spend hand-tuning every hot section. When it finally did ship, we got access to a dev system and found the Itanium systems generally struggled to surpass the previous generation systems which we were preparing to replace.
“Davidson also pointed out two areas where academic research could create a blind spot for architecture developers. First, most contemporary academic research ignored CISC architectures, in part due to the appeal of RISC as an architecture that could be taught in a semester-long course. Since graduate students feed the research pipeline, their initial areas of learning frequently define the future research agenda, which remained focused on RISC. Second, VLIW research tended to be driven by instruction traces generated from scientific or numerical applications. These traces are different in two key ways from the average system-wide non-scientific trace: the numerical traces often have more consistent sequential memory access patterns, and the numerical traces often reflect a greater degree of instruction-level parallelism (ILP). Assuming these traces were typical could lead architecture designers to optimize for cases found more rarely in commercial computing workloads. Fred Weber echoed this latter point in a phone interview. Bhandarkar also speculated that the decision to pursue VLIW was driven by the prejudices of a few researchers, rather than by sound technical analysis.”
The last part I actually agree with, at the time Itanium was made, there was a sense of "It's the next big thing (tm)".
And I don't actually disagree that VLIW is best suited for some specific workloads as opposed to general workloads. All I'm saying is that those workloads, such as decoding, video and a few others, are becoming more important as the consumer smartphone workload and that most smartphone CPUs can't do much more than 3 instructions in parallel anyways, which is close to the generally agreed 2-3 ilp that VLIW compiler can extract. Power consumption is heavily restricted on a smartphone as well. That being said, data movement is the big issue nowadays, so I'm not sure how much that will help.
A smartphone like the iPhone 7 doesn't use its A10 Hurricane Arm to decode audio or video. There's a 6-core GPU for that. This then skews the Arm's workload to be even more general purpose.
I agree that those workloads are becoming more important but as CalChris noted there's a narrow range where that's both a significant workload and not offloaded entirely to purpose-built silicon.
I would be interested in learning more about the reasoning which caused AMD to move away from VLIW in their GPU designs since that seems like a relatively favorable area: very performance sensitive, people put time into optimization and tooling, and they don't need to run arbitrary legacy business apps.
No, for GPUs it would be stupid to go VLIW only, SIMD is a much better approach and VLIW-SIMD is even better. The last one is actually the approach taken by some of the newer GPU-esque architectures like movidius (although I take serious issue to almost everything else about their architecture).
By the ways, the NOP padding argument never convinced me because you can just use variable length instructions, and even current x86 decoders are variable length.
After all the technical details, I'm more and more convinced that Intel very badly wanted to shake off AMD, so they grasped at a too-thin excuse to do so; their eye was really on how much more coin they could shake out of everyone's pockets once they'd established a true monopoly just at the historical moment when monopoly law was being most thoroughly ignored.
"The Itanium is a noteworthy example of what happens when one designs an architecture exclusively for parallelism to the exclusion of all else, and leaves all instruction scheduling to the compiler. "
Well no, it would be more accurate to say "a noteworthy example of when you release a chip that isn't in line with what common programming languages can do".
That is, there is nothing wrong with doing the above. There's something wrong with doing the above when you most people are writing sequential C++ :)
There simply wasn't enough ILP to extract :)
IE no amount of compiler advance would have saved it.
But conventional Intel processors do manage to dynamically extract quite a good deal of ILP don't they? That's been the source of most of our performance increases over the last decade I think. Why does it work dynamically but not statically?
They can extract some ILP, but most of that is because it has no limitations but the semantics of the instruction set. It also cheats and can speculate just about everything legally, because it can always undo it.
A large amount of the ILP is, AFAIK, from being able to prefetch accurately (IE guess what memory will be accessed) and guess branch directions correctly.
This is not possible to do as well statically.
You actually can do it well dynamically (IE i could make a branch guessing thread and a prefetching thread), but you can't do it fast enough.
In a lot of workloads, most time is spent stalling for memory.
Also note that in VLIW, you have to extract ILP or you are just wasting time (IE are executing NOPs). You end up with a half or 3/4 empty pipeline. So unless you can predict you are going to stall for memory well, you are wasting the pipeline.
On the other hand, modern OOO cores can just extract more ILP when possible. IE when stalling for memory, they can figure out what they could be doing otherwise.
Maybe it's like JITs vs static compilers - we're better off with JIT even if your language is static, because you can adapt to real runtime conditions.
But we aren't better off with JIT in software. x64 processors work because they are doing things dynamically in separate hardware. It is part of how they use ever increasing transistor budgets to speed up serial processing.
Really, at this point the transistor budget is mostly going to bigger caches and more cores.
It's really little short of a miracle that Intel still managed to extract double digit percent improvements in serial code for all of Sandy Bridge, Haswell and Skylake.
You cannot fill that out-of-order window if you mispredict branches. So the branch prediction has gotten better and better, to an absurd level of accuracy.
You also need more execution units, to accommodate more kinds of serial code without stalling the pipeline. In Skylake they also improved the retirement unit, apparently to improve hyperthreading performance.
There are also better prefetchers and decoders, while the decoders also become more complicated because this is good old x86's variable-length encoding, with further complications for x86-64, AVX, and so on.
TLBs also get more complex (there have been two-level TLBs for a while now), though I guess that is part of the "bigger caches" part.
I agree, I was being a little blunt, but it's not necessarily as complicated as people make it out to be. The best branch predictor is once again the perceptron, and it beats current Intel chips on benchmarks. (I would hazard a, guess that Intel uses something like the tage predictor)
> JITs can't do is to add and/or remove parallelism from the program
Of course they can. A trivial practical example is the JVM JIT using vector instructions to add parallelism to a program that didn't have any before. But a JIT compiler could also profile code and add thread-level parallelism if it wanted.
For thread-level parallelism to be added you need to perform something akin to Fourier-Motzkin resolution algorithm or Omega test to find out what dependencies program has and has not.
FM is NP-hard. Omega test is imprecise.
And in general you need more complicated checks like aliasing checks. Which are equal to stopping problem, most of the time.
Usually you can't spend that amount of time on EACH INVOCATION of a program.
The techniques you are talking about are for proving that there is parallelism. The beauty of a JIT is that you can speculate that there is parallelism, go ahead, and then sort out the mess later and recompile to sequential if there wasn't.
> You can't speculate about parallelism when dealing with shared resource
Why do you think that? What do you think the optimistic software transactional memory algorithms are doing if not speculating on parallel access to shared memory?
You've just said that you 'can't speculate about parallelism when dealing with shared resource', and then gave me an example of a system that did exactly that. Clearly, it is possible to do speculative parallelism of shared resources. We may not yet know how to make it efficient, but it is possible.
From what I gather, STM in .Net heavily relies on the static program analysis.
You may see it as a proof of your point, but I see it as a counterexample - a team of smart people was not able make speculative parallelism efficient even with static analysis. JIT will be even less efficient as it has to keep and account for tracing information and for speculation logs. These two interact in interesting way, I guess.
I always wondered how CPUs could do all of this so fast. Doesn't all the branch prediction etc. have to be done by a kind of program that is "parsing" instructions on the fly and storing state about its execution? How is that implemented, and how is it faster than doing the equivalent in software?
Branch prediction under latency constraints is a difficult task! I believe perceptrons are the SOTA right now (have recently beat out TAGE predictors to take the crown once again) but are very energy intensive, the alternative is (which is currently used I think) are two level look up tables.
One reason is that the idea behind 'just need a smart compiler' was doomed from the start. If you build something that won't shine unless everyone that builds a compiler for it makes sure to extract plenty of instruction level parallelism, it is doomed to fail. Maybe these days with LLVM it could have been a little more practical, but Intel has dominated largely by making sure poorly written programs run as fast as possible. To put all that aside and hope that compilers and programmers become much more sophisticated was never going to happen, especially because so much software is already set and won't change.
LLVM is the godsend that any new architecture (especially a VLIW) to compete with Intel needed. I'm the founder of a startup that has made a new VLIW processor with our toolchain built on top of LLVM... We get the front end and all of the languages it supports virtually for free (we've only had to make small tweaks to clang), and get to focus our effort on just the parts important to us, our actual target backend and the unique optimization steps. Right now we can take any code that compiles through LLVM and run it on a single one of our cores while finding a good chunk of ILP (we are quad issue, averaging close to 2.5 IPC, sometimes higher), and be beating out the performance of some ARM and DSP cores that are 10x the size. Multi core parallelism requires including our hand tuned libraries, but our true MIMD style of task and data level parallelism is a lot more flexible than SIMD/SIMT/SMP that you are stuck with on existing chips.
Is it the case that you can implement just a new backend and produce good machine code for your processor, or really do you need to modify the frontend to get it to generate code IR from which you can generate good machine code, because otherwise useful information has been lost by time you get the IR from the frontend?
We have played around with having "hints" in the original source code that will be preserved through the IR, but the vast majority of the optimizations and scheduling decisions we need to do are happening once it has been converted into machine code by the backend. I may be slightly off (I'm the dumb CEO/hardware architect), but I'm fairly sure 85%-90% of our compiler work has been done on the backend... I just remember being told some tweaks were made to clang due to some weird issue.
The third is the backend code generator. It lowers IR to the target ISA. There's a tablegen description, there's are MI scheduler models which abstract architectural information as well.
I think you really meant can you write an optimizing pass specifically for your target? Yes. But I don't think that modifying the real front end Clang is going to get you much of anything. Intrinsics are different; that's the programmer inserting ISA specific instructions rather than Clang extracting ISA specific information. Really, modifying Clang isn't going to get you anything.
Thomas, what were the 'small' Clang specific tweaks?
I'm not sure why GCC would have been anything different.
No, porting GCC to a new architecture is not a big deal. The really complex part is the ABI, and what I have heard from former GCC developers is that it's as much of a mess in LLVM, because it's simply a messy domain.
LLVM is much more compartimentalized, so we can make changes to the backend without touching/breaking other parts. The GCC loader was miles ahead of the LLVM loader up until the past few releases, and we were originally using a modified version of the GCC loader before switching.
LLVM also has a bunch of awesome projects like Poly for polyhedral compilation that we are able to use that don't exist at all for GCC, or are very difficult to integrate. LLVMs lego block like architecture (it was originally designed as a tool to help build toolchains) plus things like Tablegen enabled our single "real" software engineer/CTO to do initial bring up of a backend in under a month.
As a developer of out-of-order CPU (model, but studied the effects on real code and did my best to make it synthesable) I can assure you there's a lot of ILP to extract even from ordinary programs.
The problem with Itanium is that it wasn't OoO and can't be made such.
Just to give you a hint - just computing various kinds of addresses in advance can make huge difference. Compute branch address, issue a fetch, compute load address, issue a fetch (even when destination register is blocked). Voila! Your program is sped up significantly.
Itanium did have features for various kinds of speculation, though. You could load from an address even if it were potentially aliased with a subsequent store, with the processor doing a dynamic check to make sure the right value was observed (eg, performing a late load and blocking if it turned out there was an overlapping store).
It wasn't as effective as OoO in the end, but not for lack of attention to the problem.
In-order processors also leave instruction scheduling to the compiler. They're very common. The workhorse ARM Cortex-A7 is in-order, dual issue and pipelined.
We would probably all be working with Itanium servers if not for AMD introducing 64 bit x86. I believe that also accelerated Linux adoption and the decline of all the commercial Unix platforms.
Good to see AMD on the rise again. I appreciate their role in heading off Itanium.
Because Power and Sparc would have ceded to Itanium. Neither backer had the resources Intel did. The easy path to kill Itanium was x86-64, because it was inexpensive and worked with Linux and Windows. AMD didn't have to compete with Intel per se. It forced Intel to do X86-64 as well...which killed Itanium.
Had x86_64 not been released, I also think we'd see a more diverse set of serverside chips as well, such as SPARC and POWER. There wouldn't have been an obvious dominance in the marketplace of a single architecture.
Also, had it not been released, we'd probably have seen a larger fracturing of the laptop CPU market with a broader switch to ARM64.
In a sense, because AMD forced Intel's hand in supporting x86_64, they actually helped Intel have even more dominance in the market with an architecture that spans laptops to server work horses. We developers barely need to lift a finger in targeting either.
I dunno. The cost advantage of x86-64 plus Linux and Windows was a tough nut for other RISC vendors to beat. Price/Performance, plus general support. For example, x86-64 servers could get away without SSL acceleration because the handshake was done in optimized ASM for most web servers. For other architectures, it wasn't.
If amd64 had not been released, I think itanium would probably own the server market. But it would be a very different server market .. much more like the late 90s market than today's market.
Remember that Intel was twisting arms and making deals to kill the weaker RISC chips in the marketplace. In the leadup to the first Itanium, HP had acquired Compaq (which had acquired DEC), so they held both Alpha and PA-RISC. And SGI was still a fairly big player with mips64. Both HP and SGI made deals to use Itanium going forward (with HP going so far as to port HP-UX and VMS to Itanium).
I think the server marketplace would probably have been much higher margin without amd64 competition, as itanium was always a high margin player. Its main competitors were the high margin server chips above, plus POWER and SPARC. The low end was x86 controlled by intel, and they could certainly continue to artificially segregate the market and keep margins high. That's what they were good any monopoly does, and what they've been trying to do today with Xeons.
I was a huge fan of Alpha (one of 2 people who ported FreeBSD to the alpha), and I was very sad to see it go. I've always blamed Itanium for its downfall -- I think Alpha had a lot more headroom than PA-RISC, and HP would have kept it going if it were not for Itanium. So I have to admit that I'm quite happy to see Itanium die.
It's also interesting to think about what would have happened if Rick Belluzzo hadn't been in the right place to convince both HP (as a VP) and then SGI (as CEO) to make large bets on Itanium and Windows. I don't think the market would have supported that many more expensive CPUs – fab economics are brutal – but two of the major players giving control over key parts of their design to Intel & Microsoft significantly accelerated that trend.
SGI is interesting because they _owned_ the 3D space and shortly before he went to Microsoft, SGI gave nVidia and Microsoft very cheap licenses to their extensive collection of patents and other IP. It's interesting to imagine what might have happened if, say, SGI had switched to Intel but with fewer gambles on unproven technology: x86 and Linux or the internal IRIX x86 port instead of Itanium and the heavily modified Windows NT derivative which was very expensive to support and forced all of their developers to make massive code changes. Similarly, they were focused on the high-end 3-D market (engineering, science, animation, etc.) and dismissed the PC gaming business as insignificant and basically gave it to nVidia along with a bunch of their staff; had someone a little short-sighted been involved that would have made a big difference in their bottom line.
If I had to bet on the reason of Itanium failure, I would also choose AMD64. But not because it was cheap, but because it had x86 in the name. For some reason whenever Intel tried to abandon x86, they failed, like Itanium, i960, XScale.
Perhaps. At least where I was, though, it was the cost factor. We liked Solaris better than Linux. But big Solaris Sparc servers (and every other RISC server too) were 6 figure purchases, with accompanying big support contracts. A high end 4 way x86-64 box was much lower initial cost, and much lower recurring cost. They did start lowering prices when they saw the shift, but too little, too late.
Because you've already sunk an enormous amount of money into the architecture, and you're looking to defer the day when you have to admit that was a bad idea for as long as humanly possible.
1. To keep your stuff running that's stuck on that chip. Replacements are being made but not finished for many customers. The article actually covers this for HP's customers. OpenVMS was on Itanium as well with a x86 port underway. SGI was on Itanium but moved to Xeon more quickly.
2. To get benefits that chip has that others don't. Secure64 is the main use-case I know for this. For me, I considered buying some SGI Altix's off ebay that were going for $100-200 each. Reliable chip and server with built-in security features that almost nobody is targeting on top of that. Such an approach has always paid off for me in the past.
>"Intel can now focus on Xeon, which was rebranded last week to account for new technologies like co-processors and faster interconnects."
I was confused by this, Xeon has been the mainstay in the high-end workstation/server for what feels like forever now. Was the limited Itanium market and development really affecting their focus that much?
I am curious if anyone knows what companies or verticals made big investments in Itanium? I imagining Intel much have some pretty big Itanium customers if it has been around this long. Maybe this answers my first question?
Xeon wasn't losing focus to Itanium... that's just the author writing filler material.
The main customer of Itanium that I'm aware of is the VMS operating system. It was ported from Alpha to Itanium in the early days of Itanium and anyone dependent on VMS has been reliant on Itanium since then. HP paid Intel a few years ago not to kill Itanium in the meantime while they were porting VMS to x86.
Oh interesting, I did not know that. I have heard that the US government has a lot of VMS systems. If that's true then this alone would make sense for HP to fund Itanium to some extent.
Features quotes from various interviews with folks like David Cutler. Fascinating (and not surprising) to connect the dots between the NT x64 calling conventions + SEH + RIP-relative addressing and Dave Cutler's initial input.
Regarding the Itanium itself, I've always found this excerpt quite interesting:
"Davidson also pointed out two areas where academic research could create a blind spot for
architecture developers. First, most contemporary academic research ignored CISC
architectures, in part due to the appeal of RISC as an architecture that could be taught in a
semester-long course. Since graduate students feed the research pipeline, their initial areas of
learning frequently define the future research agenda, which remained focused on RISC.
Second, VLIW research tended to be driven by instruction traces generated from scientific or
numerical applications. These traces are different in two key ways from the average systemwide
non-scientific trace: the numerical traces often have more consistent sequential memory
access patterns, and the numerical traces often reflect a greater degree of instruction-level
parallelism (ILP). Assuming these traces were typical could lead architecture designers to
optimize for cases found more rarely in commercial computing workloads. Fred Weber echoed
this latter point in a phone interview. Bhandarkar also speculated that the decision to pursue
VLIW was driven by the prejudices of a few researchers, rather than by sound technical
analysis."
> Assuming these traces were typical could lead architecture designers to optimize for cases found more rarely in commercial computing workloads.
At the time Itanium was actively being pushed by Intel I was working for a company building content creation software. Intel was very interested in us porting to Itanium and provided considerable help to that effort. One of the big reasons was that they were interested in being able to see the traces, precisely for the reason you state - most of their existing test code was not representative of typical large scale commercial software.
We got it running but never released it - the binaries were huge (3x x86) which was a big issue back then, and the performance just wasn't there.
As an interesting data point, (and the article doesn't do it justice), when I saw the presentation on AMD's "Sledgehammer" architecture (the AMD64) at Microprocessor Forum I wrote to the CTO of NetApp at the time "If you're wondering, Itanium just died." And then more than 16 years later it actually is going to cease development.
It also started a multi-year effort to convince NetApp to use an AMD chip in their filer :-).
I think the events reinforce three good things to know;
1) Adapting the existing system to do new things can trump entirely new systems, even if those new systems are 'better' in some way.
2) You can change hardware "overnight for free" compared to how difficult it is to migrate large software systems.
3) "Eat your young" - Keep innovating in your own products, even if that means it makes previous versions obsolete, because if you don't your competitors will.
I agree with your assessment. It seemed Intel tried to use the 32-bit limit of x86 to drive the high end computing to Itanium. AMD did x86-64 and removed that limit, and Intel was forced to play along and x86 escaped from its artificial limits and removed much of the justification for Itanium.
97 comments
[ 3.6 ms ] story [ 105 ms ] threadI suppose a similar analogy would be doing everything in x86 with SIMD instructions and not using the scalar set at all.
With ever wider SIMD units this is actually happening to some extent. With AVX512 (e.g. on KNL) you can do 64 operations (single precision FLOPs) in vector units in the same amount of time as you can do 1 scalar operation. Combined with the low clock speed of the KNL, you really don't want to be doing scalar operations if you care at all about performance. Of course SIMD isn't the same as VLIW, since the operation you're doing has to be the same in all vector lanes, but with masking support in AVX512, it is getting a little closer to that.
Also compilers have gotten quite a bit smarter and with hardware die shrinks taking longer, I wouldn't be at all surprised if VLIW ISAs were to make a comeback. I guess the Itanium was a bit ahead of its time.
What recent advances have made VLIW any more tractable than trace scheduling already did? BTW, trace scheduling works for scheduling superscalar processors as well.
VLIW never went away. It's used in embedded, the TriMedia processors. It's used in the REX Computing NEO chip. The Mill CPU is a VLIW of sorts.
I'm not anti-VLIW but I don't know of any recent breakthroughs that make it any more tractable now for non-embedded, non-HPC general purpose computing.
[1] Trace Scheduling: A Technique for Global Microcode Compaction
https://pdfs.semanticscholar.org/5698/09af0fcbe5a42371cea8d3...
[2] Bulldog: a compiler for VLIW architectures
http://dl.acm.org/citation.cfm?id=912347
VLIW has gotten an extremely bad rap outside of the embedded space due to Itanium, which I strongly contend was not a VLIW in spirit (which you cans see me talk in more depth on here: https://youtu.be/ki6jVXZM2XU?t=441). Itanium introduced a ton of indeterminism by having static scheduling unfriendly things like branch prediction, variable latency caches, and still trying to have some level of support for x86. The core problem for Itanium is that it was impossible for the compiler to make good decisions in the vast majority of applications since the compiler could not know exactly where and when things in memory would be. I believe the underlying failure point has been computer architects for failed VLIW architectures not realizing that fancy bells and whistles and "cleverly complex" hardware designs is the exact opposite of what you want when the compiler needs to be able to do purely static allocation. Adding fancy dynamic pieces in hardware to account from this (seen in a lot of places with Itanium) just makes the problem worse.
The main improvement we have made with the Neo architecture is we have hard (exact cycle count) guarantees on all memory movements on and off the chip, which actually gives the compiler the information it needs to make good decisions and very condensed code. I think the jury is out on the applicability for "general purpose" code on VLIWs... it is not something we at REX really care about at the moment, and I think RISC-V is a great solution with improvements over x86 and ARM.
These are the same reasons why DSPs have been VLIW since the 90s, and our initial applications overlap a lot. It has been sad to see the trend of TI and other DSP makers going down the path of adding hardware caching (that reduces determinism and energy efficiency) to try to "make it easier for programmers" when they already have a decent enough architecture, but a lack of good compiler tools to take advantage of it.
I quite like RISC-V and think it's elegant and simple, but maybe I'm wrong :)
As was pointed out in the A History of Modern 64-bit Computing cite, both RISC and Itanium show an academic bias. Yes, RISC-V is elegant and simple and that helps in a one semester upper div architecture class. But that doesn't help you much outside of a one semester upper div architecture class.
So what do you tangibly get with that elegance and simplicity? You aren't going to write RISC-V assembly; compilers are better especially for such a regular and spare ISA. This is 2017 and LLVM has been a thing for quite some time. Compilers are not a big deal.
Lower power? Chip area? How much lower power and chip area are we talking about?
That being said, greater efficiency is the main point, it's not too difficult to get a RISC-V core that beats (albeit marginally) an equivalent ARM core.
Optimizing compilers can often be difficult though, especially if you want to use novel modifications to the backend to make things more efficient.
Read up on horizontal microprogramming. A RISC ISA is an encoding with an opcode. A μop is really just a long set of fields with some bits clocking some FUs on and others off. There is no decoding; there is no opcode.
A complete macroinstruction is executed by generating an appropriately timed sequence of groups of control signals (micro-operations).
http://www.frsn.utn.edu.ar/tecnicas3/manuales/Microprogrammi...
Originally, RISC cpus were hardwired but they are general purpose ISAs. RISC-V is fixed 32b ISA but has multiple formats controlled by an opcode.
https://image.slidesharecdn.com/20160130deckyfosdemportinghe...
Original microprogrammed CPUs had a writeable control store (WCS). Modern processors translate CISC instructions into μops and cache this translation. The difference is the μop cache (μstore) vs WCS. FWIW, Skylake also has a WCS for obscure instructions.
Microinstructions, μops, do not have an opcode. They are not decoded. Bits in a horizontal μop gate logic on/off directly.
The RISC-V people do say RISC-like.
https://www2.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-...
And even that's wrong. But then they never even define what a RISC is. So RISC-like could be anything.
As for upstreaming, it is something that we want to do, but are not sure if it will be accepted as our chip is not generally available at the moment. It would probably be done around the same time as our next tape out (which would be in larger quantity than our first one) when we would have generally available development kits this coming winter.
I have been working on public documentation in my spare time... originally supposed to go out in March but we have had other priorities. If you watch my Stanford talk I show the basic ISA overview as well as the WIP documentation made using doxygen. I'm hoping to get stuff out on our website before our next media push in ~June.
This is a fascinating idea, but I have a question.
What happens when the next process generation lets you improve cache timings? What about newer, faster DRAM (or whatever) timings?
Do I have to recompile the world in that case, to see any improvement in performance?
First, your silicon will be smaller - no need for pipelines, even double precision floating point multiply can be done in one clock cycle. Less registers, less silicon area. You can internally do that operation at higher frequencies than main core, timesharing die area for SIMD operations - saving silicon in other way.
Second, your compiler will be simpler - no need for NP-hard algorithms to schedule operations in the basic block, no need for fancy software pipelining for loop code to be warmed up and/or cooled down.
Third, your CPU won't be faster than memory. Your code won't get big hit from missed cache access. And cache will be used to save energy, not the time.
That's exactly the reasons behind one of exascale CPU architectures. It was long ago - circa 2012. I remember they managed to get FLOP/W ratios much higher than Intel's offering at the time.
I was talking about code which simply can't be parallelised because it is branchy or has long and unavoidable dependency chains. In those cases using SIMD instructions will only make the code larger (thus more cache-consuming) and arguably more power-consuming too, since 63/64 of those operations aren't contributing anything to the computation.
But yes, outside of carefully tuned kernels (e.g. GEMM and FFT), you're not going to see a 2x speedup. But that's no different from many other ISA extensions.
IE if everyone is writing sequential C++, making a chip that is really good for parallel fortran is .... silly.
No amount of compiler advance would have saved itanium. Research early on shows except for very contrived kernels, there wasn't enough ILP to extract.
It actually did a really good job of doing what it could :)
Even the most "sufficiently advanced" compiler can't parallelise everything.
“Davidson also pointed out two areas where academic research could create a blind spot for architecture developers. First, most contemporary academic research ignored CISC architectures, in part due to the appeal of RISC as an architecture that could be taught in a semester-long course. Since graduate students feed the research pipeline, their initial areas of learning frequently define the future research agenda, which remained focused on RISC. Second, VLIW research tended to be driven by instruction traces generated from scientific or numerical applications. These traces are different in two key ways from the average system-wide non-scientific trace: the numerical traces often have more consistent sequential memory access patterns, and the numerical traces often reflect a greater degree of instruction-level parallelism (ILP). Assuming these traces were typical could lead architecture designers to optimize for cases found more rarely in commercial computing workloads. Fred Weber echoed this latter point in a phone interview. Bhandarkar also speculated that the decision to pursue VLIW was driven by the prejudices of a few researchers, rather than by sound technical analysis.”
The rest of this PDF is well worth reading:
http://courses.cs.washington.edu/courses/csep590/06au/projec...
And I don't actually disagree that VLIW is best suited for some specific workloads as opposed to general workloads. All I'm saying is that those workloads, such as decoding, video and a few others, are becoming more important as the consumer smartphone workload and that most smartphone CPUs can't do much more than 3 instructions in parallel anyways, which is close to the generally agreed 2-3 ilp that VLIW compiler can extract. Power consumption is heavily restricted on a smartphone as well. That being said, data movement is the big issue nowadays, so I'm not sure how much that will help.
I would be interested in learning more about the reasoning which caused AMD to move away from VLIW in their GPU designs since that seems like a relatively favorable area: very performance sensitive, people put time into optimization and tooling, and they don't need to run arbitrary legacy business apps.
By the ways, the NOP padding argument never convinced me because you can just use variable length instructions, and even current x86 decoders are variable length.
Also, many VLIW purists will claim (with some veracity also) that Itanium's EPIC != VLIW
Well no, it would be more accurate to say "a noteworthy example of when you release a chip that isn't in line with what common programming languages can do".
That is, there is nothing wrong with doing the above. There's something wrong with doing the above when you most people are writing sequential C++ :)
There simply wasn't enough ILP to extract :)
IE no amount of compiler advance would have saved it.
A large amount of the ILP is, AFAIK, from being able to prefetch accurately (IE guess what memory will be accessed) and guess branch directions correctly.
This is not possible to do as well statically. You actually can do it well dynamically (IE i could make a branch guessing thread and a prefetching thread), but you can't do it fast enough.
In a lot of workloads, most time is spent stalling for memory.
Also note that in VLIW, you have to extract ILP or you are just wasting time (IE are executing NOPs). You end up with a half or 3/4 empty pipeline. So unless you can predict you are going to stall for memory well, you are wasting the pipeline. On the other hand, modern OOO cores can just extract more ILP when possible. IE when stalling for memory, they can figure out what they could be doing otherwise.
It's really little short of a miracle that Intel still managed to extract double digit percent improvements in serial code for all of Sandy Bridge, Haswell and Skylake.
You cannot fill that out-of-order window if you mispredict branches. So the branch prediction has gotten better and better, to an absurd level of accuracy.
You also need more execution units, to accommodate more kinds of serial code without stalling the pipeline. In Skylake they also improved the retirement unit, apparently to improve hyperthreading performance.
There are also better prefetchers and decoders, while the decoders also become more complicated because this is good old x86's variable-length encoding, with further complications for x86-64, AVX, and so on.
TLBs also get more complex (there have been two-level TLBs for a while now), though I guess that is part of the "bigger caches" part.
Whole program compiler can insert JIT-calling code (specializing for common pattern), JIT compiler cannot view whole program.
One of the things JITs can't do is to add and/or remove parallelism from the program (i.e., add/remove OpenMP pragmas).
Of course they can. A trivial practical example is the JVM JIT using vector instructions to add parallelism to a program that didn't have any before. But a JIT compiler could also profile code and add thread-level parallelism if it wanted.
FM is NP-hard. Omega test is imprecise.
And in general you need more complicated checks like aliasing checks. Which are equal to stopping problem, most of the time.
Usually you can't spend that amount of time on EACH INVOCATION of a program.
https://en.wikipedia.org/wiki/Speculative_multithreading
You can use hardware functionality such as TSX to support this.
It is unbelievable you even suggested it.
TSX won't save you.
Why do you think that? What do you think the optimistic software transactional memory algorithms are doing if not speculating on parallel access to shared memory?
Because Microsoft tried to add STM to the .Net and get 4x slower code after extensive effort: http://www.mail-archive.com/haskell-cafe@haskell.org/msg7973...
Should I continue from these two arguments?
You may see it as a proof of your point, but I see it as a counterexample - a team of smart people was not able make speculative parallelism efficient even with static analysis. JIT will be even less efficient as it has to keep and account for tracing information and for speculation logs. These two interact in interesting way, I guess.
The first stage is the LLVM front end, Clang and other language parsers. They generate IR.
The middle stage is the LLVM optimizer, opt, which takes IR and runs a variety of optimization passes. Opt takes and gives IR.
http://llvm.org/docs/Passes.html
The third is the backend code generator. It lowers IR to the target ISA. There's a tablegen description, there's are MI scheduler models which abstract architectural information as well.
http://llvm.org/docs/WritingAnLLVMBackend.html
I think you really meant can you write an optimizing pass specifically for your target? Yes. But I don't think that modifying the real front end Clang is going to get you much of anything. Intrinsics are different; that's the programmer inserting ISA specific instructions rather than Clang extracting ISA specific information. Really, modifying Clang isn't going to get you anything.
Thomas, what were the 'small' Clang specific tweaks?
No, porting GCC to a new architecture is not a big deal. The really complex part is the ABI, and what I have heard from former GCC developers is that it's as much of a mess in LLVM, because it's simply a messy domain.
LLVM also has a bunch of awesome projects like Poly for polyhedral compilation that we are able to use that don't exist at all for GCC, or are very difficult to integrate. LLVMs lego block like architecture (it was originally designed as a tool to help build toolchains) plus things like Tablegen enabled our single "real" software engineer/CTO to do initial bring up of a backend in under a month.
The problem with Itanium is that it wasn't OoO and can't be made such.
Just to give you a hint - just computing various kinds of addresses in advance can make huge difference. Compute branch address, issue a fetch, compute load address, issue a fetch (even when destination register is blocked). Voila! Your program is sped up significantly.
It wasn't as effective as OoO in the end, but not for lack of attention to the problem.
In-order processors also leave instruction scheduling to the compiler. They're very common. The workhorse ARM Cortex-A7 is in-order, dual issue and pipelined.
Good to see AMD on the rise again. I appreciate their role in heading off Itanium.
Also, had it not been released, we'd probably have seen a larger fracturing of the laptop CPU market with a broader switch to ARM64.
In a sense, because AMD forced Intel's hand in supporting x86_64, they actually helped Intel have even more dominance in the market with an architecture that spans laptops to server work horses. We developers barely need to lift a finger in targeting either.
Remember that Intel was twisting arms and making deals to kill the weaker RISC chips in the marketplace. In the leadup to the first Itanium, HP had acquired Compaq (which had acquired DEC), so they held both Alpha and PA-RISC. And SGI was still a fairly big player with mips64. Both HP and SGI made deals to use Itanium going forward (with HP going so far as to port HP-UX and VMS to Itanium).
I think the server marketplace would probably have been much higher margin without amd64 competition, as itanium was always a high margin player. Its main competitors were the high margin server chips above, plus POWER and SPARC. The low end was x86 controlled by intel, and they could certainly continue to artificially segregate the market and keep margins high. That's what they were good any monopoly does, and what they've been trying to do today with Xeons.
I was a huge fan of Alpha (one of 2 people who ported FreeBSD to the alpha), and I was very sad to see it go. I've always blamed Itanium for its downfall -- I think Alpha had a lot more headroom than PA-RISC, and HP would have kept it going if it were not for Itanium. So I have to admit that I'm quite happy to see Itanium die.
SGI is interesting because they _owned_ the 3D space and shortly before he went to Microsoft, SGI gave nVidia and Microsoft very cheap licenses to their extensive collection of patents and other IP. It's interesting to imagine what might have happened if, say, SGI had switched to Intel but with fewer gambles on unproven technology: x86 and Linux or the internal IRIX x86 port instead of Itanium and the heavily modified Windows NT derivative which was very expensive to support and forced all of their developers to make massive code changes. Similarly, they were focused on the high-end 3-D market (engineering, science, animation, etc.) and dismissed the PC gaming business as insignificant and basically gave it to nVidia along with a bunch of their staff; had someone a little short-sighted been involved that would have made a big difference in their bottom line.
2. To get benefits that chip has that others don't. Secure64 is the main use-case I know for this. For me, I considered buying some SGI Altix's off ebay that were going for $100-200 each. Reliable chip and server with built-in security features that almost nobody is targeting on top of that. Such an approach has always paid off for me in the past.
https://secure64.com/secure-operating-system/
>"Intel can now focus on Xeon, which was rebranded last week to account for new technologies like co-processors and faster interconnects."
I was confused by this, Xeon has been the mainstay in the high-end workstation/server for what feels like forever now. Was the limited Itanium market and development really affecting their focus that much?
I am curious if anyone knows what companies or verticals made big investments in Itanium? I imagining Intel much have some pretty big Itanium customers if it has been around this long. Maybe this answers my first question?
The main customer of Itanium that I'm aware of is the VMS operating system. It was ported from Alpha to Itanium in the early days of Itanium and anyone dependent on VMS has been reliant on Itanium since then. HP paid Intel a few years ago not to kill Itanium in the meantime while they were porting VMS to x86.
https://github.com/tpn/pdfs/blob/master/A%20History%20of%20M...
Features quotes from various interviews with folks like David Cutler. Fascinating (and not surprising) to connect the dots between the NT x64 calling conventions + SEH + RIP-relative addressing and Dave Cutler's initial input.
Regarding the Itanium itself, I've always found this excerpt quite interesting:
"Davidson also pointed out two areas where academic research could create a blind spot for architecture developers. First, most contemporary academic research ignored CISC architectures, in part due to the appeal of RISC as an architecture that could be taught in a semester-long course. Since graduate students feed the research pipeline, their initial areas of learning frequently define the future research agenda, which remained focused on RISC. Second, VLIW research tended to be driven by instruction traces generated from scientific or numerical applications. These traces are different in two key ways from the average systemwide non-scientific trace: the numerical traces often have more consistent sequential memory access patterns, and the numerical traces often reflect a greater degree of instruction-level parallelism (ILP). Assuming these traces were typical could lead architecture designers to optimize for cases found more rarely in commercial computing workloads. Fred Weber echoed this latter point in a phone interview. Bhandarkar also speculated that the decision to pursue VLIW was driven by the prejudices of a few researchers, rather than by sound technical analysis."
(Page 6 of cited PDF.)
At the time Itanium was actively being pushed by Intel I was working for a company building content creation software. Intel was very interested in us porting to Itanium and provided considerable help to that effort. One of the big reasons was that they were interested in being able to see the traces, precisely for the reason you state - most of their existing test code was not representative of typical large scale commercial software.
We got it running but never released it - the binaries were huge (3x x86) which was a big issue back then, and the performance just wasn't there.
It also started a multi-year effort to convince NetApp to use an AMD chip in their filer :-).
I think the events reinforce three good things to know;
1) Adapting the existing system to do new things can trump entirely new systems, even if those new systems are 'better' in some way.
2) You can change hardware "overnight for free" compared to how difficult it is to migrate large software systems.
3) "Eat your young" - Keep innovating in your own products, even if that means it makes previous versions obsolete, because if you don't your competitors will.
So long Itanium.
[1] http://www.anandtech.com/show/11372/intels-itanium-takes-one...