That's fair. I mean if he wants to change that he has to be in the board. But I think investors like less risky ways to make revenue (i.e. by cutting costs)
From the article: Piednoel didn’t spare words for Intel’s culture, which he said has changed drastically and promotes MBAs over those with technical prowess.
The current CEO is a finance / MBA type, so it's no surprise that the current culture favors that.
In Intel's defense, the previous CEO was a process engineer. But, unfortunately, he couldn't keep his weiner in his pants. He also was probably the wrong choice as CEO since, under his watch, Intel process development apparently went to shit.
Oh, well. Intel was a great company for many years. Perhaps it will reinvent itself.
The previous CEO, who was a process engineer, was the one who brought the current CEO to Intel. It is widely believed the reasons for his firing were merely a pretext - they felt he was performing poorly and wanted to find an excuse to get rid of him. His focus definitely wasn't on CPUs - just look at all the acquisitions under his watch, and the focus away from CPUs into everything else (AI, autonomous vehicles, etc).
As somebody noted, when you have cornered a large part of the market, you don't feel as much technical pressure, and success in selling becomes key. So sales and MBAs start to run the show, because it makes the business sense, and while doing so risk to lose the sight of the technological advances that made the dominant position possible.
Intel's founder, Andy Grove, used to say: "Only the paranoid survive". (He personally survived under a false identity in Nazi-occupied Hungary.) I'm afraid Intel had stopped being paranoid enough before it started to stumble more and more seriously.
This seems to be a reasonable explanation. Even with sales and MBAs running the show, they know that they will always need something competitive to sell. For the longest time Intel relied on shrinking dies and single-threaded performance advantage. They didn't continue to race ahead with shrinking and as noted AVX-512 is not the hit to save them. They need more but may have misjudged that or believed too much of their own sales/marketing lacking sufficient paranoia.
The future will be more consumer appliance-like devices rather than PC-like products. I would much rather use a laptop/OS that suspended/resumed like an iPad with low power consumption (cool/quiet) and extended battery life. Apple has a way to this future. AMD can run on servers and gaming rigs. Intel needs something like a ChromeOS or Fuchsia to do everything (a cloud desktop rather than a browser) to be mainstream. I would use Clear Linux if it was more than an academic experiment--had a good desktop and repos that had a maintained useful fraction of Ubuntu packages (i.e. packages are up to date, used, have good defaults, etc).
As somebody noted, when you have cornered a large part of the market, you don't feel as much technical pressure, and success in selling becomes key. So sales and MBAs start to run the show, because it makes the business sense, and while doing so risk to lose the sight of the technological advances that made the dominant position possible.
What they are really doing there is merely converting the long-term value created by the engineers into short-term numbers that look good on the quarterly accounts and pay their own bonuses. MBAs and accountants add no value at all, they are simply profiteering from what others create. And once they have sucked an engineering company dry, they move on to the next.
> But, unfortunately, he couldn't keep his weiner in his pants.
My understanding is that he met his first wife while they both worked at Intel, but that was in a different era. Meeting your spouse at work used to be a common thing.
What you're referring to is more recent, but it's hard to fault the guy when Intel didn't say anything the previous time.
> Piednoel didn’t spare words for Intel’s culture, which he said has changed drastically and promotes MBAs
As organizations grow, nobody is left with a thread of understanding across departments of what is made.
That alone can explain what happened at both Intel and Boeing.
Unfortunately, anybody who does connect that thread, even before a looming disaster, will be labelled a naysayer, with no upside to them personally.
So Linus, and this guy is saying AVX512 was a bad idea.
"“The state of software out there is really not favoring going larger vectors,” Piednoel said in the video. “In fact, you can see clearly in Cinebench for example—that is not one of my favorite benchmarks, especially for a laptop where it doesn’t make any sense—but you can see that AMD is winning the battle of throughput. It’s because they have more cores and they can afford to have more cores.”"
But in other places I've seen a lot of hype for it but without much discussion on actual use cases.
With now people saying that it isn't useful except to give intel leadership in benchmarks.
Is anyone here actually using it and seeing a benefit?
AVX-512 is Intel's best vector ISA to date and it is definitely useful beyond benchmarks.
In fact, too low penetration of AVX-512 has been a problem, rather than too much: for a long time it has only been available on server chips, not laptop (a small fraction of laptops have gotten it recently) or destkop (outside some low-volume "extreme" parts, which were really just rebadged server parts).
It would also be quite unusual that some instructions could be very useful in benchmarks that are based on real-life, heavily used applications, but not in real life. Outside of small, easily gamed benchmark that doesn't seem plausible: if the CPU is good a video encoding benchmark, it will be good at video encoding, with high probability.
The main problem with AVX-512 is lack of software exploitation. Unlike with frequency boosts, increased cache sizes, better branch predictors, etc: this speedup doesn't come for free. Either compilers have to use the new instructions, or people have to use them by hand. The former has been very limited because these instructions cause a frequency drop (so-called "license based downclocking"), so compilers have mostly disabled their use by default: otherwise, a single AVX-512 instruction could cause a large impact on surrounding code which doesn't use AVX-512.
So by-hand exploitation remains, and penetration has just been too low and the people with the skills to do this are limited.
I'm sure AVX-512 is wonderful and you can easily produce examples where it makes loops go N times faster. That
unfortunately is missing the point.
Microprocessors implementation is an extremely careful balance and effectively everything is a trade off. The area, power, and complexity of features comes at a non-zero cost to code that doesn't use it.
The fact that AVX-512 isn't on every processor AND the performance benefits are FAR from obvious (as the core clocks down when using it) really hinders its adoption, which in turn lowers the value; a downward spiral.
Intel is in a fine mess of their own making. I'm no fan of either company and AMD gets perhaps a little too much credit as things would have looked a lot different had 10 nm succeeded on the original schedule, but I bought my first AMD processor since Athlon 64 because it's better.
I'm not missing the point as I was responding to whether AVX-512 is useful in practice. It is. It is some badly designed extensions where the earlier ones were great: it's arguably better than the earlier ones.
Now adoption has been poor and there is a chicken-and-egg effect as you point out, but that's separate from the question of whether AVX-512 is useful in practice. It is.
The point about tradeoffs is well taken. It's not like Intel (or any other vendor) is rolling back progress in other areas to add larger vector units: rather they try to progress along all viable axes at once.
> It's not like Intel (or any other vendor) is rolling back progress
Performance doesn't stand still so even if they are improving, they might have improved more having spend the AVX-512 on something else. And most definitely, larger vector unit does cost you somewhere else.
To me it looks like you and the GP just have different definitions of "in practice". Your definition of "in practice" seems to exclude the (general) availability of these extensions, whereas the GP's definition specifically includes them: if, as a developer, you can't rely on certain features being available, you're forced to optimize for the non-AVX512 case anyway, and any time spent exclusively on AVX512 is additional cost for a small audience.
So, whether the instructions themselves are any good then is of secondary concern because of the other problems of Intel's own making, and you both seem to be in agreement about those.
AVX-512 isn't "bad in practice" but in the context of AVX-128, AVX-256, and AVX-1024 it is a very bad practice because the inconstant availability of the vector unit means that it doesn't get used, or you use a least-common denominator of what worked on Atom 10 years ago.
Look on the bright side: if you don't use AVX-512 you have 10% extra heat sink and heat soak on your CPU so you can turbo higher. AVX-512 makes the chip easier to cool.
AVX-512 really has two parts. It widens vectors to 512 bits, but it also makes the instruction set much more orthogonal and clean. It also adds masking of lanes.
These latter parts are what's really useful about AVX-512. The widening of the vectors is just an area tradeoff that depends a lot on the particular workload.
Unfortunately there is a clock penalty to using AVX-512 and it is only available on a small part of the chips that Intel ships (For the longest time it was only available on Xeons). AMD does not support AVX-512 yet. As a result no one really had the incentive to add support for it outside HPC where people have control of both the workload and the hardware it runs on. To further complicate matters there are multiple variants of AVX-512 and different chip families ship with different subset of the variants.
In short Intel could not have done more to discourage use of AVX-512 if they tried.
Right, that's kind of the point. They should have made the cleaned up and extended instruction set available everywhere at least for 256-bit vectors. That would have made the good, non-trade-off parts that people actually liked pervasive.
The chip family variants problem can't be overstated, it's ridiculous. For example the laptop I bought last year with a flagship latest gen cpu has far more support than the flagship latest gen desktop cpu that I bought two months ago.
The bigger and more powerful AVX instruction the more power it needs - more power results in more heat, heat that modern cpus cannot handle (reason why we dont see higher cloked cpus).
I've used it a fair bit, but it's too exotic to show up on enough machines to really focus on. It's not just a trick to make benchmarks run faster; that's an idiotic conspiracy theory. It's just that the release of AVX-512 happens to coincide with the 10nm disaster.
Probably the biggest problem is that new Intel ISAs often are associated with downclocking (this isn't new with AVX-512 - AVX2 had it as well). Usually these problems were short-lived, but with 10nm getting stalled, they've stuck around longer. This is pure poison if like me, you generally write libraries. It's scary to look at some 60% speedup you might get from some neat rewrite for your bit of the task, but then wonder whether you're going to make the rest of the task several percent slower due to a downclock in what might only be 20% of the time. So it's hard to make the call to use AVX-512 in its early stages unless you control the whole core.
Linus thinks that everything is either a kernel or a gcc execution, and happily disregards anything that doesn't fit his own workloads. AVX-512 can do Good Things for a disparate range of stuff - in-memory databases, regex/string matching, bioinformatics, CPU intensive network workloads, parsing and lexing as well as the usual scientific computing and AI workloads (many of the latter are also pretty good on GPGPU).
He said that its a bad idea to put it in every xeon chip.
AVX is great tool for specific high throughput oncore calculations, when offloading to gpu would be too costly.
Saying AVX512 is universally bad is like saying that welding machines are bad tools since carpenters never make use of it.
This is purely a total outsider's opinion, so I caveat that and please don't get all offended if I'm totally wrong.
Sometimes it's really difficult to tell the difference between a crackpot theorist and a good equities analyst. Watching this guy's video, it's hard to tell which he comes off as more.
He's just coherent and convincing enough that as an outsider I could believe his collection of observations. But the presentation is also just disorganized enough, fixated on certain details enough, and a scattered collection enough that I can't tell whether these factors matter.
Of course I'm sure an expert in the field could tell instantly.
I've seen enough interviews of former engineers (usually retired guys) of whatever <xyz> company who know about a very specific fault of the product/tool/etc and can claim convincingly up and down that it will cause catastrophic failure, fundamental strategy misstep, etc. But will it really be a major factor for overall outcome?
He proceeds from analysis of the smallest die area detail to the opposite end of the spectrum, the problem with MBAs + Marketing, within 1 minute. With typos. Also, the guy doesn't have a long history of posting analysis of this type, and he's honest about his info being 8 years old. Not sure what his motivation for making the video is. So it's hard to put too much weight on it, as if were like a Ming-Chi Kuo predicting Apple's next iPhone form factor or something like that.
I would love to know what industry people here think, and then I might study the video / findings with more attention.
Francois is a smart guy, but there's a range of opinions on the matters he raises. We argue a good deal on Twitter ("constructive confrontation" between former Intel Principal Engineers :-) ). He put together the video pretty quickly so don't dismiss it just for lack of polish.
Personally, I argue with him a lot on AVX-512. I think AVX-512 is a Good Thing (or will be shortly - the first instantiation in Skylake Server - SKX - isn't great).
The biggest meta-problem with AVX-512 is that due to process issues, the pause button got hit just as it appeared in a server-only, 'first cut' form. AVX2 had downclocking issues when it first appeared too - but these were rapidly mitigated and forgotten.
I personally feel that SIMD is underutilized in general purpose workloads (partly due to Intel's fecklessness in promoting SIMD - including gratuitous fragmentation - e.g. a new Comet Lake machine with Pentium branding doesn't even suport AVX2 and has it fused off). Daniel Lemire and I built simdjson (https://github.com/lemire/simdjson) to show the power of SIMD in a non-traditional domain, and a lot of the benefits of Hyperscan (https://github.com/intel/hyperscan) were also due to SIMD. Notably, both use SIMD in a range of ways that aren't necessarily all that amenable to the whole idea of "just do it all on an accelerator/GPU" - choppy mixes of SIMD and GPR-based computing. Not everything looks like matrix multplication, or AI (but I repeat myself :-) ).
AVX-512 is a combination of fascinating and frustrating. I enthuse about recent developments on my blog (https://branchfree.org/2019/05/29/why-ice-lake-is-important-...) so I won't repeat why it's so interesting, but the learning curve is steep and there's plenty of frustrations and hidden traps. Intel collectively tends to be more interested in Magic Fixes (autovec) and Special Libraries for Important Benchmarks rather than building first-rate material to improve the general level of expertise with SIMD programming (one of my perpetual complaints).
You more or less have to teach yourself and feel your way through the process of getting good at SIMD. That being said, the benefits are often huge - not only because SIMD is fast, but because you can do fundamentally different things that you can't do quickly in GPR ("General Purpose Register") land (look, for example, at everyone's favorite instruction PSHUFB - you can't do the equivalent on a GPR without going to memory).
Oh, I should note that an interest in large SIMD is not confined to Intel - the ARM world is (slowly) heading towards SVE (or so my friends from that quarter assure me). So it's not just a partisan pro-Intel or pro-x86 thing on my part.
RISC-V is also adding Vector extensions.
The cool thing about those is that they are variable length, so code can take advantage of newer hardware / degrade gracefully on less capable one.
I think going for variable length is a Bold Move, and am not sure I like it for an ISA that otherwise is trying to be conservative. I'll be more optimistic about it when I see SVE (a) on more than one HPC machine and (b) implemented in a variety of widths. I'll be impressed if it works, but I'm not going to be impressed ahead of time with it while it's an aspirational design because it has aspirations.
The small, medium and large sizes built into arms neon accomplishes some scalability without going back to what vector units were like in the big hair 1980s.
> so code can take advantage of newer hardware / degrade gracefully on less capable one
I never user variable-length SIMD on CPUs, but I have substantial experience with current SIMD up to AVX2, and GPGPUs. I’m not sure I like that variable-length stuff.
1. Variable-length SIMD is only suitable for vertical operations. Many useful algorithms are only possible to vectorize with shuffles and such.
2. With registers size unknown to compilers, it’s impossible to create a calling convention to pass data in registers, as opposed to RAM. RAM is slow on modern hardware, and is becoming slower over time (in relation to compute speed; nanoseconds are fine). GPUs are OK with that because they are massively multithreaded, they switch threads instead of waiting for data from memory. CPUs can’t do that, they only have 2 hardware threads per core.
> The biggest meta-problem with AVX-512 is that due to process issues, the pause button got hit just as it appeared in a server-only, 'first cut' form. AVX2 had downclocking issues when it first appeared too - but these were rapidly mitigated and forgotten.
I've worked on a project to compile a domain-specific description language down to both SIMD (by using Intel SPMD Program Compiler) and GPGPU (by using CUDA and OpenCL) Monte Carlo simulation code. We also have a Scala/Java interpreter for debugging.
Are there workloads that benefit from very wide SIMD vectors that aren't good fits for GPGPUs, as long as the GPUs support 64-bit floats close enough to IEEE-754 for your needs? I understand the overhead in shuffling data between main memory and GPU memory, and synchronization overhead, but most code I'm familiar with that does heavy number crunching suitable for very wide SIMD tends to do that number crunching off on threads that don't have much synchronization with threads doing more general-purpose computation.
On a side note, ISPC's input language is deceptively close to C, but little traps lie in wait. I remember helping an intern debug his port of some of the Java code I wrote over to ISPC, but it turns out that mulitplying a long long by a double in ISPC results in a long long. Our attempt to scale down a random 64-bit integer to the range [0.0, 1.0) was always resulting in 0.0. I get that integer calculations are faster, and I could understand disallowing implicit casts, but making a language so close to C, but with different implicit casting rules is just asking for trouble.
You're in a bit of a different domain - I've never really done all that much stuff with heavy number crunching (I've done a bit of work on random forest traversal, but that's more about logic than about FP). And I have never worked with ISPC.
I think number crunching workloads are typically quite suitable for GPGPU - I'm certainly not trying to "debunk GPGPU", just saying that there are a lot of integer/logic intensive workloads that involve rapid switching back and forth between control/GPR-based-logic sides and "SIMD tasks" (e.g. Hyperscan switching between NFA/DFA simulation and "acceleration", which was SIMD-based character skipping).
Dan Luu does brilliant things parsing strings with SIMD instructions but the programming model is busted if you have to recompile (and maybe hand code assembly) for each chip.
I was involved with wrapping up a deep neural net application just as GPGPU was about to take off. We used hand coded SIMD, performance was fine, but we were not going to recode for the next generation from chips, change our error function, or change anything SIMD related.I
Before there was MMX computers came with vector units that did not have a programmer visible size, such as the Cray 1 and the vector unit for the 3090 mainframe. As it is, people hardly use vector instructions or if they use them it is something a few generations old so people pay for vector units that they don't use.
simd is a better fit than gpu for a lot of data warehousing workloads where you aren't doing much computation on each individual value and it's not worth the overhead of sending the data to the gpu. also true for search index performance optimization.
I'm sold, and not sold. Having parsed terabytes of JSON, I was impressed by your work. Regexps and other operations are common-place, and if libraries like yours were in broad use, I could definitely see the appeal.
I'm sold on this as a technical pathway to performance.
Problem is, I do it in Python. I have a hard time laying out the breadcrumbs between simdjson and hyperscan, and practical, general-purpose code running more quickly on Intel CPUs. My code will run faster on AMD CPUs in practice even if, in abstract, given a few extra hours of coding, it could run faster on AVX-512. I'd rather wait a few hours for my computer than port my code to simdjson. I think that's true for 99% of the code out there.
Didn't we go down this path with Pentium IV's deep out-of-order pipeline and Itanium's VLIW before? Where Intel had great theoretical performance, but software didn't keep up?
For this to make a dent, Intel would need to hire a teach of engineers so Firefox and Chrome both took advantage of the new instructions. That's perhaps more practical than in the Itanium and PIV era, but that doesn't mean it will happen.
> Where Intel had great theoretical performance, but software didn't keep up?
> For this to make a dent, Intel would need to hire a teach of engineers so Firefox and Chrome both took advantage of the new instructions.
That's more or less Intel's standard practice with its Intel compiler suite and MKL-and-similar libraries.
If there are performance gains from SIMD being left on the table, some of it is on Intel for not building the middle tooling necessary to make it easy.
> On some Intel processors, AVX-512 instructions are subject to downclocking: whenever AVX-512 instructions are executed, the processor lowers its frequency. We do not observe any downclocking on our processor, nor did we find any Intel documentation referencing downclocking for this processor. To test for downclocking, we used the avx-turbo benchmarking tool: it reports the processor frequency after issuing long sequences of expensive instructions of various types,including AVX-512 multiplication and floating-point instructions.
"...SIMD is underutilized in general purpose workloads... Daniel Lemire and I built simdjson to show the power of SIMD in a non-traditional domain..."
To your point, the recently front-paged simdjson writeup made me regret ignoring SIMD. I had dabbled in 3D graphics and matrix math stuff in the 90s and somehow continued to associate those extensions with that kind of work.
Since you sound quite expert -- I would love to learn from you (or others), at a beginner's level:
Is the design + production of a chip an exercise in optimizing the combination of:
-- size of die (and therefore overall cost/yield?)
-- power consumption
-- number / type / purpose of compute and memory subcomponents packed into the area
-- speed
Is that basically the envelope that you're doing an optimization within, and until a technology jump, even Intel/AMD/ARM-based are all simply choosing values between these dimensions? And each different choice of parameters within these basically goes after a certain segment of user?
I am not a hardware design expert, but... yes. Everyone makes their choice between speed, power consumption and cost. I attended a event once at Intel where someone told the anecdote that, "If someone comes to me and says, gee, I'd like a chip that runs faster, consumes less power and is cheaper to make I tell them 'you are overpaid for whatever it is that you do'".
The technology jump is of course the thing that allows you to "have it all" - until it doesn't. So it's clear that the optimization of Intel's 14nm++++++ is better for speed right now than anything they have at 10nm, which is better for density and power consumption. It used to be that the shrinks were unambiguously better but we've entered a strange new world.
Honestly, from the design perspective, I'm a enthusiastic spectator - to the extent I have insider knowledge it's out of date, poorly understood and not shareable in any case.
Microarchitecture is still very important - a better uarch and get you big effects without process changes. The last unambigiously successful one of these was Broadwell to Skylake. I think it will be fascinating to see the backport of Sunny Cove to 14nm ("Rocket Lake") as the difference between Rocket and Comet Lakes will give us a great natural experiment in how much microarchitecture changes matter without the usual distractions of process changes.
That used to be the "tick" in "tick/tock" but the uarch change is much bigger than usual and the process changes from Comet to Rocket will (I think) be quite small.
> partly due to Intel's fecklessness in promoting SIMD - including gratuitous fragmentation - e.g. a new Comet Lake machine with Pentium branding doesn't even suport AVX2 and has it fused off
So, Intel's problem is that it is now run by marketing droids, not engineers.
A curious aspect of this is that Intel has been doing very well financially despite having been failing technologically for a long time (if you buy the old story that Intel's process lead was at the center of it's market leadership.)
High-volume production helped Intel crush SPARC, Alpha and most of other other high-end CPU architectures in the 1990s. Today ARM is the volume leader that threatens to overtake Intel in performance.
---
Intel has segmented the market in ways that have been long-term harmful. The idea is that they look at every little feature and show that hyperscalers can get $5000 of value from feature X on the chip, so they fuse it off unless they pay $4999 for the special edition. Everybody else pays in die area, development costs, etc.
Intel's I/O choices have long been about controlling the PC platform and when you see all of the "Game of Thrones" mergers involving companies like Marvel and Mellanox so much of it has to do with that. Intel has long been stingy with PCIe lanes -- on paper it might look like you have a lot of lanes but when you deal with the motherboard and (usually less than i9) CPU in from of you will find there are capricious restrictions about how you can use those lanes.
Microsoft switched to Wayland (well, WDDM, Microsoft's Wayland-like framework) with Windows Vista and that was based on having a GPU that could do bulk operations on pixel rectangles.
Intel was mortified that a ATI or NVIDIA GPU would become necessary to the PC experience so they created Intel Integrated "Graphics" that approached the minimum performance required for Vista. If it hadn't been for deep learning and crypto mining, NVIDIA might have gone under. If it hadn't been for the merger with AMD and the SoC(s) for the Xbox and Playstation, we might not AMD GPUs.
Intel did not want to see a transition to PCI4 or an increase in memory bandwidth because they didn't sell products that could take advantage of it. NVIDIA did, so they pushed NVLink. Be it GPU, FPGA, or ASIC, you can't have a revolution in performance unless your workload is crazy compute-heavy or you have a revolution in memory and I/O bandwidth.
I don’t understand your point that Nvidia (or ATI) would have gone under if it weren’t for deep learning or crypto.
Crypto on GPU was only a relatively short lived flash in the pan, and deep learning has only been a meaningful revenue generator in the past 5 years or so.
Demand for crypto bent the demand curve at a time when GPU manufacturers needed cash and an optimistic outlook to tell investors.
The astonishing thing was the long life of the 10-series; if the 10-series was an airplane it would be the 737. Hardware innovation has usually gone faster with GPUs, but the long life of the 10-series realized that performance because software had a chance to catch up.
This is the same guy who loudly proclaims Apple moved away from Intel because of bugs in Skylake and subsequent (x)lake.
That is um..... a very CPU designer preoperative to say the least. And has been rebutted by other industry experts.
At the end of the day, none of this matters. Would better SMT performance, Single Thread Performance, Higher Clock Speed, AVX512, etc etc matter if Intel cant deliver?
It is also strange that he mentions die area concern. That is the least concern of Intel because they are vertically integrated. The cost accounting is entirely different to TSMC.
But yes, MBAs were the problem. It started when Andy Bryant became the Chairman and the Patrick Gelsinger was pushed out.
> Ryzen’s “Hyper-Threading” looked good because of poor single-threaded performance
So if my CPU is one of the worst in single-threaded performance, then multi-threaded performance will be the best? That make no sense and this guy doesn't seem like he knows what he is talking about..
He's completely right. Hyperthreading is all about putting idle functional units to work when you can't extract instruction level parallelism from single threaded code. His claim is that Intel processors did better and thus only got a small relative boost from HT (IIRC it used to be 10%, maybe it's 20% now). If your processor does poorer at ILP extraction, you get relatively more out of HT.
This all agree with the benchmarks in which Intel still generally is ahead on single thread performance. For a more extreme case look to older POWER processors which were 4-way threaded, presumably because they were even worse on single threaded code.
"Extracting ILP from single-threaded code" is energy intensive and comes with potential security issues (Spectre). SMT makes sense given these conditions.
No, if your CPU is one of the worst in single-threaded performance, then hyper-threading will look like the best thing since sliced bread on it, relative to its baseline performance. Hyperthreading will seem to double your performance, because so much of your CPU is otherwise idle. But the more you improve your scheduling, the worse hyperthreading will look, again relative to your singlethread performance.
But poor single-threaded performance doesn't imply that the core is sitting idle. It may well be running as fast as it possibly can and using all the shared components of SMT leaving nothing for the other thread to use.
In the same way the single-core performance may be really good yet everything shared by SMT is still only utilized to half its potential so the 2nd thread runs almost as fast as the first.
From the very start I've always heard HT being good for memory-bound applications. That's how it was explained and marketed.
Some years ago I worked on such an application, and we got a ~80% speed increase going from 4 threads to 8 threads on a 4-core with HT enabled. I wouldn't expect compute-bound applications to get anywhere near such gains.
Makes me wonder where those with a misconception got it from.
From most of the test that I've seen [0][1], there is not that big of difference in single-threaded performance between AMD & Intel. IMO this is rather related to difference in SMT implementation, than idication of weak ST.
That makes a lot of sense in theory, but Pentium 4 HT was a real evidence disagreeing with it.
Netburst have had horrible IPC, so in theory HT should've brought big improvements, yet in practice it didn't bring significantly more relative performance improvement than HT implementations of later Intel cores with much better IPC.
Adding to that SMT in Zen cores have brings slightly better relative performance compared to Intel cores (and Zen doesn't have as terrible IPC as Netburst), I think SMT implementation is not as trivial as we thought and before and its implementation indeed matters.
On very simple FPU code (mandelbrot set computation writen in basic C) I've observed exactly double the performance when going from 24 to 48 workers on a AMD Ryzen Threadripper 3960X 24-Core / 48 Threads Processor. I've never seen this kind of HT gain on Intel processors.
I've always wondered, if I compile an application and don't specify the architecture as explicitly having AVX, just say x86 64bit, will alternate code paths be in the binary for processors with AVX? Or alternatively, if I do specify AVX support on the command line, then what happens if the binary runs on a CPU without AVX? The reason I ask is I use commercial tools for circuit simulation, and they run on a wide range of CPU architectures, but it would be a shame if thst CPU compatability comes at a performance hit when run on a more capable CPU architecture
Wondered the same. I'm also curious if Apple's Accelerate framework uses AVX underneath when available. Sounds like an OK compromise if it's the case: maybe the compiler will miss a few opportunities to optimize some of your loops, but if you are aware of Accelerate you will use it in your audio/video processing anyway, which should be a win.
I depends on the compiler you use and the approaches it has to optional CPU features and the options you pass to the compiler.
For example, with GCC you can set the baseline CPU features. You can also use function multi-versioning to provide faster functions for CPUs with newer features.
Answer is generally no; you have to compile something with specific flags for a given architecture, and you will normally only get one path. There may be exceptions for some libraries which may do clever stuff, but generally you pick your level of machine support (usually low) and live with it.
Note that there are many new machines (Atoms and low-brand stuff like "Pentium" branded CPUs - even recent ones based on Comet Lake) which don't have anything beyond SSE4.x). IIRC they don't even have BMI/BMI2 which is a huge pain.
This is interesting, the use case I was thinking about is EDA SW, which is very expensive and performance (runtime) can be days in some cases. In the tool installs I only ever see i386 and x86_64. In curious if the EDA vendors have compiled for the lowest common denomnator for these two basic architectures. If so it would seem like they are leaving a lot of performance potential on the table. Is there any tool that explores and executable and reports if vector instructions are present and follows dependencies (like ldd)?
On Linux, you can turn on the performance counters systemwide and watch instruction counts of various types of instructions tick up, if you want. I don't know how to do this on Windows.
I imagine you could also use any number of disassembly tools to peer at the instructions - looking for xmm/ymm/zmm in the textual output would be the giveaway.
Depends on the compiler. Gcc compiles to the lowest common denominator while icc offers "Processor dispatch technology performs a check at execution time to determine which processor the application is running on and use the most suitable code path for that processor. Compatible, non-Intel processors will take the default optimized code path."
For HPC they will always compile to use the full set of instructions and see how it performs. For some software there will be explicit checks and loading the right compiled code(compression, video effects etc). On a desktop most things are compiled for the minimum processor though. The CPU might still be able to use its new features like wide registers of the instructions are in just the right order.
If you run something with instructions your processor doesn't have it will crash.
> If you run something with instructions your processor doesn't have it will crash.
I would be very surprised if the processor itself crashed. I haven't encountered it in actuality so far, but as far as I'm aware, illegal instructions raise an interrupt that the OS can catch. On Unix, this should result in the corresponding user space process receiving SIGILL.
It appears that SIGILL is actually preemptible. Could this be used as a poor man's CPU feature detection mechanism?
By default, most compilers will target some old processor with basic features, like perhaps SSE2. If you up the processor to something with AVX2, for example, you’ll get those instructions but without anything else you will SIGILL on older computers.
Now, if you want to be smarter there’s a number of things you can do, some of which have compiler or OS support. You could, in your program, manually detect these features (cpuid) and then use them if available. You could have entire functions that are specialized for one architecture and pick the right one at runtime; some compilers can do this for you “transparently” with custom attributes. If the dynamic linker supports it, you could have it pick the right version of the symbol for your processor and your program is otherwise none the wiser which one gets selected; it just has to have the appropriate versions of functions. And at the extreme end you can compile entirely separate versions of your program for different base processors; on Darwin you can have a universal binary recognized by the kernel itself as supposing x86_64 (basic 64-bit features) and x86_64h (assume Haswell and above) and the right one will get loaded based on your hardware.
None of that matters as long as Intel can't ship their 10nm and 7nm nodes. This guy doesn't seem to have an opinion on what's wrong at the fabs but that is the big life or death question for Intel right now.
I am glad to hear Intel is dabbling at TMSC (they have a plan B if not an actual business plan) but it is painful to see Intel talking as if nothing was wrong while congressmen are waking up to the reality that we might not be making chips in the us anymore.
And that's what is scary. Intel is failing left and right but the rap track from Intel sounds like an ad for Disneyland.
It is a bit odd that there is no mention of the awful security problems like Spectre and Meltdown which mostly hit Intel and was very poorly handled by them.
As a consumer i care way more about my CPU actually working as advertised and not some Specre mitigation killing the performance.
The price performance ratio of AMD is just better, nobody cares about some weird extension if i can have a Threadripper with a bazillion cores for less money doing the same stuff. Even only for marketing reasons.
> It is a bit odd that there is no mention of the awful security problems like Spectre and Meltdown which mostly hit Intel and was very poorly handled by them.
Whining about "MBAs" is almost always technical person speak for general discontent about company direction, often from the non-business parts. You don't see complaints about CEOs of Apple, Google, or Microsoft much, though they all have MBAs.
MBAs improve numbers, engineers create new products.
The mentality of MBA crowd is to get the numbers right eat the cake and when things go sideways move on to a new job.
Intel is a hardware tech business, its a complicated space that needs deep knowledge to understand it before you shape the future of it.
Technical people might not be the only ones that should run business but they should be the core of it.
I have same view as the video, engineers need more power and decision making not less to win. Intel will not fight back by cutting down cost of production by 5%. But by creating next gen chip.
Thats a great direct example of the thought process.
We will save x% by doing workarounds and call it MAX instead of biting the bullet spending money and creating next iconic plane design.
The issue isn't with MBA's per se. They can certainly be valuable to a company. The problem is when the MBA's control the direction of the company as opposed to advising it.
I think it's even more than that, it's when MBA believe that a company can do well by just being "good at business", rather than good at what their business actually does.
Yes, that's definitely part of it. But wouldn't you say that such MBA's are bad MBA's?
Isn't a good MBA supposed to not just understand business, but also everything that enables business to exist (i.e. product development, research, etc)?
The problem isn't having an MBA though is it, it's (allegedly) not being competent enough in engineering. All the CEOs you mentioned have engineering degrees, while Intel's CEO does not.
It seems to me when the company is stable, and the efficiencies come mostly from organising a large workforce MBA's do well. That is what they have been trained to do. Presumably that is what the like doing.
But that all goes to pot when the technology is changing so fast, the battle between companies becomes a battle about who advances technology fastest. In a battle fought over engineering, the engineers have a home ground advantage.
It's impossible not to notice that Apple, Google, Microsoft and Facebook were all founded by true nerds and when you here their leaders speak, most engineers feel right at home with their reasoning. When Microsoft changed their leadership to a non-nerd, it started to wither and die. Fortunately for them, they fixed that.
The reason that people whinge about is MBAs is all in the name Masters in Business Administration. They're great at doing business administration. They can drive SG&A costs down to 7% of revenue or they can optimize your inventory strategy to make sure you've always already sold all your products before you have to pay to have them made. Stuff like that. Which has a role in the business. The problem is that that's not the priority of a business that's growing, that's the priority of a business that is shrinking or declining.
I'm not knowing how much of this things are true, but isn't it ironic that AMD has more benefits from SMT because they have less pipeline optimizations (and similar), but exactly this pipeline optimizations (and similar) seem to be at least partially at fault for some of the Specter style attacks and make it harder to put more cores into the system (as this additional optimizations are likely need more silicon space...).
Sometimes I'm wondering if the best (but software side maybe impractical) design for (non Threadripper level) consumer PC's would be something along the line of.
- 1 single non SMT core with most advanced pipeline optimizations and highest clock speed which only always runs one thread pinned to it (many consumers tend to only run one "costy" applications with need for highest performance at the same time, e.g. a game main event loop).
- around this 4+ SMT enabled cores which are (mostly/reasonably) specter save and have less max. clock speed (other open applications, other threads from the current game, should be good enough for most applications)
- 1+ very low power cores which might have another arch (for the background maintenance tasks of the OS and programs like e.g. update jobs, slow long polling, always-on/connected features etc.)
- 1+ crypto core with integrated hardware security module, used to thinks like signing AES keys and similar (e.g. setup but not run TLS)
Through it's probably unrealistic as e.g. thermal regulations for high speed threads is optimized by "bouncing" the logical thread between cores and software
tends to not want to do things like write the background maintenance code for a different arch then the rest of the software, well phone OS probably could force this. Microsoft + Intel kinda can't.
The problem with this type of system is both the software and OS have to be aware of what is or isn't a "main thread" job. You have to architect your whole program around job systems to spread your data between the main thread and sub-processes which was a major complaint of gamedevs for the PS3. There's also the fact that many consumer software developers just don't care and will flag all of their processes as high priority jobs.
It worked out decently well on PS3 because it was such a locked down system, but for a general purpose desktop it would be utter chaos.
Wasn't Linux installable in early PS3s, and some labs (US military?) used stacks of PS3s as distributed superishcomputers? I wonder if anyone had analysed general computing performance in CELLs...
That's what Apple's CPUs do. They have high-power cores for expensive tasks and low-power cores for background tasks and to run the device while it is locked. It's unclear at this point if they'll keep using asymmetric cores in their desktop-class CPUs, but I don't see why not. Their kernel already supports it, and the benefits are there, especially for low-power use.
I can't wait for real-world usage benchmarks of their ARM CPUs once actual production machines are out in the hands of actual customers. It has the potential to upend the entire laptop market.
Most (all?) recent Qualcomm Snapdragons have exactly such small, low-power cores to handle background work. Extending that further is interesting, though it presents some tricky scheduling problems for the operating system.
Intel's latest Comet Lake supports per-core HT enable/disable, and supports per-core frequency long before. But I don't know how it works well by cooperating to OS
I am honestly not enthusiastic about x86 anymore. It seems like basically an entrenched legacy architecture at this point.
What are Intel's plans for HSA or RISC-5? How good are their AI products? RealSense seems pretty exciting to me.
Is there going to be a successor to the PC design? My crazy hope is that we will get some type of module system where you don't have to open the case and can just plug in something like M.2 modules but with a superfast nextgen bus.
Is there an upstart chipmaker that can help push Intel out (I wish)? I have heard good things about Nuvia.
When I hear about a company that is not run by engineers making bad decisions and pissing off engineers, I am actually eager to see them die. Just like I wish Boeing would die for similar reasons. But I know that both are very unlikely to go away. One can always dream.
Speaking of AVX-512: I find it very odd, that they still produce deaktop CPU's without AVX (not even AVX2) at all (10th gen Celerons), yet they are pushing AVX-512 into laptops at the same time.
They produced a few low volume 10nm laptop parts so they could say 10nm has finally shipped. They can’t produce enough volume to move their whole product line over.
I disagree that Intel has "lost focus" to the detriment of its core competencies, after all, Intel is a large company that spends billions on R&D every quarter. Rather I call it Intel expanding its focus to expand its addressable market to include 5G, AI, autonomous driving, advanced memory, software-defined networking, etc. I think Francois is focused on Intel being a CPU manufacturing company, when Intel sees itself as a data-processing company.
It's important to note that these aren't totally unrelated pushes (a la Google X), but rather all of the new areas leverage Intel's core competency in silicon manufacturing. (Though, many of these products are due to be fabbed at TSMC, according to rumors.)
"Xeon should dump unused core space" - Intel already does this. Intel has two CPU microarchitectures - Sunny Cove[1] (used for desktops, laptops, workstations, and beefy servers) and Tremont[2] (used for low-power clients and specialized servers). Tremont ditches the AVX instructions. Notably, Tremont is used in specialized server SoCs that target the storage, networking, and IoT/embedded markets.
Could Intel differentiate its product lineup even more? At the core level, one should ask, Are there IPs that should be added to Tremont or subtracted from Sunny Cove? Can some IPs be added at the SoC level?
Except AWS and Apple can and do recompile the ocean, even over to ARM. AVX512 free chips and gutting legacy X86 are worth it, perhaps an x86_64_2020 target?
115 comments
[ 3.0 ms ] story [ 181 ms ] threadThe creator's Twitter can be found here https://twitter.com/FPiednoel
Looks like his video is him promoting himself to investors to try to get himself on the Intel board of directors.
The current CEO is a finance / MBA type, so it's no surprise that the current culture favors that.
In Intel's defense, the previous CEO was a process engineer. But, unfortunately, he couldn't keep his weiner in his pants. He also was probably the wrong choice as CEO since, under his watch, Intel process development apparently went to shit.
Oh, well. Intel was a great company for many years. Perhaps it will reinvent itself.
He also sold all the stock he conceivably could during his tenure: https://www.cnbc.com/2018/01/04/intel-ceo-reportedly-sold-sh...
Obviously this has backfired tremendously, as we now see the rise of ARM in the processor space and even AMD to conquer x86.
But intel still make the best consumer networking cards! For whatever that is worth.
Could you be more specific?
Intel's founder, Andy Grove, used to say: "Only the paranoid survive". (He personally survived under a false identity in Nazi-occupied Hungary.) I'm afraid Intel had stopped being paranoid enough before it started to stumble more and more seriously.
The future will be more consumer appliance-like devices rather than PC-like products. I would much rather use a laptop/OS that suspended/resumed like an iPad with low power consumption (cool/quiet) and extended battery life. Apple has a way to this future. AMD can run on servers and gaming rigs. Intel needs something like a ChromeOS or Fuchsia to do everything (a cloud desktop rather than a browser) to be mainstream. I would use Clear Linux if it was more than an academic experiment--had a good desktop and repos that had a maintained useful fraction of Ubuntu packages (i.e. packages are up to date, used, have good defaults, etc).
What they are really doing there is merely converting the long-term value created by the engineers into short-term numbers that look good on the quarterly accounts and pay their own bonuses. MBAs and accountants add no value at all, they are simply profiteering from what others create. And once they have sucked an engineering company dry, they move on to the next.
My understanding is that he met his first wife while they both worked at Intel, but that was in a different era. Meeting your spouse at work used to be a common thing.
What you're referring to is more recent, but it's hard to fault the guy when Intel didn't say anything the previous time.
> Piednoel didn’t spare words for Intel’s culture, which he said has changed drastically and promotes MBAs
As organizations grow, nobody is left with a thread of understanding across departments of what is made.
That alone can explain what happened at both Intel and Boeing.
Unfortunately, anybody who does connect that thread, even before a looming disaster, will be labelled a naysayer, with no upside to them personally.
"“The state of software out there is really not favoring going larger vectors,” Piednoel said in the video. “In fact, you can see clearly in Cinebench for example—that is not one of my favorite benchmarks, especially for a laptop where it doesn’t make any sense—but you can see that AMD is winning the battle of throughput. It’s because they have more cores and they can afford to have more cores.”"
But in other places I've seen a lot of hype for it but without much discussion on actual use cases.
With now people saying that it isn't useful except to give intel leadership in benchmarks.
Is anyone here actually using it and seeing a benefit?
In fact, too low penetration of AVX-512 has been a problem, rather than too much: for a long time it has only been available on server chips, not laptop (a small fraction of laptops have gotten it recently) or destkop (outside some low-volume "extreme" parts, which were really just rebadged server parts).
It would also be quite unusual that some instructions could be very useful in benchmarks that are based on real-life, heavily used applications, but not in real life. Outside of small, easily gamed benchmark that doesn't seem plausible: if the CPU is good a video encoding benchmark, it will be good at video encoding, with high probability.
The main problem with AVX-512 is lack of software exploitation. Unlike with frequency boosts, increased cache sizes, better branch predictors, etc: this speedup doesn't come for free. Either compilers have to use the new instructions, or people have to use them by hand. The former has been very limited because these instructions cause a frequency drop (so-called "license based downclocking"), so compilers have mostly disabled their use by default: otherwise, a single AVX-512 instruction could cause a large impact on surrounding code which doesn't use AVX-512.
So by-hand exploitation remains, and penetration has just been too low and the people with the skills to do this are limited.
Microprocessors implementation is an extremely careful balance and effectively everything is a trade off. The area, power, and complexity of features comes at a non-zero cost to code that doesn't use it.
The fact that AVX-512 isn't on every processor AND the performance benefits are FAR from obvious (as the core clocks down when using it) really hinders its adoption, which in turn lowers the value; a downward spiral.
Intel is in a fine mess of their own making. I'm no fan of either company and AMD gets perhaps a little too much credit as things would have looked a lot different had 10 nm succeeded on the original schedule, but I bought my first AMD processor since Athlon 64 because it's better.
Now adoption has been poor and there is a chicken-and-egg effect as you point out, but that's separate from the question of whether AVX-512 is useful in practice. It is.
The point about tradeoffs is well taken. It's not like Intel (or any other vendor) is rolling back progress in other areas to add larger vector units: rather they try to progress along all viable axes at once.
> It's not like Intel (or any other vendor) is rolling back progress
Performance doesn't stand still so even if they are improving, they might have improved more having spend the AVX-512 on something else. And most definitely, larger vector unit does cost you somewhere else.
So, whether the instructions themselves are any good then is of secondary concern because of the other problems of Intel's own making, and you both seem to be in agreement about those.
Look on the bright side: if you don't use AVX-512 you have 10% extra heat sink and heat soak on your CPU so you can turbo higher. AVX-512 makes the chip easier to cool.
These latter parts are what's really useful about AVX-512. The widening of the vectors is just an area tradeoff that depends a lot on the particular workload.
In short Intel could not have done more to discourage use of AVX-512 if they tried.
For reference: https://twitter.com/InstLatX64/status/1131489307473174528/ph...
The bigger and more powerful AVX instruction the more power it needs - more power results in more heat, heat that modern cpus cannot handle (reason why we dont see higher cloked cpus).
Probably the biggest problem is that new Intel ISAs often are associated with downclocking (this isn't new with AVX-512 - AVX2 had it as well). Usually these problems were short-lived, but with 10nm getting stalled, they've stuck around longer. This is pure poison if like me, you generally write libraries. It's scary to look at some 60% speedup you might get from some neat rewrite for your bit of the task, but then wonder whether you're going to make the rest of the task several percent slower due to a downclock in what might only be 20% of the time. So it's hard to make the call to use AVX-512 in its early stages unless you control the whole core.
Linus thinks that everything is either a kernel or a gcc execution, and happily disregards anything that doesn't fit his own workloads. AVX-512 can do Good Things for a disparate range of stuff - in-memory databases, regex/string matching, bioinformatics, CPU intensive network workloads, parsing and lexing as well as the usual scientific computing and AI workloads (many of the latter are also pretty good on GPGPU).
He said that its a bad idea to put it in every xeon chip. AVX is great tool for specific high throughput oncore calculations, when offloading to gpu would be too costly.
Saying AVX512 is universally bad is like saying that welding machines are bad tools since carpenters never make use of it.
Sometimes it's really difficult to tell the difference between a crackpot theorist and a good equities analyst. Watching this guy's video, it's hard to tell which he comes off as more.
He's just coherent and convincing enough that as an outsider I could believe his collection of observations. But the presentation is also just disorganized enough, fixated on certain details enough, and a scattered collection enough that I can't tell whether these factors matter.
Of course I'm sure an expert in the field could tell instantly.
I've seen enough interviews of former engineers (usually retired guys) of whatever <xyz> company who know about a very specific fault of the product/tool/etc and can claim convincingly up and down that it will cause catastrophic failure, fundamental strategy misstep, etc. But will it really be a major factor for overall outcome?
He proceeds from analysis of the smallest die area detail to the opposite end of the spectrum, the problem with MBAs + Marketing, within 1 minute. With typos. Also, the guy doesn't have a long history of posting analysis of this type, and he's honest about his info being 8 years old. Not sure what his motivation for making the video is. So it's hard to put too much weight on it, as if were like a Ming-Chi Kuo predicting Apple's next iPhone form factor or something like that.
I would love to know what industry people here think, and then I might study the video / findings with more attention.
Personally, I argue with him a lot on AVX-512. I think AVX-512 is a Good Thing (or will be shortly - the first instantiation in Skylake Server - SKX - isn't great).
The biggest meta-problem with AVX-512 is that due to process issues, the pause button got hit just as it appeared in a server-only, 'first cut' form. AVX2 had downclocking issues when it first appeared too - but these were rapidly mitigated and forgotten.
I personally feel that SIMD is underutilized in general purpose workloads (partly due to Intel's fecklessness in promoting SIMD - including gratuitous fragmentation - e.g. a new Comet Lake machine with Pentium branding doesn't even suport AVX2 and has it fused off). Daniel Lemire and I built simdjson (https://github.com/lemire/simdjson) to show the power of SIMD in a non-traditional domain, and a lot of the benefits of Hyperscan (https://github.com/intel/hyperscan) were also due to SIMD. Notably, both use SIMD in a range of ways that aren't necessarily all that amenable to the whole idea of "just do it all on an accelerator/GPU" - choppy mixes of SIMD and GPR-based computing. Not everything looks like matrix multplication, or AI (but I repeat myself :-) ).
AVX-512 is a combination of fascinating and frustrating. I enthuse about recent developments on my blog (https://branchfree.org/2019/05/29/why-ice-lake-is-important-...) so I won't repeat why it's so interesting, but the learning curve is steep and there's plenty of frustrations and hidden traps. Intel collectively tends to be more interested in Magic Fixes (autovec) and Special Libraries for Important Benchmarks rather than building first-rate material to improve the general level of expertise with SIMD programming (one of my perpetual complaints).
You more or less have to teach yourself and feel your way through the process of getting good at SIMD. That being said, the benefits are often huge - not only because SIMD is fast, but because you can do fundamentally different things that you can't do quickly in GPR ("General Purpose Register") land (look, for example, at everyone's favorite instruction PSHUFB - you can't do the equivalent on a GPR without going to memory).
https://gms.tf/riscv-vector.html
I never user variable-length SIMD on CPUs, but I have substantial experience with current SIMD up to AVX2, and GPGPUs. I’m not sure I like that variable-length stuff.
1. Variable-length SIMD is only suitable for vertical operations. Many useful algorithms are only possible to vectorize with shuffles and such.
2. With registers size unknown to compilers, it’s impossible to create a calling convention to pass data in registers, as opposed to RAM. RAM is slow on modern hardware, and is becoming slower over time (in relation to compute speed; nanoseconds are fine). GPUs are OK with that because they are massively multithreaded, they switch threads instead of waiting for data from memory. CPUs can’t do that, they only have 2 hardware threads per core.
https://blog.cloudflare.com/on-the-dangers-of-intels-frequen...
https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html
Given this, and the fact Ryzen doesn't support AVX-512, I would be very wary of the technology.
> The biggest meta-problem with AVX-512 is that due to process issues, the pause button got hit just as it appeared in a server-only, 'first cut' form. AVX2 had downclocking issues when it first appeared too - but these were rapidly mitigated and forgotten.
Are there workloads that benefit from very wide SIMD vectors that aren't good fits for GPGPUs, as long as the GPUs support 64-bit floats close enough to IEEE-754 for your needs? I understand the overhead in shuffling data between main memory and GPU memory, and synchronization overhead, but most code I'm familiar with that does heavy number crunching suitable for very wide SIMD tends to do that number crunching off on threads that don't have much synchronization with threads doing more general-purpose computation.
On a side note, ISPC's input language is deceptively close to C, but little traps lie in wait. I remember helping an intern debug his port of some of the Java code I wrote over to ISPC, but it turns out that mulitplying a long long by a double in ISPC results in a long long. Our attempt to scale down a random 64-bit integer to the range [0.0, 1.0) was always resulting in 0.0. I get that integer calculations are faster, and I could understand disallowing implicit casts, but making a language so close to C, but with different implicit casting rules is just asking for trouble.
I think number crunching workloads are typically quite suitable for GPGPU - I'm certainly not trying to "debunk GPGPU", just saying that there are a lot of integer/logic intensive workloads that involve rapid switching back and forth between control/GPR-based-logic sides and "SIMD tasks" (e.g. Hyperscan switching between NFA/DFA simulation and "acceleration", which was SIMD-based character skipping).
I was involved with wrapping up a deep neural net application just as GPGPU was about to take off. We used hand coded SIMD, performance was fine, but we were not going to recode for the next generation from chips, change our error function, or change anything SIMD related.I
Before there was MMX computers came with vector units that did not have a programmer visible size, such as the Cray 1 and the vector unit for the 3090 mainframe. As it is, people hardly use vector instructions or if they use them it is something a few generations old so people pay for vector units that they don't use.
I'm sold on this as a technical pathway to performance.
Problem is, I do it in Python. I have a hard time laying out the breadcrumbs between simdjson and hyperscan, and practical, general-purpose code running more quickly on Intel CPUs. My code will run faster on AMD CPUs in practice even if, in abstract, given a few extra hours of coding, it could run faster on AVX-512. I'd rather wait a few hours for my computer than port my code to simdjson. I think that's true for 99% of the code out there.
Didn't we go down this path with Pentium IV's deep out-of-order pipeline and Itanium's VLIW before? Where Intel had great theoretical performance, but software didn't keep up?
For this to make a dent, Intel would need to hire a teach of engineers so Firefox and Chrome both took advantage of the new instructions. That's perhaps more practical than in the Itanium and PIV era, but that doesn't mean it will happen.
> For this to make a dent, Intel would need to hire a teach of engineers so Firefox and Chrome both took advantage of the new instructions.
That's more or less Intel's standard practice with its Intel compiler suite and MKL-and-similar libraries.
If there are performance gains from SIMD being left on the table, some of it is on Intel for not building the middle tooling necessary to make it easy.
(2) Even if Firefox/Chrome were built with those, they JIT Javascript. It's the JIT that matters, not what compiles the browser.
Speaking of Daniel Lemire, he recently published a paper https://arxiv.org/abs/1910.05109 mentioning downclocking in section 4:
> On some Intel processors, AVX-512 instructions are subject to downclocking: whenever AVX-512 instructions are executed, the processor lowers its frequency. We do not observe any downclocking on our processor, nor did we find any Intel documentation referencing downclocking for this processor. To test for downclocking, we used the avx-turbo benchmarking tool: it reports the processor frequency after issuing long sequences of expensive instructions of various types,including AVX-512 multiplication and floating-point instructions.
To your point, the recently front-paged simdjson writeup made me regret ignoring SIMD. I had dabbled in 3D graphics and matrix math stuff in the 90s and somehow continued to associate those extensions with that kind of work.
My bad.
Thank you for daylighting this stuff.
Is the design + production of a chip an exercise in optimizing the combination of:
-- size of die (and therefore overall cost/yield?)
-- power consumption
-- number / type / purpose of compute and memory subcomponents packed into the area
-- speed
Is that basically the envelope that you're doing an optimization within, and until a technology jump, even Intel/AMD/ARM-based are all simply choosing values between these dimensions? And each different choice of parameters within these basically goes after a certain segment of user?
Thanks!
The technology jump is of course the thing that allows you to "have it all" - until it doesn't. So it's clear that the optimization of Intel's 14nm++++++ is better for speed right now than anything they have at 10nm, which is better for density and power consumption. It used to be that the shrinks were unambiguously better but we've entered a strange new world.
Honestly, from the design perspective, I'm a enthusiastic spectator - to the extent I have insider knowledge it's out of date, poorly understood and not shareable in any case.
Microarchitecture is still very important - a better uarch and get you big effects without process changes. The last unambigiously successful one of these was Broadwell to Skylake. I think it will be fascinating to see the backport of Sunny Cove to 14nm ("Rocket Lake") as the difference between Rocket and Comet Lakes will give us a great natural experiment in how much microarchitecture changes matter without the usual distractions of process changes.
That used to be the "tick" in "tick/tock" but the uarch change is much bigger than usual and the process changes from Comet to Rocket will (I think) be quite small.
So, Intel's problem is that it is now run by marketing droids, not engineers.
High-volume production helped Intel crush SPARC, Alpha and most of other other high-end CPU architectures in the 1990s. Today ARM is the volume leader that threatens to overtake Intel in performance.
---
Intel has segmented the market in ways that have been long-term harmful. The idea is that they look at every little feature and show that hyperscalers can get $5000 of value from feature X on the chip, so they fuse it off unless they pay $4999 for the special edition. Everybody else pays in die area, development costs, etc.
Intel's I/O choices have long been about controlling the PC platform and when you see all of the "Game of Thrones" mergers involving companies like Marvel and Mellanox so much of it has to do with that. Intel has long been stingy with PCIe lanes -- on paper it might look like you have a lot of lanes but when you deal with the motherboard and (usually less than i9) CPU in from of you will find there are capricious restrictions about how you can use those lanes.
Microsoft switched to Wayland (well, WDDM, Microsoft's Wayland-like framework) with Windows Vista and that was based on having a GPU that could do bulk operations on pixel rectangles.
Intel was mortified that a ATI or NVIDIA GPU would become necessary to the PC experience so they created Intel Integrated "Graphics" that approached the minimum performance required for Vista. If it hadn't been for deep learning and crypto mining, NVIDIA might have gone under. If it hadn't been for the merger with AMD and the SoC(s) for the Xbox and Playstation, we might not AMD GPUs.
Intel did not want to see a transition to PCI4 or an increase in memory bandwidth because they didn't sell products that could take advantage of it. NVIDIA did, so they pushed NVLink. Be it GPU, FPGA, or ASIC, you can't have a revolution in performance unless your workload is crazy compute-heavy or you have a revolution in memory and I/O bandwidth.
Crypto on GPU was only a relatively short lived flash in the pan, and deep learning has only been a meaningful revenue generator in the past 5 years or so.
GPUs were doing just fine without that.
The astonishing thing was the long life of the 10-series; if the 10-series was an airplane it would be the 737. Hardware innovation has usually gone faster with GPUs, but the long life of the 10-series realized that performance because software had a chance to catch up.
Kepler - Maxwell: 28 months
Maxwell - Pascal: 20 months
Pascal - Turing: 25 months
Turing - Ampere: ?, but rumors have if for sometime September, so ~25 months
And Nvidia has never been in need of cash.
The crypto boom was just a nice extra boost, followed by a nasty bust.
One could very argue that the crypto boom was a net negative.
So did IBM, for a long time. Current revenues are the product of past effort.
That is um..... a very CPU designer preoperative to say the least. And has been rebutted by other industry experts.
At the end of the day, none of this matters. Would better SMT performance, Single Thread Performance, Higher Clock Speed, AVX512, etc etc matter if Intel cant deliver?
It is also strange that he mentions die area concern. That is the least concern of Intel because they are vertically integrated. The cost accounting is entirely different to TSMC.
But yes, MBAs were the problem. It started when Andy Bryant became the Chairman and the Patrick Gelsinger was pushed out.
So if my CPU is one of the worst in single-threaded performance, then multi-threaded performance will be the best? That make no sense and this guy doesn't seem like he knows what he is talking about..
This all agree with the benchmarks in which Intel still generally is ahead on single thread performance. For a more extreme case look to older POWER processors which were 4-way threaded, presumably because they were even worse on single threaded code.
citation needed. benchmarks that I've seen [0][1] there is not that big of difference between the two.
[0] https://www.anandtech.com/show/15578/cloud-clash-amazon-grav... ZEN1
[1] https://www.anandtech.com/show/14694/amd-rome-epyc-2nd-gen/9 ZEN2
Even normalizing for the frequency isn't enough to explain the difference
In the same way the single-core performance may be really good yet everything shared by SMT is still only utilized to half its potential so the 2nd thread runs almost as fast as the first.
I think the point of the article was more that people don't understand where HT/virtual SMT gets its speedup from.
Some years ago I worked on such an application, and we got a ~80% speed increase going from 4 threads to 8 threads on a 4-core with HT enabled. I wouldn't expect compute-bound applications to get anywhere near such gains.
Makes me wonder where those with a misconception got it from.
[0] https://www.anandtech.com/show/15578/cloud-clash-amazon-grav... ZEN1
[1] https://www.anandtech.com/show/14694/amd-rome-epyc-2nd-gen/9 ZEN2
Netburst have had horrible IPC, so in theory HT should've brought big improvements, yet in practice it didn't bring significantly more relative performance improvement than HT implementations of later Intel cores with much better IPC.
Adding to that SMT in Zen cores have brings slightly better relative performance compared to Intel cores (and Zen doesn't have as terrible IPC as Netburst), I think SMT implementation is not as trivial as we thought and before and its implementation indeed matters.
For example, with GCC you can set the baseline CPU features. You can also use function multi-versioning to provide faster functions for CPUs with newer features.
https://lwn.net/Articles/691932/
Note that there are many new machines (Atoms and low-brand stuff like "Pentium" branded CPUs - even recent ones based on Comet Lake) which don't have anything beyond SSE4.x). IIRC they don't even have BMI/BMI2 which is a huge pain.
I imagine you could also use any number of disassembly tools to peer at the instructions - looking for xmm/ymm/zmm in the textual output would be the giveaway.
For HPC they will always compile to use the full set of instructions and see how it performs. For some software there will be explicit checks and loading the right compiled code(compression, video effects etc). On a desktop most things are compiled for the minimum processor though. The CPU might still be able to use its new features like wide registers of the instructions are in just the right order.
If you run something with instructions your processor doesn't have it will crash.
I would be very surprised if the processor itself crashed. I haven't encountered it in actuality so far, but as far as I'm aware, illegal instructions raise an interrupt that the OS can catch. On Unix, this should result in the corresponding user space process receiving SIGILL.
It appears that SIGILL is actually preemptible. Could this be used as a poor man's CPU feature detection mechanism?
People have mapped out the millions of "undocumented instructions" by trapping SIGILL.
Now, if you want to be smarter there’s a number of things you can do, some of which have compiler or OS support. You could, in your program, manually detect these features (cpuid) and then use them if available. You could have entire functions that are specialized for one architecture and pick the right one at runtime; some compilers can do this for you “transparently” with custom attributes. If the dynamic linker supports it, you could have it pick the right version of the symbol for your processor and your program is otherwise none the wiser which one gets selected; it just has to have the appropriate versions of functions. And at the extreme end you can compile entirely separate versions of your program for different base processors; on Darwin you can have a universal binary recognized by the kernel itself as supposing x86_64 (basic 64-bit features) and x86_64h (assume Haswell and above) and the right one will get loaded based on your hardware.
And that's what is scary. Intel is failing left and right but the rap track from Intel sounds like an ad for Disneyland.
As a consumer i care way more about my CPU actually working as advertised and not some Specre mitigation killing the performance.
The price performance ratio of AMD is just better, nobody cares about some weird extension if i can have a Threadripper with a bazillion cores for less money doing the same stuff. Even only for marketing reasons.
And GPUs stole the vector cake years ago anyway.
I recall him mentioning exactly this.
Intel is a hardware tech business, its a complicated space that needs deep knowledge to understand it before you shape the future of it.
Technical people might not be the only ones that should run business but they should be the core of it.
I have same view as the video, engineers need more power and decision making not less to win. Intel will not fight back by cutting down cost of production by 5%. But by creating next gen chip.
When they have engineers - we get 737, 747 and many more. when they have MBAs - we get 737-MAX, 787.
As result - when things (elements) are trying to getting work together they fail spectacular.
Isn't a good MBA supposed to not just understand business, but also everything that enables business to exist (i.e. product development, research, etc)?
But that all goes to pot when the technology is changing so fast, the battle between companies becomes a battle about who advances technology fastest. In a battle fought over engineering, the engineers have a home ground advantage.
It's impossible not to notice that Apple, Google, Microsoft and Facebook were all founded by true nerds and when you here their leaders speak, most engineers feel right at home with their reasoning. When Microsoft changed their leadership to a non-nerd, it started to wither and die. Fortunately for them, they fixed that.
Sometimes I'm wondering if the best (but software side maybe impractical) design for (non Threadripper level) consumer PC's would be something along the line of.
- 1 single non SMT core with most advanced pipeline optimizations and highest clock speed which only always runs one thread pinned to it (many consumers tend to only run one "costy" applications with need for highest performance at the same time, e.g. a game main event loop).
- around this 4+ SMT enabled cores which are (mostly/reasonably) specter save and have less max. clock speed (other open applications, other threads from the current game, should be good enough for most applications)
- 1+ very low power cores which might have another arch (for the background maintenance tasks of the OS and programs like e.g. update jobs, slow long polling, always-on/connected features etc.)
- 1+ crypto core with integrated hardware security module, used to thinks like signing AES keys and similar (e.g. setup but not run TLS)
Through it's probably unrealistic as e.g. thermal regulations for high speed threads is optimized by "bouncing" the logical thread between cores and software tends to not want to do things like write the background maintenance code for a different arch then the rest of the software, well phone OS probably could force this. Microsoft + Intel kinda can't.
The problem with this type of system is both the software and OS have to be aware of what is or isn't a "main thread" job. You have to architect your whole program around job systems to spread your data between the main thread and sub-processes which was a major complaint of gamedevs for the PS3. There's also the fact that many consumer software developers just don't care and will flag all of their processes as high priority jobs.
It worked out decently well on PS3 because it was such a locked down system, but for a general purpose desktop it would be utter chaos.
I can't wait for real-world usage benchmarks of their ARM CPUs once actual production machines are out in the hands of actual customers. It has the potential to upend the entire laptop market.
What are Intel's plans for HSA or RISC-5? How good are their AI products? RealSense seems pretty exciting to me.
Is there going to be a successor to the PC design? My crazy hope is that we will get some type of module system where you don't have to open the case and can just plug in something like M.2 modules but with a superfast nextgen bus.
Is there an upstart chipmaker that can help push Intel out (I wish)? I have heard good things about Nuvia.
When I hear about a company that is not run by engineers making bad decisions and pissing off engineers, I am actually eager to see them die. Just like I wish Boeing would die for similar reasons. But I know that both are very unlikely to go away. One can always dream.
It's important to note that these aren't totally unrelated pushes (a la Google X), but rather all of the new areas leverage Intel's core competency in silicon manufacturing. (Though, many of these products are due to be fabbed at TSMC, according to rumors.)
"Xeon should dump unused core space" - Intel already does this. Intel has two CPU microarchitectures - Sunny Cove[1] (used for desktops, laptops, workstations, and beefy servers) and Tremont[2] (used for low-power clients and specialized servers). Tremont ditches the AVX instructions. Notably, Tremont is used in specialized server SoCs that target the storage, networking, and IoT/embedded markets.
Could Intel differentiate its product lineup even more? At the core level, one should ask, Are there IPs that should be added to Tremont or subtracted from Sunny Cove? Can some IPs be added at the SoC level?
[1]: https://en.wikichip.org/wiki/intel/microarchitectures/sunny_...
[2]: https://en.wikichip.org/wiki/intel/microarchitectures/tremon...