Ok, this was a fun read. Two things stood out for me, one was that their partially out of order 8 issue pipeline and the other was their watts per instruction.
For a long time Intel has been top dog in the "instructions per clock" or IPC space. So if you wanted performance you used their chips, except you paid for that by consuming a lot of power. ARM on the other hand has always tried to be the 'low power' chip which you could embed and run on batteries, and ultimately slower than the Intel architecture.
But into this a couple of interesting market realities intruded, the most obvious was that at some point computers were "fast enough" for enough people to make a durable market for lower power machines. That really took off course in the smart phone and tablet market, but is inching into the "low end" server market.
If ARM can get better at performance faster than Intel can get better at "low power" they can really put a dent into Intels market dominance in the larger computer space. And as a competitor with a completely different ISA they limit Intels ability to compete with lawsuits and/or changes to the "standard."
"In more detailed terms, the Cortex A72 CPU pairs a three-wide, in-order front end with a five-wide, out-of-order back end (i.e. 8-issue)."
... That's not how it works. You don't add front-end and back-end width to get issue width. I know that Ars' architectural chops took a hit when Stokes left, but this is ridiculous.
You take the minimum of the two[1], except that (per AnandTech's far better article [2]) the decoder is actually three fused macro-ops wide, not instructions, so the truth is actually somewhere between three and five instructions wide, depending on workload.
[1] or maybe just the back-end width, since most modern CPUs can short-circuit decode in tight loops.
The question for me is whether you can retire the FE issue with a BE dispatch, and if so pull in another FE issue. So can you get the EU to the point where the all 5 of the BE units are subscribed and the three instructions in the front end have been pre-cracked waiting for a BE slot to free up?
Well there's confusion over issue vs decode, dispatch, and retirement; its both accurate to say A72 is 8-wide issue and also that the relevant number is 3-wide or maybe 5-wide if you believe ARM's numbers with uop fusion.
After all, the Cortex-A15 and A57 are also 8-wide issue. Incidentally all 3 are also 3-wide decode and dispatch, but A72 apparently decodes instructions to fewer uops.
I don't think note #1 is true for this core. Intel CPUs have had a micro-op cache only since Sandy Bridge. I don't think ARM is doing that yet; certainly this article doesn't hint that they are.
The µop cache was new in Sandy Bridge, but earlier Intel and AMD micro-architectures had a loop buffer that served decoded macro-ops for tight loops, bypassing fetch and decode (in earlier designs still, it only bypassed fetch).
IIRC ARM added a 32-entry loop buffer with Cortex-A15. Not sure about later designs.
I've not looked at the actual micro-architecture yet (has ARM put it up on their site yet he wonders ... ohh goody something is up (http://infocenter.arm.com/help/topic/com.arm.doc.100095_0001...) but when I read that I figured the 3 wide front end could ingest instructions until the 5 wide out of order back end was full (caveat pipeline bubbles from dependencies). That should boost the IPC of the A72 relative to the A57. I'll have to spend a couple of train trips reading the TRM to figure out if we could expect double the IPC on a typical workload.
Looking at a few slides and reading comments on the Ars article, I think the error here is Ars' use of "i.e.". The part probably is 8 (micro-op) issue at execute, with 3 (fused macro-op) wide decode and 5 wide retire. So the "8-issue" claim could be accurate, but the way Ars presents it is just totally clueless.
There is a more interesting dimension to that question. ARM chips tend to come with more third party accelerators. And I think there are more silicon vendors that are willing to cram more cores on to a chip. In the server space I'd think that it will be Intel with furious single core performance and maybe 10 or 20 cores total vs ARM based with 32-64+cores and some hardware acceleration for some more operations. It will some how be an asymmetric comparison unless someone starts really knocking out high performance ARM cores.
TCO is supposedly becoming an increasingly dominant metric in the sever space, of which FLOPS/watt is probably more important than IPC. So in many server segments a class-leading "WPI" (we need some kind of name) with decent IPC could actually lead ARM to victory.
Realistically, how important is FLOPS to most servers? File servers, web servers, database servers, authentication servers... none of them tend to be heavy on floating point.
Sure, HPC is all about FLOPS, and desktops tend to use quite a bit for presentation nowadays, but I would consider memory bandwidth, I/O bandwidth, thread scaling, and single-thread performance non-FP metrics such as branch latency to be much more important in the general server market.
Feel free to mentally insert your performance metric of choice. FLOPS, IPC, branch latency, etc. There is no one all-encapsulated benchmark that I know of. I say "FLOPS" as an artifact of the past.
Well I dont think TCO really flavor ARM at all. If you look at Server as a whole, putting in the ECC RAM, HDD or SSD and Network, the CPU power/performance scaling work much better in Intel's flavor.
Another element that's being added to the mix is security. From a country's perspective, open cores and and control on chip design is valuable here. And that's exactly what the Chinese government is doing: "encouraging" their private sector to move away from intel's architecture to IBM's open power (and they'll probably won't mind an open ARM).
And who knoww, maybe later we'll see other countries shift to that kind of strategy.
That's largely immaterial in the short to medium term, in the non-mobile space anyway. Intel has superior fab technology and vastly superior fab capacity, attaining parity with Intel (which would be difficult due to being behind in process technology) would still put you at a disadvantage, you'd need to be significantly better than Intel to make up for Intel's other advantages.
> If ARM can get better at performance faster than Intel can get better at "low power" they can really put a dent into Intels market dominance in the larger computer space.
Even if it remains slightly behind on that metric, ARM is is basically untouchable by Intel in the price/performance metric. Many people, including the author seem to say stuff like "the next Core-M generation will totally beat this anyway!"
Except, Core M is nowhere near the price/performance of Cortex A72. That makes Core M uncompetitive in whatever markets the two would have to co-exist (which they don't right now, because Core M is only used in $1,000+ Windows or Mac OS X ultrabooks).
Even Atom can't be competitive on price/performance. The real (unsubsidized) gap between the two is somewhere around 2x and has remained that way since Atom's existence. The only way Atom can pretend to be competitive on price in the mobile market is because it's subsidized by Intel while it overcharges for the new Atom-based Celerons and Pentiums (Intel charges for them as much as it used to charge for the Haswell versions, and what it would've charged this year for the Broadwell versions - but those don't exist anymore).
How soon will we have 16/32 core mobile processors? It would be nice if something will force the industry to investigate alternative parallel programming paradigms.
The workloads don't support that many threads. 8-core A53 chips exist, but they don't perform anywhere near what a 2-core A57 does for typical mobile workloads.
Yes, it doesn't make a whole heck of a lot of sense to add cores when they aren't cpu bound at all.
The idea that "the slow cores can just handle the background stuff" makes little to no sense in practice.
The most power effective thing to do is not add more cores (and all that entails), but get done the work something wants done as fast as humanly possible so you can go back to sleep. Having slow cores handle this for you sounds great in theory. In practice, it's been a huge bust (The one exception being dedicated-function coprocessors, etc)
This is one reason that, for example, higher speed wifi/etc chipsets often become less power intensive than the old chipsets (past the initial spike from it being new hardware) - they can sleep more
Do you have any numbers re "little to no sense in practice"?
What I've heard suggests that the tradeoffs behind high performance ARM lead to significantly worse power efficiency to the benefit of peak performance; which means that a core like the A53 ends being a very attractive proposition for power saving.
Certainly I don't know why ARM would recommend their big.LITTLE architecture (or at least why people would use it) unless it did.
Furthermore, Amdahl's law means that even in theory it should be advantageous for performance to include a few extra low power cores to boost peak performance (above adding a larger high power core).
"Certainly I don't know why ARM would recommend their big.LITTLE architecture (or at least why people would use it) unless it did.
"
Because they want to compete with others who keep upping the core numbers.
"Furthermore, Amdahl's law means that even in theory it should be advantageous for performance to include a few extra low power cores to boost peak performance (above adding a larger high power core)."
Again, in practice, exactly zero of the chips that have done this and been put in phones (nvidia's, etc) have actually saved power by doing so.
Or did things get miraculously better when nobody was looking?
> The most power effective thing to do is not add more cores (and all that entails), but get done the work something wants done as fast as humanly possible so you can go back to sleep.
And what if you can actually scale your CPU-bound tasks linearly? Then you can use the extra cores to get the work done faster so you can go back to sleep faster.
I'm not speculating here: I've seen power measurements demonstrating this in real world applications on mobile (in my case, the browser engine). If you can use all your cores to get the work done faster, you can decrease overall power consumption.
You're assuming we aren't going to parallelize our applications. I don't make that assumption.
> I'm assuming parallelizing doesn't help enough :) Which it hasn't so far. So let me rephrase "so far, it makes no sense".
Not according to my numbers. And yes, we have measured the impact of parallel rendering specifically on power consumption on mobile. On real-world Web sites, not random microbenchmarks.
Companies aren't going to sacrifice millions of dollars to encourage development of new parallel programming techniques.
Developers barely know what to do with 4 cores, especially on mobile. The usual outcome is that foreground apps use a core, maybe two if they have async rendering, and background apps can run as well.
Mobile OpenCL is the closest we'll come to massively parallel programming in people's pockets for the foreseeable future.
It seems like you are ignorant about Servo. "We are still evaluating plans to ship Servo as a standalone product, and are focusing on the mobile and embedded spaces rather than a full desktop browser experience in the next two years." https://github.com/servo/servo/wiki/Roadmap
Imagine how much it cost to develop Servo. Now imagine rewriting every mobile app from scratch using the lessons learned from Servo. The cost would be astronomical, and for what? Incrementally better battery life?
Most mobile apps are native GUI wrappers around ffmpeg, some http library, and maybe WebKit or Blink. The high-level coördination will remain relatively unchanged as the underlying GUI, media, network, and web libraries push up against Amdahl's law.
And don't forget the king of multicore-scalable APIs: OpenGL. The reason why GPU manufacturers have been able to scale so well by adding more cores is that GLSL provides a programming model that scales broadly to (more or less) any number of cores, and applications are written to that model. This allows applications to run unmodified on new hardware with more cores and see speedups.
> Now imagine rewriting every mobile app from scratch using the lessons learned from Servo.
You just rewrite the popular libraries and game engines. Then lots of applications start to achieve parallel speedups. The point of Servo is in fact to achieve just that—don't forget that a browser is not just an application but a platform, running applications that happen to be written in (up-to-now) essentially-single-threaded JavaScript, HTML, and CSS. But as a specification, CSS and HTML (especially CSS) are in fact quite surprisingly parallelizable, and our tests have shown that existing apps get great speedups unmodified in Servo.
Taking the underlying framework and parallelizing it is an effective technique to take what were previously single-threaded applications and making them parallel. In fact, in a sense that's what superscalar CPUs have already been doing for decades.
We probably won't for a very long time. Many of the architectural designs that sit behind current processors would have to be entirely rethought.
For example snoopy caches would I think stop scaling by that point, and in order to fit the cores onto the chip you'd probably have to get rid of a lot of the control flow logic (superscalar, branch predictor), so you'd lose a tonne of performance on singlethreaded code.
You really don't want to move onto 16 cores unless you have to. More likely is more powerful instructions (perhaps true vector instructions).
A72 is nice, but to really compete in the server market I think they need another product. One that drops the performance/watt metric for the "as fast as it goes" metric. That is because while a lot of people are going to want efficient chips, if ARM really takes off there will definitely be people who want to share a binary with a CPU that is screaming fast.
> A72 is nice, but to really compete in the server market I think they need another product. One that drops the performance/watt metric for the "as fast as it goes" metric.
That seems like vertical scaling instead of horizontal scaling. I'm not sure how much money there is in the former market—maybe look at mainframe markets.
Horizontal scaling is a much easier sell, especially for data centers. I will put money on the fact that Facebook will eventually invest heavily in this architecture design.
They will literally NEVER beat Intel at the speed game in servers and they know it. If you want blazing fast, you've already got a market leader that's held the crown almost their entire existence.
? I'm not sure that is accurate. Before the PPRO, there were the RISC vendors, which overwhelmingly were significantly faster. In the early 2000's there was AMD, for about 5 years was faster in nearly every regard. Still today, you have things like the POWER 8, which while not necessarily faster for every workload definitely can beat Intel's offerings for certain workloads. So at the moment it seems to be similar to the early 80's when it wasn't clear cut who had the fastest CPU because it depended on workload. Hence my expectation that ARM will get there, if they put in the effort.
Issuing up to 8 instructions per clock (3 in order, 5 OoO) is single-thread serious performance. It looks like is going to be much better than the NVidia Denver CPU (also 8 instruction/cycle, but all of them in-order). It will be scary comparing 20-70$ ARM CPUs performing similar to Intel's in the 100-200$ price range.
"There's also a reworked 3-way L1 cache that's "almost as powerful as direct-mapped cache," and a much smaller (~10 percent) and reorganized dispatch unit."
What? I'm guessing it is a typo for a 2/4 way set associative cache, or something I know not what.
50 comments
[ 4.6 ms ] story [ 106 ms ] threadFor a long time Intel has been top dog in the "instructions per clock" or IPC space. So if you wanted performance you used their chips, except you paid for that by consuming a lot of power. ARM on the other hand has always tried to be the 'low power' chip which you could embed and run on batteries, and ultimately slower than the Intel architecture.
But into this a couple of interesting market realities intruded, the most obvious was that at some point computers were "fast enough" for enough people to make a durable market for lower power machines. That really took off course in the smart phone and tablet market, but is inching into the "low end" server market.
If ARM can get better at performance faster than Intel can get better at "low power" they can really put a dent into Intels market dominance in the larger computer space. And as a competitor with a completely different ISA they limit Intels ability to compete with lawsuits and/or changes to the "standard."
... That's not how it works. You don't add front-end and back-end width to get issue width. I know that Ars' architectural chops took a hit when Stokes left, but this is ridiculous.
You take the minimum of the two[1], except that (per AnandTech's far better article [2]) the decoder is actually three fused macro-ops wide, not instructions, so the truth is actually somewhere between three and five instructions wide, depending on workload.
[1] or maybe just the back-end width, since most modern CPUs can short-circuit decode in tight loops.
[2] http://www.anandtech.com/show/9184/arm-reveals-cortex-a72-ar...
It's like when you add a few cortex m0's to your phone, and now you have a quad-core phone.
(It's 8 issue. You can issue 8 instructions to it, it just doesn't process them all in a single cycle :P)
The front end feeds the back end.
3-issue FE means you can give the BE 3 things per cycle.
5-issue BE means you can crunch 5 things from the FE per cycle.
FE = branch prediction, decode, instruction cache
BE = schedule, execute, load/store, data cache
I'll leave my comment though for the edification of others.
So ...
And then maybe zero retires, that lets Ins1 proceed and it retires 1, 2, 3, 4, which lets 5 proceed, that lets 6 retire and then 7 proceeds.But until I get a closer look at the TRM this is all speculation.
Almost certainly yes. It's supposed to be fully out-of-order, which means it should have a fully functional scheduler in between the FE & BE.
Not to mention, given modern memory latencies (vs. clock speed), letting the FE run ahead is important for performance.
After all, the Cortex-A15 and A57 are also 8-wide issue. Incidentally all 3 are also 3-wide decode and dispatch, but A72 apparently decodes instructions to fewer uops.
IIRC ARM added a 32-entry loop buffer with Cortex-A15. Not sure about later designs.
I've seen people run reddit bots using a Raspberry pi. After a few months its cheaper then a VPS!
Sure, HPC is all about FLOPS, and desktops tend to use quite a bit for presentation nowadays, but I would consider memory bandwidth, I/O bandwidth, thread scaling, and single-thread performance non-FP metrics such as branch latency to be much more important in the general server market.
And who knoww, maybe later we'll see other countries shift to that kind of strategy.
Then came the PC clones and most nations stopped making their own stuff and just imported parts, or whole computers, from Asia.
Even if it remains slightly behind on that metric, ARM is is basically untouchable by Intel in the price/performance metric. Many people, including the author seem to say stuff like "the next Core-M generation will totally beat this anyway!"
Except, Core M is nowhere near the price/performance of Cortex A72. That makes Core M uncompetitive in whatever markets the two would have to co-exist (which they don't right now, because Core M is only used in $1,000+ Windows or Mac OS X ultrabooks).
Even Atom can't be competitive on price/performance. The real (unsubsidized) gap between the two is somewhere around 2x and has remained that way since Atom's existence. The only way Atom can pretend to be competitive on price in the mobile market is because it's subsidized by Intel while it overcharges for the new Atom-based Celerons and Pentiums (Intel charges for them as much as it used to charge for the Haswell versions, and what it would've charged this year for the Broadwell versions - but those don't exist anymore).
The idea that "the slow cores can just handle the background stuff" makes little to no sense in practice.
The most power effective thing to do is not add more cores (and all that entails), but get done the work something wants done as fast as humanly possible so you can go back to sleep. Having slow cores handle this for you sounds great in theory. In practice, it's been a huge bust (The one exception being dedicated-function coprocessors, etc)
This is one reason that, for example, higher speed wifi/etc chipsets often become less power intensive than the old chipsets (past the initial spike from it being new hardware) - they can sleep more
What I've heard suggests that the tradeoffs behind high performance ARM lead to significantly worse power efficiency to the benefit of peak performance; which means that a core like the A53 ends being a very attractive proposition for power saving.
Certainly I don't know why ARM would recommend their big.LITTLE architecture (or at least why people would use it) unless it did.
Furthermore, Amdahl's law means that even in theory it should be advantageous for performance to include a few extra low power cores to boost peak performance (above adding a larger high power core).
Because they want to compete with others who keep upping the core numbers.
"Furthermore, Amdahl's law means that even in theory it should be advantageous for performance to include a few extra low power cores to boost peak performance (above adding a larger high power core)."
Again, in practice, exactly zero of the chips that have done this and been put in phones (nvidia's, etc) have actually saved power by doing so.
Or did things get miraculously better when nobody was looking?
And what if you can actually scale your CPU-bound tasks linearly? Then you can use the extra cores to get the work done faster so you can go back to sleep faster.
I'm not speculating here: I've seen power measurements demonstrating this in real world applications on mobile (in my case, the browser engine). If you can use all your cores to get the work done faster, you can decrease overall power consumption.
You're assuming we aren't going to parallelize our applications. I don't make that assumption.
I'm assuming parallelizing doesn't help enough :) Which it hasn't so far. So let me rephrase "so far, it makes no sense".
and it's not like they haven't existed for quite a while, so it's not "well, it's a chicken and egg problem".
Not according to my numbers. And yes, we have measured the impact of parallel rendering specifically on power consumption on mobile. On real-world Web sites, not random microbenchmarks.
Developers barely know what to do with 4 cores, especially on mobile. The usual outcome is that foreground apps use a core, maybe two if they have async rendering, and background apps can run as well.
Mobile OpenCL is the closest we'll come to massively parallel programming in people's pockets for the foreseeable future.
You just rewrite the popular libraries and game engines. Then lots of applications start to achieve parallel speedups. The point of Servo is in fact to achieve just that—don't forget that a browser is not just an application but a platform, running applications that happen to be written in (up-to-now) essentially-single-threaded JavaScript, HTML, and CSS. But as a specification, CSS and HTML (especially CSS) are in fact quite surprisingly parallelizable, and our tests have shown that existing apps get great speedups unmodified in Servo.
Taking the underlying framework and parallelizing it is an effective technique to take what were previously single-threaded applications and making them parallel. In fact, in a sense that's what superscalar CPUs have already been doing for decades.
For example snoopy caches would I think stop scaling by that point, and in order to fit the cores onto the chip you'd probably have to get rid of a lot of the control flow logic (superscalar, branch predictor), so you'd lose a tonne of performance on singlethreaded code.
You really don't want to move onto 16 cores unless you have to. More likely is more powerful instructions (perhaps true vector instructions).
I think they're trying to beat both to the punch. :(
That seems like vertical scaling instead of horizontal scaling. I'm not sure how much money there is in the former market—maybe look at mainframe markets.
Horizontal scaling is a much easier sell, especially for data centers. I will put money on the fact that Facebook will eventually invest heavily in this architecture design.
What? I'm guessing it is a typo for a 2/4 way set associative cache, or something I know not what.