Amazing article, here's the synopsis at the bottom of the article:
Summary of findings and conclusions drawn
* macOS 12.1 manages the four E cores in the original M1 chip, and the two in the new M1 Pro, differently.
* In response to a high load of low QoS processes, frequency of E cores in the original M1 chip remains about 1000 MHz.
In response to a high load of low QoS processes, frequency of E cores in the M1 Pro is doubled to about 2000 MHz.
* This management policy ensures that demanding background processes with low QoS can complete in similar times on the original M1 and M1 Pro chips. Had M1 Pro chips followed the same policy as the original M1, those processes could have taken twice as long.
* In response to a spillover load of high QoS processes from the P cores, frequency of E cores in the M1 Pro is increased to the maximum of 2064 MHz.
When running at 100% active residency and maximum frequency, the cluster of 2 E cores in the M1 Pro consumes 210 mW of power.
* When running at 100% active residency and maximum frequency, each cluster of 4 P cores in the M1 Pro consumes 4 W of power.
* Although there’s insufficient evidence here to conclude that all cores within a cluster are run at exactly the same frequency, their frequencies don’t appear to differ by much. Within a cluster, active residencies normally remain broadly similar but aren’t as closely correlated as frequencies.
In tight loops of predominantly floating-point code, accessing only registers as used here, performance measured by timing correlates closely with that measured by powermetrics as instructions retired.
* The two E cores in an M1 Pro, when at 100% active residency and maximum frequency can outperform a single P core at 100% active residency and maximum frequency, while using one fifth of the power.
Imagine a laptop that was just E-Cores but 1.5x the IPC, a la some future M-revision but with defects where they only shipped it with say 8 E-Cores ... that'd be perfect for me and even better if it was in the 12-inch MacBook form factor just with an edge-to-edge screen.
And the entire M1 SoC uses about 100 mW with the OS ~idle, screen on, writing this HN comment:
ANE Power: 0 mW
DRAM Power: 32 mW
CPU Power: 85 mW
GPU Power: 18 mW
Package Power: 103 mW
I've left an M1 Air powered on, screen off, WiFi connected, SSH session active and running a shell loop printing the date every minute from another machine. After 3 hours, the battery was still at 100%, and the system finally went to sleep and killed the session. Waking it up, after another 3 hours, it had dropped to 98%. That is how you do power management.
Easily installable Linux shouldn't take much longer, for those adventurous enough to be happy with a bunch of missing hardware support (but still a usable machine for a subset of use cases). We're pretty much tying up loose ends to get to that point now, I have a quickly shrinking list of TODOs. Of course, "proper" support for everything will take longer :-)
Seriously though, a big part of my role in the first year has been building tooling to make that part of the process efficient and enjoyable for everyone. We have a Python REPL for interacting with the bare metal hardware, scaffolding for building prototypes of hardware drivers in it (e.g. register and register map definitions), things like the ability to run functions on another CPU core for SMP tests, hardware exception handling so it's actually hard to crash and have to reboot, and even a hypervisor that can run full macOS and inspect all of its hardware accesses, with a rich event tracing framework that goes from raw register maps to coprocessor message passing to decoding remote procedure calls with C++ argument unmarshaling between firmware and the OS, in both directions. All controlled from any host machine running Linux or macOS, over a simple USB connection with the target.
Holy smokes! I had no idea, but honestly, that makes a ton of sense.
Something that's always fascinated me are drivers -- those lowlevel bits of software make everything so nice. Sucks when something isn't supported. But it's one thing to take a spec and having docs and things to write one but soemthing clearly more amazing to have to reverse engineer something and then write a driver for it.
Astounding! Whereas I just got a laptop from work, a Dell Inspiron that dies if I leave it in my backpack over the weekend.
How has x86-64 managed to go so wrong for mobile devices and power usage? Is it really purely because of backwards compatibility cruft, or just goals that aren’t aligned with low power usage in idle?
I think it's a bit of both, plus corporate culture at Intel/AMD not getting it (especially Intel) and then OEMs screwing things up even more with all the hardware around the CPU proper. E.g. I have an x86 laptop where the SSD gets warm in standby; clearly that's not going to do any good for standby battery life.
Since Apple controls the whole stack, they can make sure all this stuff is done properly from top to bottom.
> E.g. I have an x86 laptop where the SSD gets warm in standby
NVMe power management is frequently this hilariously bad. There's no real coordination between the NVMe and PCIe specs on power management features, so you can end up with drives that have undeclared dependencies where you aren't supposed to try to use a certain NVMe power state unless you also have a certain level of PCIe link state power management enabled. If you break the drive's assumptions, it'll usually freak out and turn off all power management, leading to power consumption often higher than when the drive's in its active power state but not doing anything. And Microsoft's NVMe driver avoids using a lot of the power management functionality that drives implement, so laptop OEMs and drive vendors have little incentive to clean up the mess (not that they're any good at ensuring the power management techniques used by Windows aren't broken).
Can you quantify exactly how much you expect 5nm to buy versus 7nm and why? We're seeing wins which are much larger than a single process shrink could explain — Intel's been lost in the woods for years but AMD has generally been doing a quite respectable job but the story isn't night-and-day better there, either.
> … or built vertically integrated by the OS vendor?
I believe this is rather the point: x86 has some overhead baked in due to compatibility reasons but that's nowhere near enough to explain the differences we're seeing. I think what we're seeing is the fact that Microsoft, {Intel,AMD}, and the ISVs have not done a great job coordinating because no one party owns power efficiency and most of the big wins require coordination between companies who are prone to suggesting the hard work be paid for by someone else. If, say, PC reviews lead with performance/watt metrics that might change quickly.
as someone else said, apple is a node ahead of AMD this round. that won't necessarily continue; they don't own TSMC. I expect the gap will close somewhat if they both launch their next gen parts on the same node.
> goals that aren’t aligned with low power usage in idle?
some of this too. intel and AMD architectures need to work for laptop, desktop, and server markets. that's a pretty wide optimization target.
on the other hand, apple's m1 targets ipads, laptops, and laptops in a desktop chassis. they have the opportunity to focus much more intensely on the low end of the perf/W curve.
some of this too. intel and AMD architectures need to work for laptop, desktop, and server markets. that's a pretty wide optimization target.
And yet, my previous passively-cooled MacBook Air M1 was faster than a workstation-class Ryzen 3700X with a large active cooling block. And my MacBook Pro with an M1 Pro is somewhere between a 5850X and 5900X performance-wise. Both have 105W TDP and require again a large active cooling block, while the MacBook's fans barely spin up.
Then, thanks to the AMX matrix multiplication unit, the M1 Pro has almost 2x sgemm throughput of a 5900X (2254 vs 1390 GFLOP/s). The M1 goes toe to toe with the 5900X (1217 GFLOP/s) while only using 7.5W.
Sure, of the performance per watt can be explained by a better process node or focus on portable devices. However, the fact that M1/M1 Pro are as fast as high-end Ryzen workstation CPUs at a small fraction of the TDP, shows that Apple also put some of the finest engineering work in the M1.
According to AMD you get >1.25 performance at 1/2 the power usage when chips go from 7nm to 5nm. So when x86 gets to 5nm the difference in performance might be fairly negligible. Its not everything but its certainly not nothing.
TSMC claims 15% higher clocks OR 20% lower power for the same architecture moving from N7 to N5. There's a HUGE asterisk sitting somewhere and I'm pretty sure I know where.
Chips with loads of cores like Milan (the one mentioned in that article) have very low clockspeeds to keep down the TDP. I'd guess that their 2.2-3GHz clocks are easily possible with high-density designs (high-density usually going hand-in-hand with lower clockspeeds), so performance won't suffer and might even improve due to shifting power savings into slightly higher clocks.
That's the upside, but there is also a definite downside.
Desktop chip clockspeeds will suffer. If they are made on these processes, they simply will not be able to achieve those high clocks (probably 3.5-4GHz at the absolute top end). That means that even if Zen 4 gets a 20% IPC increase, it just breaks even (or even regresses) with the 20-30% clockspeed decrease.
TL;DR -- While server chips are pretty much all upside, desktop chips seem to break even or suffer performance regressions.
Hah yes a fellow engineer showed me a “trick” the other day. They don’t want you trying to close their corporate malware so if you open task manager it’ll throttle down its resource usage to keep you from noticing how it’s slowing everything down.
So if you just run task manager in the background the computer will run faster.
The M1 Air has a ~50 Wh battery, ~2 % over ~six hours is ~0.15 W total system power. For comparison, I'm getting ~0.9 W total system power under similar conditions on a five year old laptop (i5-7200U, 16 GB non-LP DDR4, no memory DVFS, MX500 SATA SSD, 1080p IPS screen).
But you'll likely find that the screen/display system is the biggest power hog in light use. For the laptop above just turning the screen on at a low brightness requires around 1.3 W (for a total system power of around 2.2-2.3 W).
FWIW that's the project lead for Asahi Linux (the effort to build a distro working perfectly on M1), I expect they know what you're talking about already :)
> Although there’s insufficient evidence here to conclude that all cores within a cluster are run at exactly the same frequency, their frequencies don’t appear to differ by much.
They do run at the same frequency. There is one hardware register to control the frequency for the whole cluster. Any differences you see are measurement errors in powermetrics; for example, for the P-cores, it has a habit of reporting the requested frequency for a core, which may not be achievable if it is a boost frequency (only available with some cores in deep sleep) or there is throttling involved, and then those metrics are incorrect.
Use the cluster frequency metric and ignore the core frequencies; the latter are not useful for anything, it's synthetic data.
> The two E cores in an M1 Pro, when at 100% active residency and maximum frequency can outperform a single P core at 100% active residency and maximum frequency, while using one fifth of the power.
This varies by workload. I did some basic benchmarking to come up with numbers for this, since the Linux scheduler needs them to efficiently schedule processes. The ballpark is that one E-core has about 70% of the performance of a P-core per MHz. Given that E-cores go up to 2064MHz and P-cores up to 3204MHz, that means that a maxed out E-core can perform at around 45% of a maxed out, full boost P-core (which only works if the other 3 cores are in deep sleep); without boost it's more like 48%. However, for some workloads this is way off - Dhrystone actually gives numbers for E-cores that are a mere 32% of the P-core numbers per MHz, making a maxed out P-core 5 times faster than a maxed out E-core, suggesting that that benchmark gets a huge boost out of the wider dispatch in the P-cores. It seems to be an outlier, though, since I haven't found anything else with that disparity.
This is an obsolete branch that I'll have to rewrite, but here's the device tree description for the CPU clusters/cores and frequency/P-state settings for the M1 that I wrote for Asahi Linux:
Another fun thing in there, which is replicating macOS behavior, is that when the P-core cluster frequency exceeds 2GHz it makes a change to memory controller power management settings, to increase the time-outs before it goes into lower power modes. This reduces DRAM latency under bursty/sparse load/store patterns, which slightly increases performance for some workloads. You can actually further disable DRAM PM and get even better latencies measurable in synthetic benchmarks (designed to measure this), though I wonder if it'll make any difference in any real world workload at that point; I'll have to test it one day.
More random fun stuff: this is a table of the latency of switching between different CPU frequencies, as well as the worst-case pipeline stall during the switch. Better than Intel, at least from the numbers I could find :-) https://mrcn.st/p/Zyok1j0g - one cool thing you might not know is that it always takes longer to go up in frequency than to go down, because going up implies raising the voltage first and that takes time; going down can happen immediately and the voltage can take its time to drop down to the new minimum to save power.
That table also shows that I found that a 24MHz "pstate 0" exists, which is probably a PLL bypass (the external refclock is 24MHz) and quite useless (it is definitely below peak efficiency for the cores, below which it makes no sense to run the CPU, ever), so macOS never uses it, but just a random fun discovery. It might be that the bootloader starts out at this frequency before spinning up the PLLs and configuring the p-state ...
The point of an architecture license is that you license the ISA itself but can design your own chip from the transistor up :) There's like 15 holders of ARM architecture license. As long as Apple properly implements the ISA and passes validation, they can literally do whatever they want to the core itself (not just the SoC around it).
Most ARM users have much more limited licenses, and can only buy and fab IP blocks designed by someone else (usually ARM).
That is the case of Amazon, the Graviton used A72 cores, and the Graviton II uses Neoverse N1, which are derived from the A76 with more focus on server workloads.
I see this come up often, and it reminds me to go check the details (which I haven't done, as I write this, per Internet tradition)...
Apple was one of the original founders of the business entity that licenses the ARM architecture these days. Apple sold the share of that business that they directly controlled.
But the overall result is that specific terms of Apple's license to ARM intellectual property have never been disclosed, and are thought to be quite broad, a license at least on par with architecture and foundry-level licenses.
Not sinister, just an exceptional relationship between Apple and ARM, historically.
An ARM architecture license is not a licensed to use an ARM designed core. It is a license to implement the ARM instruction set on your own non-ARM design. "Architecture" in this case comes from ISA: Instruction Set Architecture. :D
So, Apple, Qualcomm, and Samsung (and Nvidia?) have ARM architecture licenses, and they then go and design their own implementations, from scratch (I don't know if the license lets them use ARM designed bits, you'd have to ask them :D ).
Most companies simply license an ARM designed core and blat that into their SoC - I assume it's cheaper but I don't know if that means the license is cheaper, or the total cost (e.g. including actually designing a cpu) is cheaper.
> They design chips and market them. They do both things like no other company on this planet.
Latter for sure, former not so much. Apple is not designing its own CPUs (but ARM does) nor they are physically manufacturing or improving the silicon process (but TSMC does), they build SoCs like many other companies out there. This means that they can't make an advantage over the competitors by:
1. Making the silicon process more advanced
2. Improving CPU microarchitecture
3. Improving CPU instruction pipeline
4. Improving CPU branch prediction or OOO execution
5. Making the TLB page-walker algorithms more efficient
6. Making better cache-coherency protocols
7. Implementing their own bus interconnect which is faster than others
8. Designing better RAM modules
9. Etc. etc. (I could go on and on but I guess the point is visible)
What Apple can do to make a difference is that they can fine tune the SoC microarchitectural details but hardly anything else. By looking at the specs of M1's, the biggest outliers with respect to other chips of the same or similar class currently existing on the market are:
1. RAM and CPUs are physically co-located on the same die
2. RAM is advanced to (LP)DDR5
3. CPU caches are massive (L1, L2 and TLB)
4. There's a separate cluster of CPUs for low-power mode execution
First three things from the list is what is making M1's so fast, and especially the third point. Enlarging CPU caches has been done by the server CPUs like forever and which is the reason why quite old (2014) 2x Xeon system can eat AMD Ryzen 9 5950X for breakfast in some of the multicore workloads I ran (and I mean substantially). Last thing from the list above is what is making this chip more power efficient and this technique is as old as the moment when big.LITTLE was introduced in 2011. In nowadays ARM chips it is known as DynamIQ but the concept is the same: conserving energy by running less power hungry CPUs when workload does not require the full power (which in average user is most of the time).
> Apple is not designing its own CPUs (but ARM does)
Do you have any evidence supporting this? If you were right, it would seem odd that Apple has consistently delivered significantly better performance, often by multiple product generations, than other ARM licensees since the launch of the A8 in 2014 if ARM was doing all of the real work. Similarly, it seems like they could have saved a ton of money not hiring chip architects if they didn't have anything for them to work on.
> Apple is not designing its own CPUs (but ARM does)
Apple IS designing its own CPUs. They have been designing their own CPUs since their A4 processor.
Apple has an Architecture License with ARM [1]. Other companies with the same license include Qualcomm, Samsung, Nvidia, and Broadcom.
These licensees have full architectural freedom with the cores as long as they are ARM ISA compatible. For example, Apple has added custom instructions in their M1 processor [2].
So, an Apple ARM processor core is different from a Samsung ARM core is different from a Qualcomm ARM core.
And apparently in Apple's case, they get to be a little bit incompatible (no nVHE mode, crazy custom ISA extensions, ...)
Which is also obvious proof that they're their own designs, because literally nobody else could or would implement the same Apple-proprietary ISA extensions. Here, we use some of the custom instructions in m1n1:
Nope, you need to be compatible with the architecture specification, and it lays out exactly what can be implementation defined and what can't. E.g. you're allowed some freedom in what features to implement (Apple doesn't implement EL3 and this is fine), and you can add implementation defined system registers (Apple has a huge number of them, e.g. to implement TSO for Rosetta). But you can't decide not to support mandatory features (Apple forces on VHE mode, which is not legal - VHE is optional, non-VHE mode isn't), nor can you add extensions to the core ISA. Apple added AMX, memory compression/decompression, a variant of the AT instructions that outputs to a GPR, and the whole Guarded Execution feature (two new parallel exception levels and machinery to call to/from them and lock down things to them), and possibly more, all of them in reserved instruction encoding space.
> Apple IS designing its own CPUs. They have been designing their own CPUs since their A4 processor.
That is not quite true, the A4 is an internal SoC design but commonly "CPU" in an SoC context will designate the cores (the µarch). A4 used a standard and pedestrian Cortex-A8 there.
The A6 is where they started designing the cores in-house (to very impressive and pretty universally praised results, especially for a first iteration), following which they straight nuked the entire field by releasing the first AArch64 core in A7.
> A4 used a standard and pedestrian Cortex-A8 there.
My understanding was that the A4 Cortex-A8 was a modified version by Intrinsity which apple bought out, but, you’re right. Apple didn’t start actually designing their own cores until the A6 [1].
Almost all of this is inaccurate. You can read about their silicon program in pretty much any trade magazine for details but for example: their main cores have been in-house designs for about a decade, RAM is on-package not on-die, that is probably only a moderate perf win in itself, they have been contributing to and extending the ISA for many years, etc.
What the Apple chips do appear to be is a successful application of the "lead bullets" approach to good engineering. If you compare to Zen 2 it's clear that Apple's designs are not really incomparably better than peers, but they are excellent. With the advantage of better software support and a better manufacturing process than other mobile CPUs they perform really well.
They don't directly compete with CPU vendors but hopefully their new products light the fire under their PC counterparts to build better products. I sure hate feeling trapped in their super buggy software ecosystem.
I think one thing that has been fairly unique in the industry is that Apple has been willing to throw silicon die area at a problem. For example, their cache sizes are massive in comparison to anything else in the consumer space.
If you sell chips to device manufacturers for a living, increasing die size cuts directly into how much you make per wafer.
When you sell devices containing your own custom chips, your profit margin does not depend on the cost of fabbing that chip alone.
As a bonus, when everyone else is trying to make a smaller die and cranks up the clocks for performance, you are hugely more power efficient (without giving up performance) when you throw die size at an extra wide instruction decode with a ton of execution units running at a much slower clock speed and have huge reorder buffers so you can track more instructions in flight.
It's an interesting hypothesis, I'm not sure how true this is. The newer Ryzen 3 and upcoming Alder Lake products are between M1 and M1 Pro sized, and they all have similar total amounts of cache. Arguably by using both a bit more area and a more advanced process Apple has a much higher transistor budget to work with, but throwing more transistors at the problem doesn't necessarily help power efficiency. The one area where they have a really decisive advantage is their ability to synchronize OS development with hardware roadmap, then force developers to play along (via app store rules).
>The newer Ryzen 3 and upcoming Alder Lake products are between M1 and M1 Pro sized, and they all have similar total amounts of cache.
Increasing the cache sizes to be as large as Apple's phone SOC is an extremely recent decision on AMD and Intel's part, as noted at the M1 introduction.
>Last year we had speculated that the A13 had 128KB L1 Instruction cache, Apple has confirmed that it’s actually a massive 192KB instruction cache.
That’s absolutely enormous and is 3x larger than the competing Arm designs, and 6x larger than current x86 designs
> throwing more transistors at the problem doesn't necessarily help power efficiency
Getting your performance from IPC improvements at a lower clock (at the cost of die space) instead of pushing the power envelope to hit ever increasing clock speeds seems to be paying off quite well.
Ah true, in phone land the differences are extra stark. Given the lower margins and lack of service revenue on the Android side it's not clear if it's economically feasible for Qualcomm etc to match them. It's possible we'll see something similar happen on the Mac side where a something like a M1 Max iMac resets the bar for workstations. Then again maybe not, most laptops still have terrible screens despite Apple showing it's a solvable problem at least five years ago.
With Phones, using a wide CPU design that runs at a lower clock has been Apple's strategy for quite some time.
For instance, when power leakage became a huge issue at 20nm, Apple was less affected since they ran their clocks so much lower.
>Apple is something of the exception here, with the 20nm A8 proving to be a solid SoC, thanks in part to their wide CPU design allowing them to achieve good performance without using high clockspeeds that would exacerbate the problem.
Cache, maybe. But the M1 is generally a more aggressive design. Can issue more instructions per cycle, more functional units, more aggressive GPU, larger reorder window, more extra accelerators (audio, video, machine learning, etc), in package ram, more memory channels (8 x 16 bit on the lowest end M1), etc. There's substantial advantages in memory bandwidth on the higher end M1s. M1 pro = as much bandwidth as a 8 channel AMD Epyc server chip. M1 max = as much bandwidth as a 16 channel AMD epyc dual socket. Some of the M1 products don't even have fans! I have a M1 mini that runs silent fast and makes my 1 year old MBP 16" Intel i9 look pretty terrible, even if I ignore the jet engine sounding fans that turn on when patching, video streaming, or doing nearly anything but idle.
Keep in mind that most laptops have a 128 bit memory interface as do even high end desktops like i7/i9 and the ryzen 59XX. M1 max and pro are 256, and 512 bits and running at 4266 Mhz!
Performance per core near the best Intel and AMD have to offer, but perf/watt is crazy better.
A lot of the issues with the GPU come down to legacy software that isn't ARM native and/or doesn't support Apple's GPU/compute API, Metal.
It's a similar situation to legacy x86 software that doesn't support AVX-512. It still runs, but could run faster if it supported the available hardware.
For example, World of Warcraft is available in an ARM native version that supports Metal and can run at 120 FPS on the 16 inch MacBook M1 Max with the graphics settings cranked up to 10.
I think it is their software philosophy that I find objectionable, mainly in iOS and OSX. But their design and hardware is great enough to for someone like me to own a MB Pro, and iPad Pro.
Let's set the stage here though; the most powerful, affluent company in the world made a new chip on silicon that could not be fabricated or bought by any other manufacturers in the world. This new CPU runs a foreign architecture, refuses to run x86 code unless recompiled into "native" instructions. Their decades-old chip manufacturing team manages to make a CPU that's just barely competitive with the rest of the market and... has slightly better battery life? Is it uncouth to say that I was expecting a little more here from the biggest company in the world?
I don't hate Apple because they're a "design and marketing" company; those just happen to be what they're best at. I hate them because they're lazy, they waste money and spend every drop of goodwill they ascertain on locking in consumers, reducing the choice they have inside their ecosystem and raising the wall around their garden. Apple is a company driven by the same ethos that ruined Microsoft 10 years ago, and their reckoning is fast-approaching. The M1 still can't run a ton of software. It is the compatibility underdog, and the x86 developers of the world (at least speaking anecdotally here) simply don't care.
Apple is still not the best chip designer (Nvidia and even AMD nowadays have them beat there), and their marketing is only powerful because they can say whatever they want and people will buy it anyways. You're right on one account though; nobody else markets like they do.
I am not sure what world you live in? Their entry-level chip (the M1) beats workstation-class AMD CPUs like the Ryzen 3700X in most benchmarks, while being passively cooled and using 15W for the whole SOC, while the 3700X uses at least 65W for the CPU alone and requires a large cooling block. All while being passively cooled.
In matrix multiplication workloads, that same entry-level CPU is roughly as fast as a 12 core Ryzen 5900X, while using only 7.5W (the 5900X pretty much goes up to 105W for the same workload).
And all that while delivering stellar battery life.
You can hate Apple for their politics, walled-garden approach, etc. But your comment completely mischaracterizes the engineering feat of M1 CPUs. They have gone from having no presence on laptops/desktops to setting the gold standard for performance per watt.
It is hard to compare a CPU that has existed for one year and two months to one that will only be released half 2022. This is the problem in a lot of discussions about the M1: people will say 'wait until AMD or Intel releases XYZ'. I can pick up an M1 machine at pretty much any electronics store. By the time XYZ is generally available, the M2 will already be in mass production (in this case 5nm AMD CPUs).
At any rate, this is all besides the point. Even when Apple was still on 7nm, their iPhone/iPad CPUs were already competitive with Intel parts at much better performance per watt (an also left Qualcomm et al. far behind). It has been clear for a few years already that your point that
to make a CPU that's just barely competitive with the rest of the market
Some of it is open source (search for "cluster" in XNU); some of it is implemented in the closed-source CLPC driver, which is described in this patent:
Apple M1 unified memory architecture(published on November 11, 2020.) is my "warehouse/workshop model"(hardware architecture section published on February 06, 2019), It is an architecture supported by mathematical models, unlike "von Neumann architecture".
Why my "warehouse/workshop model" can achieve high performance and low power consumption (take Apple M1 chip, Intel AVX-512, Qualcomm as examples)
Its mathematical prototype is the simple, classic, vivid, and widely used in social production practice, elementary school mathematics "water input/output of the pool". My theory rebuilt the theoretical foundation of the IT industry, It makes the computer theory system fully & perfectly related to mathematics in a simple and unified way: from hardware integrated circuits and computer architecture, to software programming methodology, architecture, programming language and so on. It solve the most fundamental and core major problems in the IT industry: The foundation and core of the IT theory lack mathematical support.
The general rule of thumb is that power usage tends to scale as the square of performance past a certain point so I'm surprised that the E cores are so efficient, I'd expect the two of them to draw half the power of a P core rather than a fifth!
Effectively the M1 CPUs couple a Raspberry 2 with a Raspberry 4.
Balancing what runs on what cores you can shut off the Raspberry 4 when you don't need it, which is most of the time as the GPU carries a heavy load.
Try watching a MP4 movie on your iPhone 4 and you'll see that allready back then all the heavy lifting was taken off the CPU = battery barely drained at all after 1 hour of video.
This is much less magic than the 5nm that basically offer very little Gflops/W benefit over the Raspberry 4 28nm.
They fixed it in later Touch Bars by adding back a physical escape key next to the Touch Bar. That at least makes it slightly easier to arrange the Touch Bar to not trigger random behaviour :D (apparently I float my little finger above the top left of the keyboard)
87 comments
[ 4.7 ms ] story [ 162 ms ] threadSummary of findings and conclusions drawn
* macOS 12.1 manages the four E cores in the original M1 chip, and the two in the new M1 Pro, differently.
* In response to a high load of low QoS processes, frequency of E cores in the original M1 chip remains about 1000 MHz. In response to a high load of low QoS processes, frequency of E cores in the M1 Pro is doubled to about 2000 MHz.
* This management policy ensures that demanding background processes with low QoS can complete in similar times on the original M1 and M1 Pro chips. Had M1 Pro chips followed the same policy as the original M1, those processes could have taken twice as long.
* In response to a spillover load of high QoS processes from the P cores, frequency of E cores in the M1 Pro is increased to the maximum of 2064 MHz. When running at 100% active residency and maximum frequency, the cluster of 2 E cores in the M1 Pro consumes 210 mW of power.
* When running at 100% active residency and maximum frequency, each cluster of 4 P cores in the M1 Pro consumes 4 W of power.
* Although there’s insufficient evidence here to conclude that all cores within a cluster are run at exactly the same frequency, their frequencies don’t appear to differ by much. Within a cluster, active residencies normally remain broadly similar but aren’t as closely correlated as frequencies. In tight loops of predominantly floating-point code, accessing only registers as used here, performance measured by timing correlates closely with that measured by powermetrics as instructions retired.
* The two E cores in an M1 Pro, when at 100% active residency and maximum frequency can outperform a single P core at 100% active residency and maximum frequency, while using one fifth of the power.
ANE Power: 0 mW
DRAM Power: 32 mW
CPU Power: 85 mW
GPU Power: 18 mW
Package Power: 103 mW
I've left an M1 Air powered on, screen off, WiFi connected, SSH session active and running a shell loop printing the date every minute from another machine. After 3 hours, the battery was still at 100%, and the system finally went to sleep and killed the session. Waking it up, after another 3 hours, it had dropped to 98%. That is how you do power management.
I can not wait for proper linux -- installable by a noob like me on an m1 -- to be available. Thank you for the 100x time again for your efforts here
Easily installable Linux shouldn't take much longer, for those adventurous enough to be happy with a bunch of missing hardware support (but still a usable machine for a subset of use cases). We're pretty much tying up loose ends to get to that point now, I have a quickly shrinking list of TODOs. Of course, "proper" support for everything will take longer :-)
Seriously though, a big part of my role in the first year has been building tooling to make that part of the process efficient and enjoyable for everyone. We have a Python REPL for interacting with the bare metal hardware, scaffolding for building prototypes of hardware drivers in it (e.g. register and register map definitions), things like the ability to run functions on another CPU core for SMP tests, hardware exception handling so it's actually hard to crash and have to reboot, and even a hypervisor that can run full macOS and inspect all of its hardware accesses, with a rich event tracing framework that goes from raw register maps to coprocessor message passing to decoding remote procedure calls with C++ argument unmarshaling between firmware and the OS, in both directions. All controlled from any host machine running Linux or macOS, over a simple USB connection with the target.
Something that's always fascinated me are drivers -- those lowlevel bits of software make everything so nice. Sucks when something isn't supported. But it's one thing to take a spec and having docs and things to write one but soemthing clearly more amazing to have to reverse engineer something and then write a driver for it.
How has x86-64 managed to go so wrong for mobile devices and power usage? Is it really purely because of backwards compatibility cruft, or just goals that aren’t aligned with low power usage in idle?
Since Apple controls the whole stack, they can make sure all this stuff is done properly from top to bottom.
NVMe power management is frequently this hilariously bad. There's no real coordination between the NVMe and PCIe specs on power management features, so you can end up with drives that have undeclared dependencies where you aren't supposed to try to use a certain NVMe power state unless you also have a certain level of PCIe link state power management enabled. If you break the drive's assumptions, it'll usually freak out and turn off all power management, leading to power consumption often higher than when the drive's in its active power state but not doing anything. And Microsoft's NVMe driver avoids using a lot of the power management functionality that drives implement, so laptop OEMs and drive vendors have little incentive to clean up the mess (not that they're any good at ensuring the power management techniques used by Windows aren't broken).
I suspect that hurts windows’ power usage no matter what the poor engineers at MS want :-/
My XPS i9 dies if I left it in my backpack overnight ! I can't wait to have Linux properly running on an M1 Pro, so I can justify buying one.
Can you quantify exactly how much you expect 5nm to buy versus 7nm and why? We're seeing wins which are much larger than a single process shrink could explain — Intel's been lost in the woods for years but AMD has generally been doing a quite respectable job but the story isn't night-and-day better there, either.
> … or built vertically integrated by the OS vendor?
I believe this is rather the point: x86 has some overhead baked in due to compatibility reasons but that's nowhere near enough to explain the differences we're seeing. I think what we're seeing is the fact that Microsoft, {Intel,AMD}, and the ISVs have not done a great job coordinating because no one party owns power efficiency and most of the big wins require coordination between companies who are prone to suggesting the hard work be paid for by someone else. If, say, PC reviews lead with performance/watt metrics that might change quickly.
> goals that aren’t aligned with low power usage in idle?
some of this too. intel and AMD architectures need to work for laptop, desktop, and server markets. that's a pretty wide optimization target.
on the other hand, apple's m1 targets ipads, laptops, and laptops in a desktop chassis. they have the opportunity to focus much more intensely on the low end of the perf/W curve.
And yet, my previous passively-cooled MacBook Air M1 was faster than a workstation-class Ryzen 3700X with a large active cooling block. And my MacBook Pro with an M1 Pro is somewhere between a 5850X and 5900X performance-wise. Both have 105W TDP and require again a large active cooling block, while the MacBook's fans barely spin up.
Then, thanks to the AMX matrix multiplication unit, the M1 Pro has almost 2x sgemm throughput of a 5900X (2254 vs 1390 GFLOP/s). The M1 goes toe to toe with the 5900X (1217 GFLOP/s) while only using 7.5W.
Sure, of the performance per watt can be explained by a better process node or focus on portable devices. However, the fact that M1/M1 Pro are as fast as high-end Ryzen workstation CPUs at a small fraction of the TDP, shows that Apple also put some of the finest engineering work in the M1.
Node size isn't a very good argument at this point.
https://www.tomshardware.com/news/amd-unveils-zen-4-cpu-road...
https://www.anandtech.com/show/12727/tsmc-details-5-nm-proce...
Chips with loads of cores like Milan (the one mentioned in that article) have very low clockspeeds to keep down the TDP. I'd guess that their 2.2-3GHz clocks are easily possible with high-density designs (high-density usually going hand-in-hand with lower clockspeeds), so performance won't suffer and might even improve due to shifting power savings into slightly higher clocks.
That's the upside, but there is also a definite downside.
Desktop chip clockspeeds will suffer. If they are made on these processes, they simply will not be able to achieve those high clocks (probably 3.5-4GHz at the absolute top end). That means that even if Zen 4 gets a 20% IPC increase, it just breaks even (or even regresses) with the 20-30% clockspeed decrease.
TL;DR -- While server chips are pretty much all upside, desktop chips seem to break even or suffer performance regressions.
So if you just run task manager in the background the computer will run faster.
But you'll likely find that the screen/display system is the biggest power hog in light use. For the laptop above just turning the screen on at a low brightness requires around 1.3 W (for a total system power of around 2.2-2.3 W).
They do run at the same frequency. There is one hardware register to control the frequency for the whole cluster. Any differences you see are measurement errors in powermetrics; for example, for the P-cores, it has a habit of reporting the requested frequency for a core, which may not be achievable if it is a boost frequency (only available with some cores in deep sleep) or there is throttling involved, and then those metrics are incorrect.
Use the cluster frequency metric and ignore the core frequencies; the latter are not useful for anything, it's synthetic data.
> The two E cores in an M1 Pro, when at 100% active residency and maximum frequency can outperform a single P core at 100% active residency and maximum frequency, while using one fifth of the power.
This varies by workload. I did some basic benchmarking to come up with numbers for this, since the Linux scheduler needs them to efficiently schedule processes. The ballpark is that one E-core has about 70% of the performance of a P-core per MHz. Given that E-cores go up to 2064MHz and P-cores up to 3204MHz, that means that a maxed out E-core can perform at around 45% of a maxed out, full boost P-core (which only works if the other 3 cores are in deep sleep); without boost it's more like 48%. However, for some workloads this is way off - Dhrystone actually gives numbers for E-cores that are a mere 32% of the P-core numbers per MHz, making a maxed out P-core 5 times faster than a maxed out E-core, suggesting that that benchmark gets a huge boost out of the wider dispatch in the P-cores. It seems to be an outlier, though, since I haven't found anything else with that disparity.
This is an obsolete branch that I'll have to rewrite, but here's the device tree description for the CPU clusters/cores and frequency/P-state settings for the M1 that I wrote for Asahi Linux:
https://github.com/AsahiLinux/linux/commit/6b4a8c07239a42093...
Another fun thing in there, which is replicating macOS behavior, is that when the P-core cluster frequency exceeds 2GHz it makes a change to memory controller power management settings, to increase the time-outs before it goes into lower power modes. This reduces DRAM latency under bursty/sparse load/store patterns, which slightly increases performance for some workloads. You can actually further disable DRAM PM and get even better latencies measurable in synthetic benchmarks (designed to measure this), though I wonder if it'll make any difference in any real world workload at that point; I'll have to test it one day.
More random fun stuff: this is a table of the latency of switching between different CPU frequencies, as well as the worst-case pipeline stall during the switch. Better than Intel, at least from the numbers I could find :-) https://mrcn.st/p/Zyok1j0g - one cool thing you might not know is that it always takes longer to go up in frequency than to go down, because going up implies raising the voltage first and that takes time; going down can happen immediately and the voltage can take its time to drop down to the new minimum to save power.
That table also shows that I found that a 24MHz "pstate 0" exists, which is probably a PLL bypass (the external refclock is 24MHz) and quite useless (it is definitely below peak efficiency for the cores, below which it makes no sense to run the CPU, ever), so macOS never uses it, but just a random fun discovery. It might be that the bootloader starts out at this frequency before spinning up the PLLs and configuring the p-state ...
But it's not my first rodeo reversing a SoC, and I've also worked with FPGAs and HDLs several times.
This also keeps me gobsmacked by what a good job Apple has done, designing these bad boys.
Yes, exactly. They design chips and market them. They do both things like no other company on this planet.
https://news.ycombinator.com/item?id=10190521
Most ARM users have much more limited licenses, and can only buy and fab IP blocks designed by someone else (usually ARM).
That is the case of Amazon, the Graviton used A72 cores, and the Graviton II uses Neoverse N1, which are derived from the A76 with more focus on server workloads.
Both are designed by and licensed from ARM.
Apple was one of the original founders of the business entity that licenses the ARM architecture these days. Apple sold the share of that business that they directly controlled.
But the overall result is that specific terms of Apple's license to ARM intellectual property have never been disclosed, and are thought to be quite broad, a license at least on par with architecture and foundry-level licenses.
Not sinister, just an exceptional relationship between Apple and ARM, historically.
An ARM architecture license is not a licensed to use an ARM designed core. It is a license to implement the ARM instruction set on your own non-ARM design. "Architecture" in this case comes from ISA: Instruction Set Architecture. :D
So, Apple, Qualcomm, and Samsung (and Nvidia?) have ARM architecture licenses, and they then go and design their own implementations, from scratch (I don't know if the license lets them use ARM designed bits, you'd have to ask them :D ).
Most companies simply license an ARM designed core and blat that into their SoC - I assume it's cheaper but I don't know if that means the license is cheaper, or the total cost (e.g. including actually designing a cpu) is cheaper.
Latter for sure, former not so much. Apple is not designing its own CPUs (but ARM does) nor they are physically manufacturing or improving the silicon process (but TSMC does), they build SoCs like many other companies out there. This means that they can't make an advantage over the competitors by:
What Apple can do to make a difference is that they can fine tune the SoC microarchitectural details but hardly anything else. By looking at the specs of M1's, the biggest outliers with respect to other chips of the same or similar class currently existing on the market are: First three things from the list is what is making M1's so fast, and especially the third point. Enlarging CPU caches has been done by the server CPUs like forever and which is the reason why quite old (2014) 2x Xeon system can eat AMD Ryzen 9 5950X for breakfast in some of the multicore workloads I ran (and I mean substantially). Last thing from the list above is what is making this chip more power efficient and this technique is as old as the moment when big.LITTLE was introduced in 2011. In nowadays ARM chips it is known as DynamIQ but the concept is the same: conserving energy by running less power hungry CPUs when workload does not require the full power (which in average user is most of the time).Do you have any evidence supporting this? If you were right, it would seem odd that Apple has consistently delivered significantly better performance, often by multiple product generations, than other ARM licensees since the launch of the A8 in 2014 if ARM was doing all of the real work. Similarly, it seems like they could have saved a ton of money not hiring chip architects if they didn't have anything for them to work on.
Apple IS designing its own CPUs. They have been designing their own CPUs since their A4 processor.
Apple has an Architecture License with ARM [1]. Other companies with the same license include Qualcomm, Samsung, Nvidia, and Broadcom.
These licensees have full architectural freedom with the cores as long as they are ARM ISA compatible. For example, Apple has added custom instructions in their M1 processor [2].
So, an Apple ARM processor core is different from a Samsung ARM core is different from a Qualcomm ARM core.
[1] https://www.anandtech.com/show/7112/the-arm-diaries-part-1-h... [2] https://news.ycombinator.com/item?id=25559145
And apparently in Apple's case, they get to be a little bit incompatible (no nVHE mode, crazy custom ISA extensions, ...)
Which is also obvious proof that they're their own designs, because literally nobody else could or would implement the same Apple-proprietary ISA extensions. Here, we use some of the custom instructions in m1n1:
https://github.com/AsahiLinux/m1n1/blob/main/src/gxf_asm.S#L...
That won't work on any non-Apple core.
That is maybe a little excessive, technically they could probably pay an other architecture license holder to do it for them.
That is not quite true, the A4 is an internal SoC design but commonly "CPU" in an SoC context will designate the cores (the µarch). A4 used a standard and pedestrian Cortex-A8 there.
The A6 is where they started designing the cores in-house (to very impressive and pretty universally praised results, especially for a first iteration), following which they straight nuked the entire field by releasing the first AArch64 core in A7.
My understanding was that the A4 Cortex-A8 was a modified version by Intrinsity which apple bought out, but, you’re right. Apple didn’t start actually designing their own cores until the A6 [1].
[1] https://seekingalpha.com/amp/article/4293287-setting-record-...
What the Apple chips do appear to be is a successful application of the "lead bullets" approach to good engineering. If you compare to Zen 2 it's clear that Apple's designs are not really incomparably better than peers, but they are excellent. With the advantage of better software support and a better manufacturing process than other mobile CPUs they perform really well.
They don't directly compete with CPU vendors but hopefully their new products light the fire under their PC counterparts to build better products. I sure hate feeling trapped in their super buggy software ecosystem.
If you sell chips to device manufacturers for a living, increasing die size cuts directly into how much you make per wafer.
When you sell devices containing your own custom chips, your profit margin does not depend on the cost of fabbing that chip alone.
As a bonus, when everyone else is trying to make a smaller die and cranks up the clocks for performance, you are hugely more power efficient (without giving up performance) when you throw die size at an extra wide instruction decode with a ton of execution units running at a much slower clock speed and have huge reorder buffers so you can track more instructions in flight.
https://www.anandtech.com/show/16226/apple-silicon-m1-a14-de...
Increasing the cache sizes to be as large as Apple's phone SOC is an extremely recent decision on AMD and Intel's part, as noted at the M1 introduction.
>Last year we had speculated that the A13 had 128KB L1 Instruction cache, Apple has confirmed that it’s actually a massive 192KB instruction cache.
That’s absolutely enormous and is 3x larger than the competing Arm designs, and 6x larger than current x86 designs
https://www.anandtech.com/show/16226/apple-silicon-m1-a14-de...
> throwing more transistors at the problem doesn't necessarily help power efficiency
Getting your performance from IPC improvements at a lower clock (at the cost of die space) instead of pushing the power envelope to hit ever increasing clock speeds seems to be paying off quite well.
For instance, when power leakage became a huge issue at 20nm, Apple was less affected since they ran their clocks so much lower.
>Apple is something of the exception here, with the 20nm A8 proving to be a solid SoC, thanks in part to their wide CPU design allowing them to achieve good performance without using high clockspeeds that would exacerbate the problem.
https://www.anandtech.com/show/9686/the-apple-iphone-6s-and-...
Keep in mind that most laptops have a 128 bit memory interface as do even high end desktops like i7/i9 and the ryzen 59XX. M1 max and pro are 256, and 512 bits and running at 4266 Mhz!
Performance per core near the best Intel and AMD have to offer, but perf/watt is crazy better.
It's a similar situation to legacy x86 software that doesn't support AVX-512. It still runs, but could run faster if it supported the available hardware.
For example, World of Warcraft is available in an ARM native version that supports Metal and can run at 120 FPS on the 16 inch MacBook M1 Max with the graphics settings cranked up to 10.
https://www.youtube.com/watch?v=JZV3bOpv2Rs
x86 games on OpenGL don't run nearly as well, as expected.
I don't hate Apple because they're a "design and marketing" company; those just happen to be what they're best at. I hate them because they're lazy, they waste money and spend every drop of goodwill they ascertain on locking in consumers, reducing the choice they have inside their ecosystem and raising the wall around their garden. Apple is a company driven by the same ethos that ruined Microsoft 10 years ago, and their reckoning is fast-approaching. The M1 still can't run a ton of software. It is the compatibility underdog, and the x86 developers of the world (at least speaking anecdotally here) simply don't care.
Apple is still not the best chip designer (Nvidia and even AMD nowadays have them beat there), and their marketing is only powerful because they can say whatever they want and people will buy it anyways. You're right on one account though; nobody else markets like they do.
I am not sure what world you live in? Their entry-level chip (the M1) beats workstation-class AMD CPUs like the Ryzen 3700X in most benchmarks, while being passively cooled and using 15W for the whole SOC, while the 3700X uses at least 65W for the CPU alone and requires a large cooling block. All while being passively cooled.
In matrix multiplication workloads, that same entry-level CPU is roughly as fast as a 12 core Ryzen 5900X, while using only 7.5W (the 5900X pretty much goes up to 105W for the same workload).
And all that while delivering stellar battery life.
You can hate Apple for their politics, walled-garden approach, etc. But your comment completely mischaracterizes the engineering feat of M1 CPUs. They have gone from having no presence on laptops/desktops to setting the gold standard for performance per watt.
At any rate, this is all besides the point. Even when Apple was still on 7nm, their iPhone/iPad CPUs were already competitive with Intel parts at much better performance per watt (an also left Qualcomm et al. far behind). It has been clear for a few years already that your point that
to make a CPU that's just barely competitive with the rest of the market
is nonsense.
https://patents.google.com/patent/US10417054B2/en?oq=US10417...
Why my "warehouse/workshop model" can achieve high performance and low power consumption (take Apple M1 chip, Intel AVX-512, Qualcomm as examples)
https://github.com/linpengcheng/PurefunctionPipelineDataflow...
The Math-based Grand Unified Programming Theory: The Pure Function Pipeline Data Flow with Principle-based Warehouse/Workshop Model
https://github.com/linpengcheng/PurefunctionPipelineDataflow
Its mathematical prototype is the simple, classic, vivid, and widely used in social production practice, elementary school mathematics "water input/output of the pool". My theory rebuilt the theoretical foundation of the IT industry, It makes the computer theory system fully & perfectly related to mathematics in a simple and unified way: from hardware integrated circuits and computer architecture, to software programming methodology, architecture, programming language and so on. It solve the most fundamental and core major problems in the IT industry: The foundation and core of the IT theory lack mathematical support.
Balancing what runs on what cores you can shut off the Raspberry 4 when you don't need it, which is most of the time as the GPU carries a heavy load.
Try watching a MP4 movie on your iPhone 4 and you'll see that allready back then all the heavy lifting was taken off the CPU = battery barely drained at all after 1 hour of video.
This is much less magic than the 5nm that basically offer very little Gflops/W benefit over the Raspberry 4 28nm.
if only that had been their ingenious plan.
They fixed it in later Touch Bars by adding back a physical escape key next to the Touch Bar. That at least makes it slightly easier to arrange the Touch Bar to not trigger random behaviour :D (apparently I float my little finger above the top left of the keyboard)