I honestly don't know anything about the big.LITTLE approach beyond the name and the basic idea.
As I understand it, it makes sense in battery-powered devices. But does it advantages in desktop or server machines?
I vaguely recall reading a blog post - many years ago, though - by a laptop-battery-lifetime-nerd who claimed that running the CPU at a higher clock speed actually saved energy, because it got done faster and could go back to a low-power sleep mode which consumed a lot less energy than the "awake" CPU running at a lower clock speed.
Was this ever accurate? Is it still? I do not know, so I'd appreciate it very much if someone who does know something about this topic could shed some light on this.
Completely talking out of my ass here but I assumed that the intuitive logic for big.LITTLE with regards to desktops and servers was so that low priority apps could be delegated to the lower power efficiency cores which would free up resources for more performance hungry processes on the performance cores.
"Rush to idle state" saves energy on a CPU which is designed for a higher turbo-boost frequency. When the choice is whether to run a given workload on a high-power CPU design versus a slower, lower-power one, low-power designs also have lower energy use because power use scales more than linearly with frequency.
Even CPUs designed for a higher turbo-boost frequency have disproportionate power usage increases at their highest freq, compared to the marginal perf gains. I suspect the rush to idle must on tons of model be an equilibrium esp in freq selection if you really want to optimize for power consumption. Probably why user selectable power plans make sense.
That might definitely happen on some modern CPU's where turbo boost frequencies might be overvolted, and idle frequencies conversely undervolted. But even then, the optimal frequency is going to be relatively high because these dynamic voltage variations are going to be quite limited.
AAAAAH! Had I only bothered to finish reading TFA, it would have answered my question:
"Ten years ago, I asked Intel’s smartphone SoC designers why they were relying on DVFS — Dynamic Voltage and Frequency Scaling — to keep Atom’s power consumption competitive rather than big.Little. According to Intel’s architects, DVFS was competitive with what big.Little could deliver when one considered the silicon die space requirements and the overall power savings.
That may have been true at the time — Medfield was generally competitive with the midrange Cortex-A9 devices it was intended to compete against — but it doesn’t seem to be true today."
I assume the idea is that if you're using the LITTLE cores, you turn off the big cores entirely (or at least some of them).
What's stopping you from shutting down cores with the traditional big-only design? I'm looking at per-core power consumption of my AMD 5800H right now, and I can see that it is able to fully shut down up to 7 cores out of 8, bringing down the entire package power consumption as low as 3.8 W.
Nothing, but if you need to run tasks then you need to wake these cores up.
If you have low-priority or very short tasks to run, then you can wake just a few small cores, and that's way less expensive than to wake big cores.
An other advantage is that it can help progress background tasks more efficiently: if you run background tasks on small cores then you don't need to suspend tasks which need the big cores or migrate them between threads to run background tasks.
Rush to idle is a compromise. There are two basic kinds of loads. User responsive and background tasks.
User tasks want to use the rush to idle because it improves responsiveness. Most background tasks don’t really care.
High performance cores trade power efficient for execution speed by using lots of transistors to analyze the code and optimize on the fly (literally a semi-programmable core in itself to run your core faster). In addition, power consumption is linear with clocks in theory, but not at all in practice.
Let’s consider AMD specifically. In order or semi in order cores are much narrower. All the speculation bits go away. The extra load/store complexity (very significant) also disappears because the core simply can’t use it. SMT will also go away. As it adds a ton of duplication, that also reduces size (10% if I remember Intel’s claims from a few years ago). Simple cores get much better performance per watt and per transistor, but at the expense of time. Makes a lot of sense if time isn’t critical.
On desktop, a zen 3 core at 4.5GHz will use around 12w. Dropping frequency just 200MHz will lower power usage per core by almost 2w. (from Anandtech).
In the space of that one core you mention, you could fit 4 complete in order cores with lots of room to spare. They could all run at a couple GHz and still use less than half the 3.8w you mention.
Compared to ARM, Intel has the additional problem that their Atom descendant little cores don't implement the full instruction/feature set of the big cores. That could be because the full set is (power/die-size wise) expensive, or because of market segmentation strategy, but it probably makes an Intel big.LITTLE far less straightforward than ARM.
In a similar vein, I remember the palm pre actually saved battery life when you over clocked the cpu because tasks got done faster and there was less cycling up and down, so I think it does happen.
> But does it advantages in desktop or server machines?
Yes - because some tasks are highly parallel and are better suited to running on lots of small cores, and some tasks are highly sequential and are better suited to running on a single bigger core. big.LITTLE is the compromise that suits both.
It's not just energy, chip surface is still a thing. Performance core designs will jump at almost any opportunity to go faster. 50% more transistors to get 5% more single thread performance? What are you waiting for, do it! But when it comes to highly parallelizaable loads spending the transistor budget on more cores of slightly lower performance will certainly be better.
That was the most common principle used in dual+ core smartphones as cores shut down once they're done, which made it more efficient to run them at full speed then shut down (within limits; the difference in power consumption between 2GHz and 1.4GHz is negligible, but the performance is noticeable).
Of course, laptop and desktop CPUs got that feature as well, so that idea still holds true today.
It's kind of like Auto stop/start on engines - saves more fuel than it wastes.
"Rush to idle" is very much a real thing, and a useful thing, and remains a core principle of CPU design and scheduling, however some things tend to keep running but are pretty cheap e.g. OS background processes which mostly do IO or are semi-actively waiting.
In that case "rush to idle" is not that useful, and waking a big and expensive core (not to mention ramping up its frequency) leads to much higher power consumption for limited gain.
In fact recent big.LITTLE scheduling has been leveraging this: in the oldest model the SoC or kernel would activate either the big or the little cluster[0] depending on what it believed was useful, then the CPU got split "vertically" so the kernel would see each pair of big and little cores as a single vcore, and whichever of the real cores was best for the workload on that vcore would get selected.
More recent big.LITTLE implementations have gone fully heterogenous, if you have 4 small and 4 big cores the system sees and shows 8 actual cores with different performances, and the scheduler tuning selects which tasks go where.
[0] big.LITTLE clusters are the "banks" of respectively big and little cores
>But does it advantages in desktop or server machines?
Somebody once suggested on HN that slower cores would execute unsafe code (e.g js) and they'd have all those speculation mechanisms turned off, meanwhile fast core would execute local code (e.g ffmpeg) with all those speculative mechanisms ON
that was interesting idea, I'm curious how does it reflect reality
It always seemed odd to me that my whole machine pays the cost for spectre mitigation when the only processes that need it (to me) are JavaScript vms in the browser.
Even my electron apps don't need it, they have access to the filesystem which is far better than any spectre attack.
You should have a syscall to opt into spectre protections. Like "I run untrusted code". Then the OS scheduler can treat it differently, keep the hyperthread only for other untrusted processes like that, flush the cache when scheduling something else on that core, etc.
I'm surprised we're not seeing the same thing for graphics cores. You'd think a modern desktop could be handled by a lot less horsepower than the latest Unreal based game. Maybe that's because the much higher parallelism make it easier to use a low percentage of graphics cores?
There's a rudimentary version of these on laptops: the integrated graphics and discrete graphics are co-opted to make a hybrid GPU, but it has a lot of gotchas (especially when it's not AMD-AMD (which is super rare and will still be if their new laptop designs still flopped the market)). I think that it will matter in the future for two reasons: 1) Intel being competent in graphics and 2) East Asian and European power consumption regulations (which is sometimes commented as draconian, and the reason that Intel even bothered making ATX12VO).
I find the term Hybrid a bit off in usage (maybe me) but this is just one set of full cores and another set of lesser lower power cores - alas Big/Little ARM style.
For me the term Hybrid CPU means a CPU that has more than one core design that runs different architecture. But can see how this could apply if you omit that last caveat, albeit this is power management.
What would be good, is an advanced in design tools to accommodate asynchronous(clockless) CPU design over synchronous(clock fixed ala most CPU's and all X86 today). But then the design overheads really are the limiting factor, many (not all) could be reduced with advances in design tooling. Though be mindful, how long and the state of open source FPGA tooling and how long those have been around.
[EDIT] Fixed synchronous /asynchronous in the wrong place before anybody noticed. And added some clarity)
> What would be good, is an advanced in design tools to accommodate asynchronous(clockless) CPU design over synchronous(clock fixed ala most CPU's and all X86 today). But then the design overheads really are the limiting factor, many (not all) could be reduced with advances in design tooling.
Asynchronous means that all your busses have to carry, and your logic units calculate, readiness information. That's extra work to be explicit in detail about something that's implicit in clocked designs where it's all pre-calculated and summarized into the (externally documented) maximum frequency.
I wonder if Linux is ready for it. How would tasks get scheduled? It’s really broken on Mac. The tasks get a flag for whether they are user-initiated or not, and the usual tools like renice can’t influence scheduling as far as I have found. This leads to priority inversions like the Xcode upgrade running on the “efficiency” cores while I sit there and wait for five hours. Bad experience all around and one fears for how it would work on Linux.
If I understand the article correctly, the CPU itself would take care of moving tasks between BIG and little cores, so the operating system would not need to know about it.
I'm not sure that's the best way to handle this, but I don't think I'm qualified to make a judgment one way or the other.
ARM-based phone CPUs for Android devices that use the hybrid design have been used for a while now, according
to https://en.wikipedia.org/wiki/ARM_big.LITTLE, so scheduling in Linux is either ready or at least the groundwork has been laid for it to be ready on desktop/server machines.
A big question in my mind is "do the big and little cores have the exact same feature set?".
If they do, then any process can run on any core. But even the little cores have to support fancy SIMD stuff and have hundreds of registers.
If they don't, then compilers need to be aware that simd or other features could suddenly vanish anytime. There needs to be logic for checking CPU capabilities at the start of every loop and switching to a different implementation depending which core we're currently running in. There probably also needs to be some kind of exception handling in the OS to emulate unimplemented instructions for backwards compatibility.
From reading the title, I thought they meant an ARM cpu with the same magical x86 memory ordering mode for emulation that apple put in the m1. I would love to see that!
31 comments
[ 2.7 ms ] story [ 73.7 ms ] threadAs I understand it, it makes sense in battery-powered devices. But does it advantages in desktop or server machines?
I vaguely recall reading a blog post - many years ago, though - by a laptop-battery-lifetime-nerd who claimed that running the CPU at a higher clock speed actually saved energy, because it got done faster and could go back to a low-power sleep mode which consumed a lot less energy than the "awake" CPU running at a lower clock speed.
Was this ever accurate? Is it still? I do not know, so I'd appreciate it very much if someone who does know something about this topic could shed some light on this.
"Ten years ago, I asked Intel’s smartphone SoC designers why they were relying on DVFS — Dynamic Voltage and Frequency Scaling — to keep Atom’s power consumption competitive rather than big.Little. According to Intel’s architects, DVFS was competitive with what big.Little could deliver when one considered the silicon die space requirements and the overall power savings. That may have been true at the time — Medfield was generally competitive with the midrange Cortex-A9 devices it was intended to compete against — but it doesn’t seem to be true today."
I assume the idea is that if you're using the LITTLE cores, you turn off the big cores entirely (or at least some of them).
If you have low-priority or very short tasks to run, then you can wake just a few small cores, and that's way less expensive than to wake big cores.
An other advantage is that it can help progress background tasks more efficiently: if you run background tasks on small cores then you don't need to suspend tasks which need the big cores or migrate them between threads to run background tasks.
User tasks want to use the rush to idle because it improves responsiveness. Most background tasks don’t really care.
High performance cores trade power efficient for execution speed by using lots of transistors to analyze the code and optimize on the fly (literally a semi-programmable core in itself to run your core faster). In addition, power consumption is linear with clocks in theory, but not at all in practice.
Let’s consider AMD specifically. In order or semi in order cores are much narrower. All the speculation bits go away. The extra load/store complexity (very significant) also disappears because the core simply can’t use it. SMT will also go away. As it adds a ton of duplication, that also reduces size (10% if I remember Intel’s claims from a few years ago). Simple cores get much better performance per watt and per transistor, but at the expense of time. Makes a lot of sense if time isn’t critical.
On desktop, a zen 3 core at 4.5GHz will use around 12w. Dropping frequency just 200MHz will lower power usage per core by almost 2w. (from Anandtech).
In the space of that one core you mention, you could fit 4 complete in order cores with lots of room to spare. They could all run at a couple GHz and still use less than half the 3.8w you mention.
(https://overclock3d.net/news/cpu_mainboard/intel_s_hybrid_bi...)
Yes - because some tasks are highly parallel and are better suited to running on lots of small cores, and some tasks are highly sequential and are better suited to running on a single bigger core. big.LITTLE is the compromise that suits both.
See Amdahl's Law, Gustafson's Law, etc.
Of course, laptop and desktop CPUs got that feature as well, so that idea still holds true today.
It's kind of like Auto stop/start on engines - saves more fuel than it wastes.
In that case "rush to idle" is not that useful, and waking a big and expensive core (not to mention ramping up its frequency) leads to much higher power consumption for limited gain.
In fact recent big.LITTLE scheduling has been leveraging this: in the oldest model the SoC or kernel would activate either the big or the little cluster[0] depending on what it believed was useful, then the CPU got split "vertically" so the kernel would see each pair of big and little cores as a single vcore, and whichever of the real cores was best for the workload on that vcore would get selected.
More recent big.LITTLE implementations have gone fully heterogenous, if you have 4 small and 4 big cores the system sees and shows 8 actual cores with different performances, and the scheduler tuning selects which tasks go where.
[0] big.LITTLE clusters are the "banks" of respectively big and little cores
AFAIK different C-states dictate that per-core and different caches power on.
Given that, surely this can be extended further with ALUs, FPUs etc only being powered on as necessary? I assume that is already the case no?
Or is this more towards having smaller denser cores with fewer interconnects and components for even higher power efficiency?
Somebody once suggested on HN that slower cores would execute unsafe code (e.g js) and they'd have all those speculation mechanisms turned off, meanwhile fast core would execute local code (e.g ffmpeg) with all those speculative mechanisms ON
that was interesting idea, I'm curious how does it reflect reality
It always seemed odd to me that my whole machine pays the cost for spectre mitigation when the only processes that need it (to me) are JavaScript vms in the browser.
Even my electron apps don't need it, they have access to the filesystem which is far better than any spectre attack.
You should have a syscall to opt into spectre protections. Like "I run untrusted code". Then the OS scheduler can treat it differently, keep the hyperthread only for other untrusted processes like that, flush the cache when scheduling something else on that core, etc.
For me the term Hybrid CPU means a CPU that has more than one core design that runs different architecture. But can see how this could apply if you omit that last caveat, albeit this is power management.
What would be good, is an advanced in design tools to accommodate asynchronous(clockless) CPU design over synchronous(clock fixed ala most CPU's and all X86 today). But then the design overheads really are the limiting factor, many (not all) could be reduced with advances in design tooling. Though be mindful, how long and the state of open source FPGA tooling and how long those have been around.
[EDIT] Fixed synchronous /asynchronous in the wrong place before anybody noticed. And added some clarity)
It's like we wouldn't call a purely electrical car with two different specced motors inside a hybrid.
Perhaps "asymmetric cores" would be a better name?
Asynchronous means that all your busses have to carry, and your logic units calculate, readiness information. That's extra work to be explicit in detail about something that's implicit in clocked designs where it's all pre-calculated and summarized into the (externally documented) maximum frequency.
I'm not sure that's the best way to handle this, but I don't think I'm qualified to make a judgment one way or the other.
If they do, then any process can run on any core. But even the little cores have to support fancy SIMD stuff and have hundreds of registers.
If they don't, then compilers need to be aware that simd or other features could suddenly vanish anytime. There needs to be logic for checking CPU capabilities at the start of every loop and switching to a different implementation depending which core we're currently running in. There probably also needs to be some kind of exception handling in the OS to emulate unimplemented instructions for backwards compatibility.