That's the impression I got as well. That made me discount Agner's arguments since they apply to Apple's M1, and the M1 is great in both raw performance and power efficiency.
Difference: Apple controls software + hardware and can this immediately utilize hardware efficiencies like a new Arch with a simultaneous release. Intel doesn’t control Windows.
The goal of this kind of design is M1 like energy efficiency, not performance (I don't know if the Alder Lake P cores can outperform an M1 P core.) M1 has a similar architecture with performance and efficiency cores, but so far I haven't heard of it causing problems for software developers. All versions of macOS that support M1 also have a scheduler that is able to move threads between E and P cores nicely.
I thought the secret sauce to M1 was largely that it was a new architecture, without the decades of x86 backwards compatible baggage, with an OS and software capable of running natively on it?
Both, sort of. The secret sauce to M1 battery life is a combination of OS thread scheduling and efficiency cores. The secret sauce to M1 performance is having an architecture that doesn’t need to be thermal throttled as much as in other platforms.
It’s not that Apple invented chips that are remarkably faster than Intel’s or NVIDIA’s, it’s that Apple invented an integrated experience on this new architecture with split performance & efficiency cores and did so while maintaining backwards compatibility with macOS x86 apps in a way that’s indistinguishable from M1-native apps. (So they kept the “baggage” as it were, and it still performs fine.)
I expect Microsoft’s Surface line will catch up eventually, but the disconnect between Qualcomm/Intel as chip makers and Microsoft as OS vendor will slow down the transition for a good 4-6 years easy, and we’re 2 years into this transition to ultra-energy efficient architectures with less thermal throttling. (Full disclosure, I’ve a few Apple shares.)
It's quite close to the process AMD are on but not 5nm that apple use.
A lot of M1's advantage is in terms of density. That's why they were theoretically able to make such an absolutely enormous processor and keep it cool.
Making that theory work in practice came from decades of low power semi experience. They're extremely good chip designers.
The Alder Lake node is about equivalent to TSMC's N7, maybe 10% better. N5, the node Apple is using for M1, is about 1.8x the density of N7 with 40% lower power usage.
AMD and Intel are on similar nodes at the moment, but Apple has a very notable advantage. It's not "secret sauce" so much as paying many billions for exclusive access.
This is the biggest part of the 'sauce' but having E-cores and extremely close memory (faster memory access - less time the cores need to wait for data - less time spent doing nothing and burning watts) surely helps.
My understanding is that Alder Lake P-cores have the highest single-thread performance currently available, but M1 Firestorm cores are close behind, and so is AMD Zen 3. Performance varies by benchmark, and each of them gets some wins depending on the application. See here for SPEC2006 single-thread totals: https://www.anandtech.com/bench/CPU-2020/2797
Compared to the M1, Intel is still restricted by the existing memory and I/O interfaces and by x86 compatibility, which means they can’t do certain things the M1 did, like the integrated unified memory (for massive memory bandwidth) and the relaxed memory model. And as the siblings explain, the P/E design is more about power efficiency than about performance, although the increased power efficiency does allow for more cores at the high end.
Not really, the M1 has a relaxed memory model, the 11th gen intel core series has a strict memory model. Generally the relaxed memory model allows better bandwidth and performance out of the same memory system.
Not to mention there's the M1 pro, max, and ultra if you need more memory bandwidth.
I feel that P and E split requires putting an asterisk next to the core count. it is dishonest to sell a CPU with N fully featured cores and M gimped cores as a CPU with N+M cores
Even worse (to me) is the P-cores have two way hyperthreading enabled so Intel says the CPU supports N(2)+M threads. While Intel has always behaved as if their hyperthreading meant a chip had N(2) core of performance, in the real world it's was more like N(1.5) at best.
So a CPU with 20 threads has maybe eight running at full speed of a P-core and the other twelve pretty gimped comparatively.
I wish there was at least a switch to allow the CPU to report the proper CPUID on each core and ideally, also keep AVX512 enabled. You could have a bios setting writing to an MSR. But I guess some idiot gamer somewhere might still enable this setting without understanding a word of the warning it's accompanied by and Intel wants to avoid that.
But I'm hopeful this is a stop-gap solution and we'll just make the p-cores more power efficient under medium load, and merely have the OS clock them lower when something is determined a background task. Then we "only" have to get that part to become smart enough.
Yes, an attempt at sarcastically describing Intels train of thought was made. But really, why would a child change random bios settings anyway? You can easily render your system unbootable by changing random things there. Better make knifes illegal before an unsupervised toddler hurts themselves?
I very much don't understand why Intel didn't disable AVX-512 on the P cores, unless and until the OS writes to a new MSR that means "I understand that the P cores can do AVX-512, while the E cores cannot", and then enable AVX-512 on the P cores.
Old OS versions continue to work fine, and newer OS versions can opt into the new world and benefit.
My guess would be that this is because of thread migration.
(After reading the TFA: that's what Agner says right there in the 4th paragraph.)
(After re-reading the comment: I guess that the OS changes would need to be extensive with little to no benefit: running AVX2 on all cores will likely be faster than running 2 P cores with AVX512. The only thing that is really affected is the code that could use AVX512_FP16, but I doubt there's a lot of it outside of Intel.)
> I guess that the OS changes would need to be extensive
I don't think that is true. In the simplest case, you could modify the #UD handler to notice when the fault is caused by an AVX512 instruction running on an E-core, and then simply and pin the process to the P-cores, migrate the process, and continue. All existing scheduler functionality.
> The only thing that is really affected is the code that could use AVX512_FP16, but I doubt there's a lot of it outside of Intel.
AVX512 is a lot more than just extending the vector width, and that extended functionality can be very useful for quickly emulating other CPU's vector instruction sets.
They have, haven't they? And officially it was never supported. The article even mentions it: some BIOSs had the ability to override it, but it sounds like that's been disabled in the microcode?
I understand even less why Intel has been creating numerous AVX extensions (1,2 and 512) instead of focusing on a single programmable variable vector length extension that would not require the addition of new opcodes every time the CPU supported maximum vector length increased.
It has been clear for a long while that the vector length was going to gradually increase every few years to support new data processing requirements, so the writing has been on the wall all along.
Variable-width is the new hotness but it doesn’t work well on all tasks. If your task is fixed-width, or the algorithm changes based on task width, you kind of need to know the hardware width. And shuffling, perjuring, swizzling, and bit-shifting don’t translate very well to these approaches. Nor is there really an easy way to operate these in a fixed-width mode.
It’s not all bad but the people who just say “just treat it like an n-element array and let the hardware handle it bro!” are hand waving an enormous amount of algorithms that may not actually be translatable to that.
And broadly speaking, having a couple code-paths and choosing at runtime is not that bad. HPC is used to working “close to the metal” and tuning their code to run optimally. You can’t make that complexity go away - things will always run better on X hardware but Y hardware runs that code path way worse. In this domain all you do with clever auto-programming magic is obscure the problem - you are now writing for the optimizer instead of writing for the hardware, but different hardware still runs differently and you need multiple code paths to hit that optimally. So now you have two problems - keeping the hardware running optimally and keeping the optimizer from messing you up.
Since you are now dispatching large operations that will not complete atomically in one cycle (they simply won’t, you can’t do 2M elements on a 128-bit vector) it doesn’t seem likely that we will stay strictly in-order on these, and the next step is you start extracting parallelism from the stream and that’s the problem that immediately emerges from that. Now you have an optimizer running a SIMT program (opmasks are basically SIMT) extracted from the instruction stream, and you need to not just write code that runs fast - you can’t - you need to write code that the optimizer turns into code that runs fast, and that seems like absolute hell on an architecture where you can’t even know the number of registers-per-thread in advance. Maybe I am reading this all wrong but that’s where you’d really end up going with a “run this op on an N element array” model. If you run one vector-instruction at a time you are in cache/register hell constantly spilling to memory and back, so you want to optimize that to keep things in-register/in-cache, and that means extracting parallelism from the stream to run more stuff while it’s still somewhere reasonably hot.
This is the exact garden-path that GPU drivers went down and it was a huge mistake with OpenGL/DX9/DX11 that had to be walked back with the DX12/Vulkan APIs to get back closer to the metal, because the optimizer stage became completely inscrutable and managing it to keep it from doing something stupid became impossible.
There is a level of irreducible complexity in HPC and exposing the hardware is the best way to avoid it. Leaky abstractions make things worse.
Writing for a phone is not HPC but if you care about energy efficiency you still need to make sure you’re running reasonably optimally and not wasting cycles, especially with a super wide vector unit. That means you’re leaning heavily on the “sensible” code path having decent efficiency, which means you’re leaning on the hardware to behave sanely. So it’s the same thing.
Again, not saying it’s a completely bad idea, but you can already see the abstraction starting to leak with the various types of operations that don’t really work on a variable-width hardware concept, and that’s worrying. There is a lot more opportunity for this to go bad than I think people are at first glance.
Hardly new. The VAX 6000 Series
Vector Processor supported variable length vectors with up to 64 vector elements:
With the VAX, the vector register has a maximum length of 64 elements. Each element can contain up to 64 bits. The elements used can be enabled or disabled by setting bits in a Vector Mask Register (VMR). The programmer usually determines the range, or limits the number of elements used through a Vector Length Register (VLR) (see Figure 1–2). This range can vary, for example, from 0 to 64 elements. Of course, if the vector length = 0, no vector elements will be processed.
I am pretty sure the concept predates the VAX days and might go as far back as the CDC STAR-100 from 1974 and potentially also the Cray-2, but I have had a trouble locating authoritative sources on the details of the ISA of the former.
Variable length vectors have been available much earlier than any VAX.
The most notorious vector computer is Cray-1 from 1976, and it already had variable length vector operations using a vector length register (and also 64-element vector registers).
All later vector computers and also both the vector extension of the RISC-V ISA and the SVE extension of ARM have been strongly inspired by the ISA of Cray-1.
CDC STAR-100 introduced a few influential ideas, but it had a low performance because it did not have vector registers (the operands of its vector instructions were arrays stored in the main memory).
Intel is in general heavily constrained by whatever Microsoft chooses to do with Windows. It's not like Apple where they can introduce a new hw feature and then immediately use it in the operating system. For example the M1 Macs have the big little architecture integrated with Grand Central Dispatch. Many, many interesting Hardware features have died in the Intel world because the Windows team simply did not add good enough support for it and there was nothing Intel could do.
In this case detecting DRM software and scheduling it differently would require sophisticated support in the operating system, or even ideally DRM support implemented by the operating system. However these things are probably beyond Microsoft's capabilities in its current state.
Well it would be nice if Microsoft found a way to implement them and coordinate with Intel, with backwards compatibility of course (which is the key point of Windows).
Then this brings another question to my mind: if Intel is so contrained by Microsoft's decisions (which make sense) they should have been in a good relationship designing things that work nice together. Why did they design their CPU in a way that would break in Windows usage in the first place?
They must have much more R&D than saying "let's design this hybrid CPU and put a new instruction set on some cores and see if it works on Windows and DRM software else we disable those features" kind of approach.
Other than obvious stuff like differences in features (AVX512 - although I thought that wasn't even officially supported at one point even though they haven't disabled it?) between the P and E cores, how does ARM manage it with big/LITTLE cores? Surely that's similar to a degree (Out of order P cores, in order E cores) to what Intel are attempting?
Are Intel just taking it too far in terms of disparity between the architectures?
On an android device I worked on some 5 years ago (so might be out of date) it used a simple cpu mask.
Background tasks always used only the little cores, foreground apps always always used only the big cores. I do not believe it was possible for a single app to run threads on both big and little cores.
And my understanding is that every arm big.LITTLE combination supported exactly the same ISA and were intentionally 100% binary compatible. For example, you couldn't pair arm A53 little cores (that support only armv8-a) with A75 big cores (which support armv8.2-a)
With some software work, you could pair differing cores. You'd just deliver a signal to a process saying 'youre about to be moved to a different core type, please recheck the capabilities'.
Then, you'd have a compatibility shim that emulates unimplemented instructions using traps for software that hadn't yet been rewritten to understand the above signal.
Personally I don't really see a point in having a hybrid architecture like this unless it'll lead to massively increased core counts, and so far it doesn't look like it does. AMD still beats Intel in both core counts and power efficiency, and they're only using P cores so... what's the point of having E cores?
But maybe that's just because it's a first generation technology for Intel? I hope things improve in the future because I sure would love to have more cores available (currently rocking a 32 core Threadripper) and AMD is sure as hell not interested in actually pushing HEDT forward since they became the top dog. (I have a few grand burning a hole in my pocket waiting for a product they don't want to release.)
I see nothing particularly hard in utilising 32 cores. When I launch all the services in the project I’m working on my os has plenty work to schedule. Right now I have 3732 threads running on my machine. It’s totally separate question if OS can schedule so many threads effectively on so many cores, and how much coordination overhead there will be. But that’s just something we have to figure out and fix.
Anyone working on even a medium sized C++ project needs a very high end machine in order to compile. For large projects, 32 core work stations aren't crazy at all. Try building something like Firefox or Chromium on a quad core and you will see what I mean. The cost of the workstation is tiny compared to the cost of the engineer and you really don't want them to be waiting around for half their day (quite literally) because of a miserly equipment budget.
I talked about linux recompiles because sometimes you do need the cores; not arguing. But that's the long tail and many devs don't. I'm not against blowing cash if needed, but really it's about asking if there are more efficient ways. I guess we agree.
Compilation is largely memory bandwidth limited - some of the acceleration from cores might be that they have different memory channels rather than more CPUs.
M1 builds a lot faster than Intel CPUs even with fewer cores because of the memory bandwidth.
It's pretty easy to eat up cores when you're developing graphics rendering algorithms, which can be far more convenient and ergonomic to do on the CPU in a general purpose compiled language, and scaling out via plain old threads.
A CPU with so many general purpose high-perf cores that it manages to look like a GPU in terms of execution units count is a beautiful thing.
The pure thread count is not that interesting. Running windows with pretty much just firefox open shows me about 5000 active threads, and and recent hardware shouldn't have a lot of problems with a 5digit number. Heavily loaded threads that actually require CPU power to execute are however definitely a different story.
It depends very much what you're doing of course (compiling linux kernel repeatedly perhaps) but if you have a little you can get a heck of a lot out of it with experience, although you surely will hit a wall sometime. But 32 cores not enough is just unimaginable for me.
Can't speak for the guy you're replying to, but I regularly saturate my 16-core CPU as a web developer. I often work on environments that need 20+ services locally virtualized to properly simulate the environment they'll be used in.
Worked in a place where the local architecture astronaut had taken over and we were subject to his insane designs that were virtualisation and microservices everywhere. A monolithic lump on a server would have been better, cheaper, faster in every way and needed many fewer cores.
But your case, ok, understood.
BTW The biggest slowdown IME is not from lack of cores but lack of RAM. Odd that no-one's mentioned that. Or using a server with larger numbers of memory channels to keep their cores fed. I/O > cycles more often than not.
So in $dayjob I have used an 8-core (16-thread) CPU to 100% regularly, as well as fully utilising 64-GB of RAM. I would be able to use 16 cores, 32, or more.
Random examples:
Trawling through 70 GB of web server log files going back years with ad-hoc analytics. I tried ingesting this into a cloud service but it was taking forever, and cost a non-trivial amount of money. It was also dog slow compared to just doing it locally with SQL Server Clustered Columnstore.
Compiling Chromium. I'm not even a browser developer or anything, but there was a "global" change that I had to preview ahead of time, so I made a custom Chrome build that had the change embedded in it. I also made a matching build without the change to enable A/B comparison testing.
Server base image task sequence development. You would think a server is ideal for this, but no! It's very I/O heavy, and doing it locally in Hyper-V using an NVMe drive can make this 4-5x as productive as any shared platform with throttled disks. There are zero cloud services that can even approach a Samsung Pro 980 SSD for reasonable costs.
Multi-server lab tests. Compared to cloud services, it's faster and easier to build a local pool of machines for testing how they interact. With 64 GB of memory, I can (and have) easily run 20+ machines. Compared to cloud platforms, I can do things like thin-cloning to build (and then delete and rebuild) these nearly instantly -- about 5 seconds or less in some cases. Even the relatively heavyweight Windows Server 2022 VMs boots in just 1.4 seconds! Some build jobs require nearly 100 GB of content to be deployed, and the upload to the cloud is a problem in some customer environments. E.g.: due to security restrictions. Doing it locally is easier and faster.
Photography and video editing. I do both as a hobby. Lightroom will cheerfully use whatever number of CPUs (and GPUs!) you have to process images. My recent hardware upgrade made a very noticeable difference to the workflow. Similarly, photogrammetry and other advanced 3D modelling workflows can use up virtually any amount of compute.
Password cracking. I occasionally need to reverse a password hash for work, typically to fix lost access to some ancient piece of network gear. Sometimes it's for verifying password strength where the plain-text password is not known. I could distribute this across a pool of cloud VMs, but my lab subscriptions have hard limits and this is sensitive data.
Automated searches through code for a breaking change. E.g.: scripting the checkout, build, test, etc... in a loop. (Git bisect doesn't solve every problem)
VS Code now has ripgrep integrated, so "find in files" will use every CPU. I have about 100 GB of files on my laptop from various old projects, so this is a very helpful feature for digging up some code snippet that I only vaguely remember writing. It can find a match for a regular expression at 700 MB/s using 16 threads at 100%.
Defragmenting moderately large (50-500 GB) databases. I have some customers that have built up horrendously fragmented databases over decades, but can't afford the IOPS and/or downtime to fully rebuild all of the tables and indexes. I recently utilised the aforementioned Samsung SSD to defragment an 80 GB database in just minutes, that would have taken days or weeks on the production server. Again, transferring this out to a cloud is a no-go, and the problem was that typical server storage is much slower than a typical laptop. Locally I could get 200K IOPS sustained. SQL Server Developer Edition is the same as Enterprise, and will utilise all cores to rebuild indexes in parallel.
Etc, etc...
If you know what you are doing, you can make the CPUs go "brr...".
Cloud stuff is never going to make sense, local machine it is. I do see what you're saying and here it seems like a good fit for a massive machine. No argument here.
NB. "Trawling through 70 GB of web server log files" unless you partition your search you're going to be doing a lot of IO due to having only 64-GB of RAM. Chucking in extra mem to keep it all ram-resident will beat cores any day.
The query engine in Microsoft SQL Server is absurdly advanced, and simply beats the pants off just about everything else for complex, general-case queries. There are other engines that are better tuned for special cases, but if you've got "stuff" that's within the capacity of a single box, it's really hard to beat MSSQL.
Let's expand on my example above: Clustered ColumnStore.[1]
It's basically PowerBI or Vertica embedded within the traditional MSSQL engine. This extension enables the ability to have a per-column (vertical) on-disk storage format instead of the traditional row-store format most databases have.
You can have it as a secondary acceleration index, or as a clustered (primary) index. In the latter case, it'll compress your raw table data. For my log files it was 33-to-1 compression (3% of the original file size).
Second, even if the data doesn't fit into memory, it'll utilise the columnar storage to simply ignore (skip) columns it doesn't need for the query. I was computing performance statistics, so it was just reading in some numbers instead of the text. A further 100-to-1 optimisation can be gained here.
It then parallelises this across all cores/hyperthreads, so I get 100% utilisation on 16 cores.
All of this I/O is fully asynchronous with a huge queue depth. Expect to see 200K IOPS on a laptop, easily.
Once in memory, it uses Intel AVX SIMD instructions to chew through the data 8 rows per clock tick per core. With 8 cores going at once, this is 64 rows per clock, or up to 234 billion rows processed per second.
What's neat about it is that it's fully general: Columnstore can be mixed -- in a single query -- with in-memory tables, on-disk tables, traditional row-store, and external tables streamed in over the network. The engine will just... "figure this out". It'll build indexes on the fly if it has to, ideally in memory, but failing that... it'll stream disk-to-disk as required. I've seen "sort spills" reach 600K IOPS on my laptop.
If I have to scan through years of logs and be able to join it against additional tables for "enriching" it, then it's hard to beat. Sure, there's a scale where the are dedicated log analytics databases, but they have their quirks, limitations, and costs.
I take your incredulity that others can fully utilise their computers as a sign that general knowledge of just what is possible is poor. People have pointed out that some simple shell scripts can outperform Hadoop clusters up to a surprising scale. Similarly, knowledge of the capabilities of traditional database engines like MSSQL is oddly lacking, even amongst developers. Not to mention more esoteric ways of making your computer do your bidding, such as Mathematica, Julia, GPU codes, or whatever.
Your computer is a power tool for your squishy, manual-labour meat brain! A lever for the mind. Learn to utilise it better and you'll be better at thinking.
I've not used columnstore in MSSQL but in fact I likely know more about the innards of MSSQL than you. WRT memory, all I mant was that if your working set doesn't fit in RAM, you're screwed regardless. Nothing will save you, so check that first. I don't know what your 70GB means in this context - please check.
> It then parallelises this across all cores/hyperthreads, so I get 100% utilisation on 16 cores.
that suggests your working set is indeed smaller than ram. Good.
However be aware that the parallelise feature of mssql is much better that it was but is still moderately crap. Disable it and see if you get a 16X slowdown. My guess is you won't, which means when enabled you won't be getting a 16X speedup (also hyperthreading can actually slow things down - disbale it and see what happens). Edit: I'd not be surprised if you were only getting 4X to 5X speedup (I may be wrong but I do have experience in this).
> With 8 cores going at once, this is 64 rows per clock, or up to 234 billion rows processed per second.
(giggle) Oh, I don't think so!
> I've seen "sort spills" reach 600K IOPS on my laptop.
Then you're hitting IO and yer cores are useless. It spills to disk because it's overflowed ram - you may have a problem.
> Learn to utilise it better and you'll be better at thinking.
... agreed, but the reverse is true also.
edit 2: Depending on what you're looking for, and if you do it often, check if MSSQL's free-text indexing can index it, if so you may get a far bigger speedup from that than anything else. Maybe. Depending.
I do know a thing or two about MS SQL optimisation also. 8)
So keep in mind that the SSD I'm referring to can read at 7 GB/s. Not seven gigabits... 7 gigabytes per second. I've benchmarked some not-so-old servers (the entire box, not a VM!) as capable of only 35 GB/s streaming reads from main memory! So these days "disk" is more like "5x slower memory". It used to be 100-1000x slower.
I told one customer to get a 16x PCIe gen 4 adapter card with 4 NVMe M.2 ports on it. This would -- in theory -- be able to pump out 28 GB/s of reads, equivalent to a 200 Gbps NIC. Total storage would be 32 TB with CoTS 8TB drives.
That's a mind-blowing amount of IO in a single box!
The hardware has the capability, it's the software and inability to utilise that is the bottleneck now..
SSDs will have higher latency than ram, and I seriously wonder if they actually can hit 7GB/sec, or only theoretical. Ram is faster, always, in latency and bandwidth.
> I've benchmarked some not-so-old servers (the entire box, not a VM!) as capable of only 35 GB/s streaming reads from main memory!
Then you have either a low-end server or something is very wrong (I think you benchmarked wrong, see below). You should be getting maybe 15 GB/sec per channel, so 4 channel = 60GB/sec, for a decent server you may well have 6 or 8 channels. (figures may be a bit off but 35 is puzzling me).
You may have measured on a single core, but each core has its own mem reading stuff so it should multiply up if your benchmark uses extra cores.
Anyway, I have to get back to work on my dual-core haswell. Good luck!
Whenever I read stuff like this my inner optimiser starts wondering where the CPU is going and why - massive GC cycles, inefficient regexp stuff, poor algo choice, bad cache locality, etc etc.
At the risk of ill manners but i must ask, have you looked at all this because I've got in some cases orders of magnitude speedup by popping the hood and instrumenting.
ai. Our models train in a reasonable time (or we push them onto gpu) but doing the pipelines and etl when developing is a big time sink.
We could sit there and optimize the approach, but the time that would take and the complexity added, it would be cheaper to just buy more powerful computers.
For example, on reasonably new 8 core 64gb ram, 7gbs HDD machine, I optimized a calculation for one of the features down from 2-3 days to about 15 minutes. Throwing cores or ram or disk at this algo before would’ve got nowhere.
Now htop glows like a Christmas tree when it’s running.
This was costly to do but it was the only way.
We could talk about python being slow and blah blah blah use C but once again, whenever we do tricky stuff with code that then gets shared amongst a bunch of python devs I need to think about the holistic cost including maintenance, ease of changes, bug fixing, resource that actually know c, client appetite, what the production load is going to look like etc
And finally, at some point a client will just come along with another million rows of data
> wtf do you need more cores for??? Wat are you doing?
I have a bunch of embarrassingly parallel workloads which I have to run regularly (text analysis, machine learning which is not easily accelerated on a GPU, etc.).
> consider buying a server.
I looked into it; not really an option. Those are vastly more expensive, really loud (I need a quiet workstation), have significantly lower boost clocks (I also need single-thread performance), and are either really hard or even impossible to actually buy in my country through normal retail channels.
Alas you can't do that for every workload, and even for those where you can that brings extra complexity. On a single machine I can just replace an `.iter()` into a `.par_iter()` in my Rust programs and boom, I parallelized my workload across every core with only a single line change. It's not that simple with multiple machines. (I might do that one day, but I'm not there yet.)
Gotcha. But FYI I wrote a work farmer years ago and it wasn't difficult. You just manually started a program over there and started the farmer over here and that was all. It could use every machine in the office. These days there may be better solutions than hand-rolling it, never used it but <https://en.wikipedia.org/wiki/GNU_parallel> might work. Also see if beowulf software has anything to make it easier. Good luck!
> On a single machine I can just replace an `.iter()` into a `.par_iter()` in my Rust programs and boom, I parallelized my workload across every core with only a single line change.
If you’re using Rayon for anything you expect to benefit from a lot of cores, my experiences have been very negative in the past. Rayon is often extremely inefficient once you allow it to use more than a handful of cores.[0]
If your unit of work is absolutely massive, you might not be as affected by Rayon’s scheduler, but it makes me sad that this issue still hasn’t been resolved. My comment on that thread (documenting just one instance where I ran into this issue) was almost 4 years ago! And the issue itself is almost a year older than that. Sure, I could try to contribute a general solution, but it’s just easier to stop using Rayon.
I no longer consider Rayon to be an advantage for Rust. There are plenty of good ways to wire up a parallel work pool, but they do require more than a one line change.
AMD beats them because they are a process node ahead. Dunno why every comment implies there's some design gap that leads to M1 or AMD being more powerful or efficient.
It's 90% the process node. There is no magic design sauce that makes those chips better.
The big/little design change did seem to improve performance quite a bit for Intel, even using the same manufacturing node as before
I don’t completely agree. A MacBook Air M1 is 25% faster than a 10th generation Intel 10700K, running the same code full of floating point instructions and matrices. It’s not a simple matter of process node.
The same MacBook Air can do hour long Skype video calls with ~9% charge consumption, with no discernible heat difference on the body too.
I'm not telling that process difference isn't the part of the picture, but the process difference is at most half of the story.
Working on HPC field and with generations and generations of Intel hardware, I can tell you that two node difference (alone) never gives you 25% real world performance in the realm of Intel, let alone theoretical.
Even a single node difference can easily come close to a 25% increase in performance: "N5 technology provides about 20% faster speed than N7 technology or about 40% power reduction." [1]
Real world performance pretty much always lines up with these estimates. It's the main reason why we have exponential growth in computing power in the first place.
It would be a more fair comparison to compare the M1 to AMD's ZEN 4, which should be coming out next quarter.
I'm all for competition to be honest. While Intel is the least favorite CPU maker of mine (due to their unethical moves in the past), I'd love to see fierce competition rather than a single dominant power.
We've seen how the latter went until AMD came back to the game.
> I don’t completely agree. A MacBook Air M1 is 25% faster than a 10th generation Intel 10700K
Why is everyone insistent on comparing Apple’s latest CPUs to Intel’s parts from multiple generations back?
Intel is on 12xxx series now. AMD also has new generation laptop CPUs that perform very well. Let’s compare to those.
The M1 is power efficient but much of that is due to the TSMC process they used. I really don’t understand this desire to make arbitrary comparisons to old Intel CPUs.
Because that’s what I had access when I did the tests. I’ll do a series of tests on our cluster and post these.
Will these be meaningful for you?
The thing is, M1 on air is a relatively low clock speed CPU with passive cooling. 10200K is actively cooled, unlimited beast of its generation. The libraries I use are X86 optimized. Yet the code I’ve written is 25% faster on ARM.
I'm not saying it isn't, but the "low clock speed" part is often brought up in a discussion like this, implying that Apple could make a M1KFS running at much higher clocks.
> 10200K is actively cooled, unlimited beast of its generation
Curious what you meant here since that part doesnt exist, and if it did it would be near the bottom of the stack for the generation (101xx and 103xx are the low core count i3 and i5 parts).
Written that comment on mobile on a plane, before take-off, so I either typo'd and didn't see, or auto-correct changed it deeming 10200K is more correct than 10700K.
If you look to the original comment where I sparked the discussion, I've mentioned it as 10700K already.
> The same MacBook Air can do hour long Skype video calls with ~9% charge consumption, with no discernible heat difference on the body too.
> M1 is a different beast.
The M1 has other advantages which are decidedly not rooted in the hardware: software. Apple seems to do profoundly better than Windows and Linux at applications actually using hardware accelerators. This makes a gigantic difference in performance and power efficiency.
On a Windows or Linux system it is not at all a given that even very easily hardware-accelerated tasks, like video decoding, actually use the hardware that's there.
They are not. If you check out Wikipedia, Intel 7 (or for that matter Intel 10nm) is very comparable to TSMC 7nm in parameters (transistor density etc.).
There is a massive design gap between all Apple designed chips and non Apple designed chips. Apple’s chips are ridiculously humongous. They’re built for extremely wide, high performance applications for the past 10 years. A single A14 core can do 64 32-bit floating point operations in a single clock cycle. AMD/Intel were long at 16 operations and only the latest Intel went up to 32 operations.
Apple optimises for performance per watt. That the biggest Mac Studio is almost silent when fully loaded is remarkable when compared to my Intel desktops that start screaming the moment I start video conferencing.
The Intel parts that perform similarly to their Apple counterparts need at least twice as much power (and cooling). There’s no magic in either.
I find the different balances of performance and efficiency cores interesting. I’d love to have a lot more Icestorm cores on a Mac to play with background tasks that don’t need the firestorm ones.
On the Intel side, I’m also disappointed by not having the option to implement the missing instructions in software.
apple's performance per watt is a remarkable accomplishment. but the second half of your comment has more to do with the anemic thermal design of windows OEMs than anything else. my intel desktop (9700k with 5ghz all-core OC) is also barely audible while running anything less intense than prime95. it just takes a large air cooler with a sensible fan curve. dell could easily do better with a custom case if they actually cared.
But even being one behind, it outperforms M1, with lower efficiency. Thus reasonable to speculate that a process node advancement would at least put Intel's chip on par with M1 on both perf and efficiency
> Thus reasonable to speculate that a process node advancement would at least put Intel's chip on par with M1 on both perf and efficiency
We'll see, but I highly doubt that Intel-at-equivalent-of-TSMC-5nm will be performance/watt competitive with the M1. Apple gets that power efficiency via an extremely wide-and-deep design that intentionally trades off a surprisingly low max clock ceiling (3.2GHz) for power efficiency while maintaining an absurdly high IPC (something like 75% higher than Intel's IPC at only one process node smaller) that makes it much more competitive on performance than conventional industry wisdom ever expected (there is and has been for years a strong belief that a design as wide as Apple's would be grossly inefficient because you could not possibly extract enough instruction level parallelism to keep so many decoders/execution units saturated -- hence we've spent over a decade with 4-5 wide frontends being the "practical maximum" that allows them to still target ~5GHz turbo speeds, which was believed to be basically the best/only path to high performance within what was believed to be the limit of "achievable" IPC. The M1 blows these fundamental assumptions out of the water).
So Intel makes conventional design decisions that make up the performance of their lower clock-for-clock IPC by turboing well north of 5GHz, at the cost of consuming something like 7x the max power of the M1. A single node shrink isn't going to buy them a 75% increase in IPC (their last 4 or 5 node shrinks combined haven't bought them that much IPC increase), so Intel@equivalent-of-TSMC-5nm still going to have to burn power like crazy to stay competitive on performance, and that's going to cost them efficiency that the node shrink just can't paper over.
It will lead in absolute perf, but its not going to be nearly as efficient.
Some people care only about maximum absolute performance, others care for performance per Watt. I find the latter much more sensible. What performance Apple can reach in, say, 50W, Intel reaches in 150W.
I have a list of instructions per clock from SPECINT results which I compiled from Web sources. (I didn’t verify the results myself.) I added in rumored future processors with their highest rumored performances:
Cortex-A73 (ARMv8-A) 64.4
Pentium M 69.0
Bulldozer (AMD FX) 76.2
K10 (Phenom II) 76.7
Goldmont Plus (Atom 3rd gen+) 77.5
Core 80.0
Bobcat 81.3
Nehalem 90.0
Cell (PS3) 93.5
Jaguar (PS4) 93.5
Sandy Bridge 100.0
Tremont (Atom 4th gen) 100.7
Ivy Bridge 105.8
Cortex-A76 (ARMv8.2-A) 114.7
Zen 115.9
Haswell 117.6
Zen+ 119.3
Gracemont (Atom 5th gen) 120.8
Broadwell 121.5
Skylake 124.0
Zen 2 139.8
A10 (ARMv8.1-A) 139.8
Cortex-A77 (ARMv8.2-A) 141.0
Sunny Cove Ice Lake 146.3
Cypress Cove Rocket Lake 147.6
Zen 3 167.4
Golden Cove Alder Lake 175.6
A11 (ARMv8.2-A) 186.0
Cortex-X2 (ARMv9-A) 190.2
Rumored Golden Cove+ Raptor Lake (’22) 193
Rumored Zen 4 (’22) 207
A12 (ARMv8.3-A) 213.8
Rumored Ocean Cove (’23) 223
A13 (ARMv8.4-A) 235.0
A14/M1 (ARMv8.5-A) 236.7
> AMD/Intel were long at 16 operations and only the latest Intel went up to 32 operations.
Intel has had 256-bit dual-issue FMA since Haswell (2013); that's 32 flops/cycle/core. AVX-512 doubled the vector length for Xeon Phi and was generally available in 2017 Skylake.
not really. for whatever reason, ryzen cpus have very little headroom for overclocking. no matter how much you increase the voltage, you're not gonna get much more performance. intel has increasingly pushed the limits of their designs over the last few generations, but you can usually get more performance with aggressive voltage settings and beefy cooling.
"Dunno why every comment implies there's some design gap that leads to M1 or AMD being more powerful or efficient."
That is because there is a huge design gap. Intel architecture and design teams have always assumed on the inevitability of their own manufacturing superiority and process advantage and thus never had to really consider where to place transistors for the most benefit, or think really hard on better ways to improve single-thread performance instead of just turning a knob that was invented two decades ago.
In any case, your statement is just plain wrong. Dennard scaling has been dead for a decade now. Planar area decreases with new processes come with increased power density, i.e. new process nodes reduce die sizes but the transistor performance does not improve the way they used to.
source: worked at Intel for CPU design for a decade, then worked on the same things elsewhere using TSMC process
LOL yeah. More powerful, i.e. it consumes more power. Fun fact: the M1 big core caps itself around the same wattage where most Intel SKU consider the core to be below stock wattage. As in, in order to achieve your “more powerful” claim, it takes Intel cores 2-7x the watts, depending on the workload, to eek out a marginal performance uplift… or sometimes just to match the M1.
Nobody in CPU design cares about unconstrained power performance. Even at Intel, they never use the PL2 power level as the design landing zone. All modeling is done at much lower power target, even below PL1.
That is to say: unconstrained power performance is not an engineering effort, it is a marketing and SKU differentiation effort. Your “more powerful” statement is a matter of how much power Intel/Apple/AMD/whoever is willing to shove into their piece of silicon, which in turn dictates the form factors and customers they can try to satisfy.
Apple only needs a few watts per big core and still get the peak perf it needs, and put that chip in all the form factors it sells. Obviously if M1 sucked down the kind of power as any Intel chip, it cannot go into an iPad, or a fanless macbook air, without totally compromising performance.
Intel on the other hand needs grossly more watts to hit that same performance, and therefore is losing badly on form factors that care about power: laptops and servers. Some form factors/customers just don’t care, which is why Intel cooks up marketing targeting the gamer market, or the 8-pound laptop market. That’s all fine, just don’t pretend Intel is not getting absolutely hammered on the CPU technical specs for the money-making markets that matter.
BTW, the DIY gamer CPU market does not matter. The 8-pound laptop market does not matter.
Ok, so when Intel and AMD get on 5nm equivalents and their chips are both more powerful and efficient than the 5nm Apple chips, you'll issue a mea culpa here, right?
The fanboyism runs so deep it's amazing. Let's compare PS5 to GameCube next and talk about Sony's great engineering advantage
And when it doesn’t, will you apologize for calling everyone else a fanboy because they don’t buy into your excuses?
Go compare the perf/power curves of the alder-lake cores (both P/E) versus the last-gen Apple cores from TSMC n7+. TSMC n7+ is roughly comparable to Intel 10nm ESF in transistor performance, so it is a fair comparison.
Do your research then get back to me about how the process is entirely responsible for every perf/power difference and we (the silicon design teams) are just identical robots doing the same thing across the board in every company.
I do like how you claim there is no design magic sauce, but then Apple will lose big when Intel/AMD come out with their 5nm equivalents, as in, they found some magic sauce? WTF?
It also doesn't help that their design pipeline and their process pipeline were tightly linked. When Intel's "10 nm" process pipeline stalled, their design pipeline did as well. Alder Lake/Golden Cove is (or seems) to be the first really new design since Skylake, and it uses a lot of watts to compete. We'll see if they get their grove back with the next couple releases, I guess.
It's quite possible AMD has plans for this type of thing as well: despite their cores being more energy efficient and powerful per watt than Intel, I don't think they can match Apple yet (some of that is due to 5 nm process though).
A lot of Apple’s advantage here comes from the regular instruction size, the relaxed memory semantics, and how those allow efficient instruction reordering.
Not saying x86 can’t ever replicate that, but the ISA makes it harder.
For workstations and servers there's still hundreds of low priority housekeeping threads that can run on efficiency cores. Using them means the work gets done for less power and thus less heat. While a server isn't worried about battery life every saved bit of power is saved dollars in aggregate in a data center.
> Using them means the work gets done for less power and thus less heat.
It also avoids scheduling conflict between housekeeping / background tasks and "user" stuff, which means more predictable and stable throughput and better cache utilisation (your cache doesn't get blown because the system needed some time to zero memory or update something).
The top Alder Lake desktop CPU has 16-cores, 8P+8E, compared to 8P in the 11-th gen. The E cores get around half the performance of the P cores, so the 8P+8E is get roughly the performance of a hypothetical 12P+0E CPU. The very high core parts are reserved for Xeon (a 56 core Alder Lake Xeon has been leaked, but I think the P/E split is still unknown).
What people need to realize is that a P cores takes around 4 times the die area/transistors of an E core, and use probably 3 to 4 times the power of an E core, but performance is only around double that of an E core. So with the same area, you get of the 8P+8E you can get 10P cores, not 12, and the question is if a 10P (rather than a 16P or even 12P) core Alder Lake is better than a 8P+8E. (Btw, these ratios are pretty similar as those for the Firestorm/Icestorm cores in Apple's M1.)
If your workload is very parallelizable you get more bang from your silicon and Watts with E cores than with P cores. If you have only embarrassingly paralleled workloads, you're probably better off with 32E cores than with a 8P cores.
It’s also not very intensive, especially since video decode usually runs on the GPU.
(Or it should - browsers are not always very good about this but the tools are there, they just don’t want to pay licensing fees to use them.)
I always cringe a bit when people insist they need 6 cores “so they can have discord and Spotify open while they game”. People don’t generally have a very good intuitive sense of just how little cpu time it takes to run those (a lot of times the “at the desktop” measurements are not even with the processor clocked up, it’s running in the lowest active power state), and there is almost always a little cpu headroom due to non-perfect game scaling. Context switching is cheap, your OS is running thousands of threads at any given time, but people don’t have a good intuitive sense of the overhead of context switching either.
Playing a video and writing a document is something that can be trivially done on a core2duo era processor, or the modern equivalent of a low-end dual-core nuc with e-cores (atom cores) like a J4005. You don’t even need performance cores for that.
The tradeoff Intel is making is that some tasks need performance cores because they can’t really be parallelized all that well, and embarrassingly parallel stuff is more efficient to implement with lots of smaller cores. In principle this is also more power efficient, like big.LITTLE in phones, but Intel actually seems to have fairly poor power scaling - they are space efficient but not all that power efficient. This is also partially due to the fact that the little cores share a core voltage with the big cores - the big cores need higher voltage to clock up and that kills efficiency on the little cores. At that point you may as well clock the little cores up too since they have to share the voltage anyway, and that’s an inefficient way to run them.
As Agner Fog mentions, the AVX-512 is also present in silicon yet disabled, which hugely blows up core size for almost no benefit (it’s dark silicon which does have some minor thermal benefits). Alder Lake is in many ways not that well-designed at a high level, but it is magnificently fast on those performance cores and the e-cores would be very nice if they weren’t conjoined awkwardly to the p-cores. Gracemont-based server and atom chips are going to be nifty when they finally arrive.
Apple is still way ahead with Icestorm though - those cores are absolutely absurdly tiny for the performance they give. They are 0.5mm2 each iirc, even considering the node shrink that is teeny-tiny, they are half as many transistors as a Zen3 core and almost the same performance (meaning much higher IPC) or something along those lines. Firestorm is neat but Icestorm didn’t get nearly the attention and it is really insanely fast for how few transistors it uses.
> Playing a video and writing a document is something that can be trivially done on a core2duo era processor, or the modern equivalent of a low-end dual-core nuc with e-cores (atom cores) like a J4005. You don’t even need performance cores for that.
Playing a video and editing a document, sure. But change that to being on a video call and having a few moderately intensive web apps open (say Gmail, Google Docs and some kind of project management software) and your average dual core processor from a few years ago is going to start to struggle.
I do some work on my ThinkPad t420s. It's about a decade old now. I understand it's anecdotal but fwiw. Note that it's plugged into Ethernet.
I do ALL my work on a t25 which is just about 5 years old now. This includes daily calls with dozens of video participants (ms teams and WebEx mostly) while I have chrome and Firefox open with multiple tabs including outlook and Gmail and slack, couple of MS words, couple of PowerPoints, Visio, etc. That's a dual core processor from more than couple of years ago and it's nowhere near capacity with that workload.
It does have lots of ram and Ethernet to residential Fibe 1gbps. But CPU just hasn't been an issue and I plan to work on this another 5 years
With my t25? Participant, absolutely! As I said, it's my main daily workstation. Over last two years that included using built-in webcam, a dsrl, and currently Logitech external webcam.
Heck I've run full PeopleSoft multi vm setups on my ancient t420 back in the day. A little CPU really goes a longer way than people realize.
Video encode is also hardware-accelerated these days. J5005 may be Nehalem performance but unlike Nehalem it has a hardware video encoder onboard. Intel has had it for a long time, I think maybe back to Sandy or Ivy days so it’s basically in everything at this point. It’s not archival quality but it’s more than fine for videoconferencing starting from around Haswell. Goldmont actually has a fairly modern media block (newer than skylake - it has full hevc main10 decode and hdmi 2.0b support) that is backported from the Ice Lake architecture during the years Intel stalled out on 10nm.
I’m just using Nehalem and Conroe as a touchstone that people understand in terms of cpu performance. That is what you get in general cpu performance, but it’s also got better media support and crypto acceleration and so on, so it’s more usable today. Actually a closer look at the numbers puts it just below Sandy Bridge i5 actually.
AMD cpu users (non-apu) and RX 6500/6400 owners obviously need to look elsewhere however. NVIDIA’s 1030 chips are ancient pascal based crap but AMD dropping encoder support on a chip designed during the pandemic era of work-from-home is a bit tone-deaf.
(Yes, 6500 and 6400 were designed during the pandemic, the architecture took longer to develop but 6500 wasn’t the lead ship in its architecture, and taping out a new chip configuration takes much less time than designing a whole architecture from scratch… AMD definitely knew it would be used as a desktop chip too. It was a deliberate space-saving move in the face of their wafer shortages, not something they got blindsided by.)
> It’s not archival quality but it’s more than fine for videoconferencing starting from around Haswell.
My 2015 MacBook Pro with Broadwell i5 would indeed work "fine" for videoconferencing (in my case Google Meet), but it would also use most of a CPU core to do it. Which is fine if that's all you're doing. But if I wanted to share my screen and work on something else while doing so then it could quite quickly get to the point where I needed to start shutting down other programs, and even then it could cause lag when typing in less well-implemented apps.
My new MacBook with M1 Pro cpu it still uses most of a cpu core for videoconferencing, but it's a single E core and I have 9 others cores. Videoconferencing now makes zero difference to system responsiveness.
Most people won't need 8 cores, but I reckon the average office worker in 2022 would benefit from 4.
Sure chat apps and video playback are light, but what isn't is browsing the web. Web browsing has become a heavy task. If you tried to install a browser and then use it today on a core 2 duo, you'd have a very bad time. And everyone browses the web, usually quite a bit.
I can't speak for core 2 duo specifically, but I do tons of browsing on ThinkPad t420s (per parallel thread) which has Core i5-2540M. Depending on specific core 2 duo this shows as about 20 percent faster per interwebs - could be wrong. I find for browsing memory is king. Note I don't have any noscript or js killer, just pure browsers on windows 10.
Core 2 Duo on a late 2007 Macbook running Firefox on Linux Mint. It's doing surprisingly ok. It would not work as a daily driver because waiting one second for a moderately heavy page is not something anyone is willing to accept nowadays but totally functional whereas browsing on a single core 2004 iMac is excruciating.
Does it seem strange to anyone that this might be the case, that web browsing is more resource intensive than streaming and playing video? If true, is it because everyone builds SPAs for everything using heavy frameworks like angular and react?
I recently picked up a 2008 17" MacBook Pro. (2.5ghz core 2 Duo).
It browses the web perfectly fine, plays YouTube, and runs VS code, all that the same time. The only website I've notice it struggles on is Google maps, but Google maps has a 2D mode than runs great.
It did actually surprise me how well it runs and now I've been using it as my main machine for a few weeks.
Google makes enough money off Chrome and youtube they could afford to pay licensing fees so that end-users have a good experience. Software decoding vs hardware decoding is a massive difference in user experience.
Hardware decoding avoids license fees by anything that uses it. The fees are paid by the manufacturer of the hardware doing the encode/decode.
Even then it’s not like they’re paying per unit. All of the patent pools and licenses I know of have a cap and AMD, Intel, Nvidia, etc are easily already there.
> I always cringe a bit when people insist they need 6 cores “so they can have discord and Spotify open while they game”.
I mean, they're not wrong. I have Discord sitting open at the login screen and it's burning 10% of a core. Run a few Electron apps and your core2duo suddenly becomes a core1uno.
> Playing a video and writing a document
Using what? VLC + notepad, or Chrome + YouTube + Google Docs? There's a huge difference.
Discord gets a core, Spotify gets a core, Chrome with a few webapps open gets a few cores. That leaves 1 or 2 for a game if you're lucky. God forbid you also try to stream to Twitch. I'll cut the rant short here. You're right though; you should cringe if they insist on 6 cores, instead of 8 or more.
Bloated software is the problem. The worst part is that efficient alternatives exist (Discord -> Ripcord, Spotify -> spotify-tui) but they're against ToS and can get you banned from both services. From what I hear Discord is a lot more likely to ban you. I've not heard of any Spotify bans. But as a user, even if you know the problem, your only choices are to risk an account ban, or buy an "overpowered" CPU to run your chat client.
EDIT: Am I wrong? If you're skeptical, go find a Core 2 Duo and try watching a YouTube video while you're in a Discord call, and let me know how it goes. Those are the kinds of apps people use these days, like it or not. Don't shoot the messenger.
> I mean, they're not wrong. I have Discord sitting open at the login screen and it's burning 10% of a core. Run a few Electron apps and your core2duo suddenly becomes a core1uno.
That is what I said about “people look at the measurement at the desktop and freak out about single-digit utilization without even realizing their processor is sitting in the lowest frequency state”.
Is your processor even boosting all its cores when that 10% measurement is taken? In my experience usually it is not. Maybe different for Zen2/Zen3 but cpu measurement gets very complicated in general for those architectures since it can move between boost states so fast - it can boost so agilely that you can end up with garbage numbers because it’s moving between boost states faster than the task manager can enumerate the threads. It can move between boost states between the time you take the frequency measurement and the time you start enumerating the cpu load. This was a whole thing a couple years ago when people started looking at Zen2 and it turns out most third-party tools can’t handle this and only Ryzen Master gives you an accurate picture of what it’s doing.
In general it is such a small amount of cpu that it’s not worth caring about.
Also, typically while you game you’d have the overlay up and not the actual UI and that’s even faster - showing the login screen is likely slower than actually in game.
Load up a game and the cpu will be loaded up harder and will stay boosted and then take your measurement of discord.
People who are casually making observations about how intensive discord are never do that, it’s just “I see it using 6% at the desktop” [with the processor in the idle state rendering a full ui instead of just the overlay].
I ran a quick `while true {}` in the background. It used 7B cycles per second (twice the clock speed for some reason?), while Discord used about 700M. Discord does drop lower if I minimize the window. But still, thousands of context switches per second, while completely logged out. I have no idea what it could be doing.
You're right that I'm probably not benchmarking rigorously enough. But people don't care about any of this stuff. They care about "my FPS gets lower when Discord is open, pls fix". And they don't want to micromanage CPU usage either. Make sure you minimize Discord after starting the call, make sure you close that gmail tab during the game then open it back up when you're done so you don't miss notifications? No thanks. I'd rather just pay for a few more cores.
I guess what triggered me is you saying you cringe at the end users. They don't deserve that. They just want the stuff they buy to work reliably. And if they need a bunch of extra headroom for that to happen, well, that's just the sorry state of the industry. If you must cringe at someone, direct your cringe at developers.
I don't think Spotify will ban you for optional clients. Other clients are the whole point of their API. There's a bunch of rules you need to follow (largely about attribution) but I think at worst the dev will get their API key revoked - users should be perfectly safe.
I’ve seen degradation in game play when I have 20+ tabs open in chrome. I have a modern 8core 16thread cpu (ryzen 3700x) 32gb of ram, nvme, and a 3070. It’s noticeable, either bec the tabs are saturating the eth or bec they’re pegging the cpu, but you can clearly feel slower.
That's a fair point, although some people do constantly multitask.
I have ADHD and I'm capable of doing work while focussing entirely on two tasks, often I'm looking at one screen where I'm writing code while reading an article on another screen.
Sometimes I feel like "reality" is too slow and I've already figured out the code for the unit test (or whatever I'm working on) in my head and I'm impatiently waiting for my hands to catch up while filling in a boilerplate template. I often focus on two things at once because the main work I'm doing isn't stimulating enough.
A lot of people question how I'm able to do this without making constant mistakes, but before I started on my medication my brain and the world around me was so incredibly noisy. Not necessarily sound, it's just that everything around me was a distraction. When I'm taking my medication, the distractions don't exist and there's almost so little happening that I find it unsettling.
I don't really know how to describe it properly so this probably made little sense. Just imagine you have a really angry baby screaming at you 24/7, it would probably effect your work? Now the baby is suddenly being quiet and you're able to work really effectively, but you got so good at coping with the baby distracting you that now you have all this spare CPU time in your brain which used to be devoted to denoising.
Absolutely. But you first have to factor in that my ADHD is performance degrading.
As an example: Mirtazapine is an antidepressant used to treat major depressive disorder, but that doesn't mean you can give it to an otherwise emotionally healthy person and it'll give the same effect delta to them.
If you put a crutch on a healthy leg, it doesn't suddenly become a better leg.
My medication allows me to experience the world without crippling anxiety and constant distraction, it just so happens that 30 years living with those issues means that a large portion of my attention and time was devoted to dealing with them.
So, yes, my ADHD medication is performance enhancing, but that's an incredibly reductive way to look at it. That doesn't mean that anyone can take it and perform better, in fact studies of studies have found that the results of stimulative medication for people without ADHD (such as those who are using it to try and improve cognition during studying) can be similarly triggered by placebo effects:
As it remains unclear whether stimulant medication has the same effect on healthy individuals as for those with ADHD, it is possible that many reported effects of prescription stimulants in healthy individuals may stem from placebo effects. Looby and Earleywine (2011) examined whether placebo effects influence reports of subjective mood and cognitive performance among college students who endorsed several risk factors for prescription stimulant misuse (e.g., low grade point average, fraternity/sorority involvement, binge drinking). Interestingly, participants believed that they had better ability to focus and persevere, particularly for a sustained amount of time, when they expected to receive MPH (Looby and Earleywine 2011). This is similar to circumstances in which participants may engage in nonmedical-stimulant use to study or cram for extended hours. On the other hand, when experimental participants did not expect to receive MPH, their attention appeared disrupted resulting in inconsistent reaction times throughout the CPT.
I take my medication because I have a disability. I was diagnosed later in life and have a well tested / carefully controlled history of use with this medication where I'm frequently checked against diagnostic guidelines. In almost every metric of my life the disability had a negative effect, you can see a history of that here: https://news.ycombinator.com/item?id=28739990#28741432
If my medication is performance enhancing, then literally any antidepressant, antianxiety, anticoagulant, anti-inflammatory, antihistamine, antibiotic, etc., is also performance enhancing by the same thought. What we need to consider is the baseline it lifts you from and the place it takes you to.
It's probably worth mentioning that I'm in the UK, not the USA, where access to stimulant medication is a _lot_ stricter and it can be very difficult to get a diagnosis of ADHD. I had to go through several hour long sessions with specialised psychiatrists to receive the diagnosis after spending years being tested for other conditions like Mixed-Anxiety Disorder.
An excerpt from my remission notice is here showing the clinically recognised reduction in symptoms, it mentions that the decision was based on both subjective and objective justifications: https://i.imgur.com/RJQCAs6.png
The diagnostic screening questions for the ASRS1.1 are here showing the areas where I subjectively improved: https:/...
I do! I was on 30mg for a week, then increased to 50, and I've been stable there ever since.
I was worried at the start I would end up on 70mg, but it just seemed to work from day one onwards.
Do you get the reduced appetite? I basically don't eat from 11am until 8pm most days, which has worked quite well for weight loss. I wasn't super heavy (18st, now 14.5st) but I had put on quite a lot during COVID.
> It's worth making it explicit that many home computing tasks are highly parrallelizable because the work is actually completely separate programs.
That's a great point. At any point in time there are already dozens of processes running, of which only a fraction are tied to a GUI. Even GUI apps are currently resorting to launching multiple processes. For example, Firefox announced last year their new site isolation security architecture where they load each site as an independent process.
That's not highly parallelizable, that's only 2 task.
If those two tasks aren't themselves multithreaded (which they arn't if you're doing them in your browser) then you need at most 3 cores.
All the other background processes in your browser and OS are not using any significant CPU time, and can all comfortably share a fraction of a single core.
the concept of “e-cores” is a rebranding of Atom. The Atom name is extremely tarnished from the days when it was super slow in-order netbook cores but Intel started seriously revamping the design in 2012-2014 with silvermont and airmont. They have been good for several years now - Goldmont (eg J5005) is between core2quad and Nehalem i5 (non-SMT) level performance despite running lower clocks (ie IPC is higher), and there have been 2 subsequent generations since - but they haven’t been able to escape the stigma of the atom branding until now.
Denverton is actually a very interesting platform for the wattage - that’s 16 Goldmont cores in a 32W package (despite being 14nm!) with full RDIMM support and QuickAssist acceleration. And as mentioned there is a new atom-based server platform with newer cores (I think they’re not the Gracemont but actually the next iteration?) coming in the next year or two.
If you want something with a lower core count now, look for something with a J3405, J5005, or Tremont-based processor. Intel has some NUCs with those chips and there are also itx-form factor motherboards from other companies like Asrock and gigabyte.
Actually a lot of the 2-bay synology NASs come with those chips, I believe it’s the ones ending in J. They have fairly nice video encode/decode support for their price bracket which makes them nice for plex/etc.
As an aside, I don't think even the early Atom processors were as terrible as their reputation branded them. I had an early Atom-based netbook, and it was totally acceptable... if running Linux (or XP).
Kind of like the "Vista debacle", I think Atoms were just unsuited to running Windows 7. Maybe I'm too forgiving or forgetful, but I used that netbook for many years, and it wasn't that slow with Linux. (7 was pretty sluggish, though)
...I do remember intensely disliking the introduction of early gnome 3, while running Fedora on that netbook. Not kind to the integrated graphics :-)
I had an eeepc 901, used it for years with linux. First it ran gnome 2 I think, then xfce. Perfectly useful little machine. In later years I switched to lxde to eke a little more out of a little less.
In that form factor, and costing what it did, I was massively impressed.
I had a college professor who loaned me one for a project and it was absolutely unusable in windows. Took like 10 minutes to boot to the desktop (although maybe he had viruses, wouldn’t have surprised me, and he certainly didn’t have an SSD in it).
Lxde is great though and sure id believe that. I ran lxde on a single-core Phenom low power thing and with a SSD it was marginally usable even with that complete shit hardware.
Yeah I think the solid state storage card really made it workable, I got an add-on 64Gb card at no small expense, doubled the RAM (to 2GB!), and linux was generally very happy on there.
100%, SSDs make a massive difference in perceived performance. That laptop (it was an AMD V140, it was the Compaq CQ56 shit-top they sent out to replace laptops after bumpgate) was completely unusable with a HDD in windows but between a SSD and lxde it became marginally tolerable. It certainly wasn't fast but it could browse the web and it could play hardware-accelerated youtube if you used H264, etc.
I actually would not be surprised if, at the extreme low-end like the V140 or the Atom netbooks, that a SSD actually produced measurable increases in CPU performance. Not only do you have iowait which generally doesn't count as "idle" time according to linux (and probably windows) cpu utilization metrics, but even if you account for iowait, swapping off for another task while you iowait likely does have a performance hit in terms of caches that are no longer hot and so on, so performance is likely diminished when you come back to it too. And with no SMT, and only a single thread, context switching does become comparatively much more expensive in that fashion.
I agree, the Atom processors have tended to be fine. Lots of questionable accompanying technology choices or marketing practices though.
1st gen Atoms with the chipset (945?) that was bigger and hotter than the processors was kind of amusing.
That Intel can't help itself from making confusing product names is pretty inexcusable. If a 'Pentium' is an Atom or not makes a big difference in what the specifications mean, and it's not clear to regular people that a dual-core mainstream Intel is about performance equal to a quad-core Atom (assuming similar age chips).
Laptop makers love to combine atom CPUs with amazingly slow spinning hard drives, and they're usually pretty small too. Windows 10 performance on a spinning drive is trash, and it's even worse when it's a low rpm laptop drive. Doesn't help that they're often as little ram as possible and soldered down (although even with 8GB ram, Windows 10 will flog a spinning drive just sitting at the desktop; who knows what it's doing).
It would be interesting to see a 4p+24e or 6p+16e setup for parallelized loads though. Not sure there's currently a need for that on the desktop though.
In my analysis, only Gracemont (Atom 5th gen) is good and that’s for two reasons:
1. It has roughly the IPC of Skylake. 2. It’s the first Atom with AVX2. Even in 2021 Intel released brand new Atom cores without AVX2, which was introduced with Haswell (Haswell New Instructions). And in most tests, merely enabling AVX2 will yield 10% higher performance. That’s why Clear Linux has higher performance as they’re all tuned for Skylake. That’s also the reason why Red Hat couldn’t target x86-64-v3 with AVX2 for their new releases.
All previous Atom cores are slower than you imply:
Tremont (Atom 4th gen) has the IPC of Sandy Bridge. Goldmont (Atom 3rd gen) has the IPC of Core. It’s not exceeding Core like you imply. And Atom first gen is dog slow. It is like running a Pentium II/III CPU or a first generation Raspberry Pi. I find them an unsuccessful attempt of Intel of trying to create a mobile chip. In essence they just brought back a Pentium era chip.
No, it’s really doing OK for what it is. A Xeon Bronze 3104 is a 6C6T 1.7 GHz Skylake that gets a 88W tdp and a J5005 is a 2.8 GHz 4C4T Goldmont running in 15W. That is 10.7 core-cycle-units for the Skylake and 11.2 core-cycle-units for the atom. The Skylake is not really all that far ahead - maybe 50% ahead on average in this suite, so the J5005 is running about 2/3 of Skylake IPC. I assume c-ray is probably using AVX there? And the J5005 still does fine.
There is obviously a dearth of actual practical benchmarks of real tasks apart from STH and a few others, but if you look at Passmark or other generic benchmarks, like I said, it’s certainly faster than core2quad enthusiast desktop, which is completely reasonable and even impressive given its power budget. Actually in Passmark it’s even faster than a full Nehalem Core i7 with SMT, it actually works out just a bit under a Sandy Bridge i5 of similar clock - so I was a bit conservative there. And Passmark corroborates my “3104 is about 50% faster” guesstimate.
(UserBenchmark’s actual measurements - not the speed rating - is likely more accurate than Passmark but I know I’ll get my head bitten off for it!)
(Do remember that Intel progress through that era wasn’t really as bad as people say… I’ve seen people say “5% a generation” and the only generation close to that low was Broadwell. Skylake is a ton faster than Sandy Bridge clock for clock, and J5005 still does fall a bit below Sandy Bridge. There’s your two generations of progress since Goldmont - Atom went from below Sandy Bridge to matching Skylake during the Tremont/Gracemont generations.)
And like I said, I knew I was gonna get my head bitten off for that. This is where you distinguish the bandwagon hangers-on from the people actually interested in technical discussion.
The commentary UserBenchmark provides is terrible and the “effective speed” composite rating is terrible. The actual int/fp benchmarks are perfectly fine and actually tend to be more reflective of actual benchmarks like spec than Passmark has been, in my experience. Passmark does sometimes have oddities, UserBenchmark hasn’t.
But bandwagoners can’t accept that, they just see “UserBenchmark” and their vision goes red and they shake as they struggle to type out “nobody should EVER use UserBenchmark”. I do apologize for being colorful but that’s how it is.
Like I said, there’s a reason I didn’t link it, no matter what you say, there’s a significant number of people who just can’t act mature enough to handle separating the data from the editorial positions.
It would be nice if we had good Spec2017 benchmarks for everything. I would prefer that. If you want broad comparisons of completely obscure hardware “what does this ultra-budget Xeon Bronze look like against this atom vs a desktop processor from 2007” the options are limited. You have Passmark, you have UserBenchmark, and you have Geekbench. And people still lose their shit over Geekbench too.
Even Passmark has only seen that xeon a total of 14 times.
The situation is even worse for GPUs where UserBenchmark is the only reasonably broad benchmark database available for truly ancient stuff that 3Dmark won’t run on. What if you want to compare a GTX 9800 to a Vega 7 and a Intel HD 605? It’s UserBenchmark or nothing. It sucks but it is what it is, and people need to just grow up. Including the owner of UserBenchmark. But the effective speed is just a composite benchmark and he’s never actually tampered the underlying int/fp benchmarks.
Phoronix is also not particularly good and has some serious methodological problems too. But nobody else does linux benchmarks apart from STH and Phoronix, and Phoronix pumps them out like crazy and just survives on volume even if his testing is kinda shit. Look at his “Linux games” test and he’s mixing multiple resolutions into a single result set, including the same game multiple times, etc, and then pulling out averages, everything is framerates and not frametimes (meaning, averages and not minimums), etc. From what I’ve heard his application testing isn’t any better but I don’t remember specifics.
Just goes to show: you can be scum and put out good data, and you can be a pillar of the community and put out bad data. Reddit tone police don’t change that.
There is a awful lot of incredibly useful data in the world that came from people who are extremely disagreeable on a personal level - or much, much worse.
Well it would make some sense to have perhaps 2 or 4 single core speed focused cores for the main process that you're currently working on, and then add as many of the smaller E cores as you can, to handle the hundreds of background processes so that the P cores can run uninterrupted by them.
If this works that way in practice is anyone's guess though.
I think a single E core can handle all the background processes when they're not doing any real work. I currently have around 250 processes and 3500 threads running on my computer, and the CPU utilization is 1%. It would probably make sense to use an extremely power efficient core that's only used for background services, one core that could be clocked much higher than all the rest for the foreground thread(s), and maybe some combination of E and P cores for the rest of the threads doing actual calculations. But adding lots of simple E cores for parallel work feels a bit like Larrabee.
It's clear that we need something in between regular quad core CPUs, and GPUs connected through PCIe. Some kind of parallel compute unit should be standard in pretty much all CPUs (or SoCs) eventually.
> It's clear that we need something in between regular quad core CPUs, and GPUs connected through PCIe. Some kind of parallel compute unit should be standard in pretty much all CPUs (or SoCs) eventually.
> What people need to realize is that a P cores takes around 4 times the die area/transistors of an E core, and use probably 3 to 4 times the power of an E core, but performance is only around double that of an E core... (Btw, these ratios are pretty similar as those for the Firestorm/Icestorm cores in Apple's M1.)
Apple's strategy with their in house chips has always involved optimizing for performance and power draw, and not so much for die area. They throw execution hardware at the problem and run the clocks slower for power efficiency.
For instance, going back to the 2012 A5x:
>the A5X is approximately 2x the size of NVIDIA's Tegra 3, and even larger than a dual-core Sandy Bridge desktop CPU
While it's technically possible for Apple to have ramped up CPU clocks in pursuit of higher performance, Apple has traditionally been very conservative on CPU clock frequency. Higher clocks require higher voltages (especially on the same process node), which result in an exponential increase in power consumption.
> Personally I don't really see a point in having a hybrid architecture like this unless it'll lead to massively increased core counts
What about massively reduced energy consumption and associated loss of head dissipation/noise and increased battery life? Isn't that why Apple and mobile phone SoCs introduced that design in the first place?
Yeah, that too, but as I've said, right now compared to AMD this advantage seems to not be there for Intel, and personally I mostly care about desktops where power consumption isn't as important as raw power.
> What about massively reduced energy consumption and associated loss of head dissipation/noise and increased battery life?
I hope their drivers would work properly with the OS. Else, the user would be locked on either the performance cores or the efficiency cores - it happens with laptops with dual graphics card (intel inbuilt and high performance Nvidia or AMD graphics)
I think it’s a real shame we haven’t learned to use more threads and cores in our software. It’s not like having dozens of small cores is something new.
I wonder if we’ll see more of a movement towards virtual threads Java Loom style where cheap threads in the language and cheap cores on the cpu will combine to create a programming style where you’ll just dispatch little worklets all over the place without worry about “is this a large enough amount of work to justify a thread” the way we do now. I also wonder if that will cause a trend towards optimizing startup latency in both language threads and the efficiency cores.
It's so they don't have to change the topology from the ring bus in the consumer parts while boosting the effective core count. Look at a 12900K block diagram. Four efficiency cores plus their combined L2 take up the same amount of die area as a P core. This means they don't have to deal with mesh interconnects or something like Infinity Fabric (which they already tried on Ivy Bridge-EP and were sorely dissatisfied with) and kick the can further down the road.
It's because Intel is behind in process nodes (transistor density) and chiplet technology.
Once Intel is matching AMD for density, Intel will be able to jam far more small cores into a multi chip architecture than AMD will with just "big" cores.
Big/Little is a very important step forward towards core density (and idle power consumption, switching off big cores under load is a huge saver in laptop power usage).
I see. Yet, this particular entry only contains tiny amounts of well known information. I feel like this product line has been well covered for months now.
The key to understanding the popularity of stories that contain "well known information" is that not everyone has the same life experiences. For every piece of info you know, there are a lot of people who have never heard of it. Including on HN.
Was it really "well known" that the complex number multiplication opcodes needed the destination to differ from the source or that the intermediate results were rounded to ensure they were commutative?
So the low power high power cores do seem to be inspired by the M1. If they can implement that easily in x86 is another thing but if anyone can Intel can I guess.
The half prescion fps are obviously aimed at ML but how on earth are the hoping to compete with gpus on this front? Where is the use case on this (genuine question).
The complex numbers bit makes me think of quantum circuit simulation (but that could be bias on my part as that's what I'm looking at). But this is a relatively small field at the moment, and just through normal use nvideas backend simulator blows cpu performance out of the water (both for speed and number of qubits simulated).
I assume I'm missing the point here and it is heavily used in compression or something but again if someone had an idea of the usecase for complex number instructions that would be useful!
Thank you
FP16 is a common GPU format (ML is better with a different one called BF16). The reason to add it to your CPU is so you can work on the same data without having to do software emulation which is really slow.
Alder lake so far hasn't been anywhere near the efficiency of the M1.
> The half prescion fps are obviously aimed at ML but how on earth are the hoping to compete with gpus on this front? Where is the use case on this (genuine question).
Sure, it's not GPU competition, but still helpful, after all a fair chunk of the computers on the planet to do not have discrete GPUs.
My understanding is that Intel boosts power so a single core can chew through a program quicker than being inherently energy efficient (performance-per-watt). Most of the time this benefits Intel, because a core will boost-up-then-down quicker over completion of the program than a more efficient AMD processor core.
(This is my layman understanding.)
I solidly believe AMD is the king for efficiency, but I wish I could find better benchmarks showing idle power use for AMD vs Intel (not peak power use). My understanding is that Intel has deeper power states its processors settle into.
I'm surprised someone doesn't artificially limit or undervolt their Intel proc to approach or surpass what we're seeing from AMD. Would it significantly lengthen the total execution time of the program? Would there still be a significant difference in "performance"?
Performance-per-watt is important, and the total amount of power to execute the program. Would this be in Kilowatt/hours?
I want to see how many Kilowatt/hours something like Cinebench consumes on similar AMD & Intel processors, so we can derive the "real computing efficiency".
Besides hybrid cores, Intel and AMD are using the same techniques. They both turbo and they have the same low-power modes. This year Intel is slightly ahead in performance and AMD has longer battery life but next year it will probably be different.
For a desktop or mobile processor this might make sense, but many hosting providers sell dedicated hardware with desktop processors to be used in a server environment. (E.g. Hetzner https://www.hetzner.com/dedicated-rootserver/ex100)
So the provider can save energy and make sense of this, but customers won't notice the energy savings in their pocket and they only notice that they can only use 8 cores.
When I tested this a few months ago under Linux every request of our server application would land on a different CPU leading to funny performance characteristics. Even Ubuntu 22.4 does not seem to have the kernel included 5.16+ that fully supports this CPU (https://www.phoronix.com/scan.php?page=article&item=adl-linu...)
As we use the JVM the only interesting use case I can imagine would be to assign the slower efficiency cores for the garbage collector.
While there might be some edge cases where it's useful, heterogenous cores don't make a tonne of sense in the datacentre, hence why Intel hasn't made Xeon's with that design.
It seems to me this falls firmly in the category of caveat emptor.
> As we use the JVM the only interesting use case I can imagine would be to assign the slower efficiency cores for the garbage collector.
Generally speaking trying to run code that shares memory between P and E cores only results in pain. It could work but the software needs to be designed for it.
This reminds me of the 'transition' to multi-core code started in the "Core" days (or actually AMD's Athlon X2). The switch to multi-core/thread/process (as this all were steps towards such approach) needed to be implemented by all layers (h/w, os, s/w).
We're now getting from symmetrical to heterogeneous computing becoming the mains-stream on all platforms.
Just a few weeks ago there was here the issue with measuring efficiency which is flawed on Apple's Activity Monitor.
As the multi-thread code paradigms thrived, asymmetrical-cores will also become part of the toolchain eventually.
You get a bunch of smart hardware guys into a room, they design this funky exotic architecture. Then the software goes "Allocate these threads to whatever is idle" and suddenly you've completely lost any possible advantage and are thrashing around with no idea what you're doing. The big-little architecture from Apple was accompanied by software that basically handles that for you. From what I heard there were similar problems with Xeon Phi - great theoretical performance but a very difficult programming model and as a result very challenging sales for the Intel sales guys.
I remember Mono, a .NET runtime, having to change its allocations to fit inside a smaller core. This was because a thread could start on a bigger core but then be shunted onto a smaller core.
That fixed the crashing but obviously came with a loss of performance.
It was a bit more complicated than that: the issue was that one of the early big.LITTLE designs (Samsung's Exynos 8890) had different cacheline sizes on the big and little cores.
glibc's `__clear_cache` would cache the cacheline size on first call, so if the program was started on a big core then migrated onto a little core it would only flush every other cacheline. Which was an issue for any program needing to explicitely clear the caches, like most any JIT.
But fundamentally this is the 8890 being broken: as the Mono post notes, technically nothing precludes core migration in the middle of clearing the cache, which would also lead to broken behaviour, with no mitigation.
Right, this design model works only if there’s a certification process for software. If they can convince the virus scanning and HIPS companies to implement a certification standard, and then convince businesses that this certification process matters when purchasing enterprise numbers of clients, then it will start to make a dent.
So true, my experience with the BIG.little ARM platform and a regular Linux kernel is that you always end up with the wrong scheduling and the system underperforms because it uses the little CPU for a compute-intensive single-thread task… I'll just avoid these kind of systems.
Big little is fundamentally the correct architecture. It's not Intel's fault that the software guys making the Linux Kernel haven't fully supported heterogeneous yet.
In the long term, it'll be the optimal call for so many workloads. Thousands of server and laptop workloads care only about performance per watt, not peak single thread.
It's true that the software kills the theoretical performance that the hardware makes available. But I actually think this is an indictment of hardware not software engineers. It's basically "We've created this incredibly difficult problem for you, good luck" from the hardware engineers. How the hell is the OS meant to know whether this thread is going to turn out to be a rendering operation or a logging thread?
With these architectures I think often what the management should really look at is: "We currently employ 100 hardware engineers, and 30 software engineers, with this architecture, we're going to need 50 hardware engineers and 500 software engineers. Do we still think this is the right call?"
True, and not just the system software, but the larger software ecosystem as well. Apple paved the way for the M1 by years and years of goading developers to be honest with themselves about what QoS priorities they really need on their DispatchQueues, and now there are plenty of threads explicitly marking themselves as being good candidates for running on the E cores.
How did Apple do that? I've seen the documentation while building stuff with Xcode but I never saw anything that would have actually forced my app to behave (outside of iOS).
It sounds like you're not asking about technical enforcement — as opposed to prodding, which is what I was talking about. You're right, though, they do also apply pressure through technical means. They did this at design time, by thinking through what the default QoS will be when it is left unspecified.
As part of alder lake intel includes thread director which handles thread dispatch via hardware and software, I believe. The main downside as far as I know is you have to run a very recent kernel to have access to this.
I may be way off here, but to me the transition to heterogeneous cores is more of a long term than short them play, anticipating the era where we'll have different kinds of cores in our systems.
Even the performance cores of today have circuitry that's there to make certain workloads faster (video decoding, crypto operations, etc.). Other stuff has mostly been offset to the GPUs (actual grahphics-related stuff, machine learning).
In an architecture where you can mix/match different kinds of cores, it doesn't sound crazy to have cores that are specially designed for those tasks. Processors could ship with one "crypto-core", a "video-decoding-core", etc.. These should provide similar performance to full performance cores at _much_ lower cost, energy consumption and die area.
If (and that's a big if) they can get to a point where:
a) Designing/implementing new specific-purpose small cores is not outrageously expensive because the whole coordination/integration part is already solved
b) They have a combination of cores that provides similar performance to current p-cores-only cpu's at similar processor cost and power budget.
That would be a huge win for processor companies. They would be able to open lines of specialized processors commanding huge premiums for certain industries (think about processors with 128 ml-cores, 128 video-encoding-cores, etc.).
Luckily video codecs also hardly change in an incompatible way after they ship. There's also varying levels of support that can be fully or partially HW decode/encoded and the pure CPU fallback.
Intel's Quicksync is best-in-class and even available on low power Celeron/Atoms
It's strange Agner seems so against this hybrid design despite the fact it's been proven to work so well for ARM. I'd agree this architecture is very difficult to optimize a single application for, but on the other hand he doesn't seem to consider whether it can lead to better performance and battery life in everyday use cases with mixed workloads using less than 100% of available performance.
He seems more annoyed about bad software thread management in Windows 11 (and none at all in Linux and MacOS) and that P & E cores have different instruction set support, which messes with apps.
It's weird to be annoyed by the difference in instruction set because by default they do have the same instruction set, and neither support avx512. You have to go to great lengths to enable avx512 on the P cores and as he says only certain motherboards can do it. I'll admit it is a shame that neither core supports avx512 by default though. If he'd said he was upset the gracemont E cores didn't emulate avx512 in microcode then I'd agree completely.
Anyone else slightly annoyed that the CPUID had to be made the same for both P & E cores, just to accommodate DRM? Making it harder for all software to properly utilize the technology effectively.
I feel like it's a fundamentally bad idea to give P and E cores disparate instructions sets. Apple did it right on the M1 — the cores have different designs, but the E cores support all the instructions the P cores do.
It’s not just DRM; It’s about being homogeneous. If a process is on an E core and checks CPUID to see that AVX-512 isn’t available, it won’t execute those instructions even if AVX-512 was supported on a P core. As Agner said:
> It is completely unrealistic that an application program can handle this situation in a reasonable manner and optimally allocate different threads to the different cores.
A suggestion I saw here a lot was “pretend to support it, but detect attempts to use the instructions and pin the task to the P cores.” This would have issues with libraries like libc that would attempt to vectorize themselves and end up forcing an E core appropriate process onto a P core.
252 comments
[ 814 ms ] story [ 1109 ms ] threadIt’s not that Apple invented chips that are remarkably faster than Intel’s or NVIDIA’s, it’s that Apple invented an integrated experience on this new architecture with split performance & efficiency cores and did so while maintaining backwards compatibility with macOS x86 apps in a way that’s indistinguishable from M1-native apps. (So they kept the “baggage” as it were, and it still performs fine.)
I expect Microsoft’s Surface line will catch up eventually, but the disconnect between Qualcomm/Intel as chip makers and Microsoft as OS vendor will slow down the transition for a good 4-6 years easy, and we’re 2 years into this transition to ultra-energy efficient architectures with less thermal throttling. (Full disclosure, I’ve a few Apple shares.)
You won’t match its power/performance by doing one thing and suddenly making it there. Intel’s process is quite close to TSMC’s already.
A lot of M1's advantage is in terms of density. That's why they were theoretically able to make such an absolutely enormous processor and keep it cool.
Making that theory work in practice came from decades of low power semi experience. They're extremely good chip designers.
AMD and Intel are on similar nodes at the moment, but Apple has a very notable advantage. It's not "secret sauce" so much as paying many billions for exclusive access.
This is the biggest part of the 'sauce' but having E-cores and extremely close memory (faster memory access - less time the cores need to wait for data - less time spent doing nothing and burning watts) surely helps.
Not to mention there's the M1 pro, max, and ultra if you need more memory bandwidth.
I have only seen the 8+8core referred to as a 16 core machine once. And it was Apple making a comparison to Intel and calling Intel's cpu a 16 core.
So a CPU with 20 threads has maybe eight running at full speed of a P-core and the other twelve pretty gimped comparatively.
But I'm hopeful this is a stop-gap solution and we'll just make the p-cores more power efficient under medium load, and merely have the OS clock them lower when something is determined a background task. Then we "only" have to get that part to become smart enough.
Old OS versions continue to work fine, and newer OS versions can opt into the new world and benefit.
(After reading the TFA: that's what Agner says right there in the 4th paragraph.)
(After re-reading the comment: I guess that the OS changes would need to be extensive with little to no benefit: running AVX2 on all cores will likely be faster than running 2 P cores with AVX512. The only thing that is really affected is the code that could use AVX512_FP16, but I doubt there's a lot of it outside of Intel.)
I don't think that is true. In the simplest case, you could modify the #UD handler to notice when the fault is caused by an AVX512 instruction running on an E-core, and then simply and pin the process to the P-cores, migrate the process, and continue. All existing scheduler functionality.
> The only thing that is really affected is the code that could use AVX512_FP16, but I doubt there's a lot of it outside of Intel.
AVX512 is a lot more than just extending the vector width, and that extended functionality can be very useful for quickly emulating other CPU's vector instruction sets.
It has been clear for a long while that the vector length was going to gradually increase every few years to support new data processing requirements, so the writing has been on the wall all along.
https://gist.github.com/zingaburga/805669eb891c820bd220418ee...
It’s not all bad but the people who just say “just treat it like an n-element array and let the hardware handle it bro!” are hand waving an enormous amount of algorithms that may not actually be translatable to that.
And broadly speaking, having a couple code-paths and choosing at runtime is not that bad. HPC is used to working “close to the metal” and tuning their code to run optimally. You can’t make that complexity go away - things will always run better on X hardware but Y hardware runs that code path way worse. In this domain all you do with clever auto-programming magic is obscure the problem - you are now writing for the optimizer instead of writing for the hardware, but different hardware still runs differently and you need multiple code paths to hit that optimally. So now you have two problems - keeping the hardware running optimally and keeping the optimizer from messing you up.
Since you are now dispatching large operations that will not complete atomically in one cycle (they simply won’t, you can’t do 2M elements on a 128-bit vector) it doesn’t seem likely that we will stay strictly in-order on these, and the next step is you start extracting parallelism from the stream and that’s the problem that immediately emerges from that. Now you have an optimizer running a SIMT program (opmasks are basically SIMT) extracted from the instruction stream, and you need to not just write code that runs fast - you can’t - you need to write code that the optimizer turns into code that runs fast, and that seems like absolute hell on an architecture where you can’t even know the number of registers-per-thread in advance. Maybe I am reading this all wrong but that’s where you’d really end up going with a “run this op on an N element array” model. If you run one vector-instruction at a time you are in cache/register hell constantly spilling to memory and back, so you want to optimize that to keep things in-register/in-cache, and that means extracting parallelism from the stream to run more stuff while it’s still somewhere reasonably hot.
This is the exact garden-path that GPU drivers went down and it was a huge mistake with OpenGL/DX9/DX11 that had to be walked back with the DX12/Vulkan APIs to get back closer to the metal, because the optimizer stage became completely inscrutable and managing it to keep it from doing something stupid became impossible.
There is a level of irreducible complexity in HPC and exposing the hardware is the best way to avoid it. Leaky abstractions make things worse.
Writing for a phone is not HPC but if you care about energy efficiency you still need to make sure you’re running reasonably optimally and not wasting cycles, especially with a super wide vector unit. That means you’re leaning heavily on the “sensible” code path having decent efficiency, which means you’re leaning on the hardware to behave sanely. So it’s the same thing.
Again, not saying it’s a completely bad idea, but you can already see the abstraction starting to leak with the various types of operations that don’t really work on a variable-width hardware concept, and that’s worrying. There is a lot more opportunity for this to go bad than I think people are at first glance.
Hardly new. The VAX 6000 Series Vector Processor supported variable length vectors with up to 64 vector elements:
I am pretty sure the concept predates the VAX days and might go as far back as the CDC STAR-100 from 1974 and potentially also the Cray-2, but I have had a trouble locating authoritative sources on the details of the ISA of the former.The most notorious vector computer is Cray-1 from 1976, and it already had variable length vector operations using a vector length register (and also 64-element vector registers).
All later vector computers and also both the vector extension of the RISC-V ISA and the SVE extension of ARM have been strongly inspired by the ISA of Cray-1.
CDC STAR-100 introduced a few influential ideas, but it had a low performance because it did not have vector registers (the operands of its vector instructions were arrays stored in the main memory).
It might reinvolve changes to DRM software but looking at the current compromises, looks like a better deal.
Of course Intel should have thought of this and didn't pick this route. I'd love to see the reasoning though.
In this case detecting DRM software and scheduling it differently would require sophisticated support in the operating system, or even ideally DRM support implemented by the operating system. However these things are probably beyond Microsoft's capabilities in its current state.
Well it would be nice if Microsoft found a way to implement them and coordinate with Intel, with backwards compatibility of course (which is the key point of Windows).
Then this brings another question to my mind: if Intel is so contrained by Microsoft's decisions (which make sense) they should have been in a good relationship designing things that work nice together. Why did they design their CPU in a way that would break in Windows usage in the first place?
They must have much more R&D than saying "let's design this hybrid CPU and put a new instruction set on some cores and see if it works on Windows and DRM software else we disable those features" kind of approach.
Are Intel just taking it too far in terms of disparity between the architectures?
Background tasks always used only the little cores, foreground apps always always used only the big cores. I do not believe it was possible for a single app to run threads on both big and little cores.
And my understanding is that every arm big.LITTLE combination supported exactly the same ISA and were intentionally 100% binary compatible. For example, you couldn't pair arm A53 little cores (that support only armv8-a) with A75 big cores (which support armv8.2-a)
Then, you'd have a compatibility shim that emulates unimplemented instructions using traps for software that hadn't yet been rewritten to understand the above signal.
But maybe that's just because it's a first generation technology for Intel? I hope things improve in the future because I sure would love to have more cores available (currently rocking a 32 core Threadripper) and AMD is sure as hell not interested in actually pushing HEDT forward since they became the top dog. (I have a few grand burning a hole in my pocket waiting for a product they don't want to release.)
wtf do you need more cores for??? Wat are you doing?
> not interested in actually pushing HEDT forward since
consider buying a server.
M1 builds a lot faster than Intel CPUs even with fewer cores because of the memory bandwidth.
A CPU with so many general purpose high-perf cores that it manages to look like a GPU in terms of execution units count is a beautiful thing.
It depends very much what you're doing of course (compiling linux kernel repeatedly perhaps) but if you have a little you can get a heck of a lot out of it with experience, although you surely will hit a wall sometime. But 32 cores not enough is just unimaginable for me.
But your case, ok, understood.
BTW The biggest slowdown IME is not from lack of cores but lack of RAM. Odd that no-one's mentioned that. Or using a server with larger numbers of memory channels to keep their cores fed. I/O > cycles more often than not.
Random examples:
Trawling through 70 GB of web server log files going back years with ad-hoc analytics. I tried ingesting this into a cloud service but it was taking forever, and cost a non-trivial amount of money. It was also dog slow compared to just doing it locally with SQL Server Clustered Columnstore.
Compiling Chromium. I'm not even a browser developer or anything, but there was a "global" change that I had to preview ahead of time, so I made a custom Chrome build that had the change embedded in it. I also made a matching build without the change to enable A/B comparison testing.
Server base image task sequence development. You would think a server is ideal for this, but no! It's very I/O heavy, and doing it locally in Hyper-V using an NVMe drive can make this 4-5x as productive as any shared platform with throttled disks. There are zero cloud services that can even approach a Samsung Pro 980 SSD for reasonable costs.
Multi-server lab tests. Compared to cloud services, it's faster and easier to build a local pool of machines for testing how they interact. With 64 GB of memory, I can (and have) easily run 20+ machines. Compared to cloud platforms, I can do things like thin-cloning to build (and then delete and rebuild) these nearly instantly -- about 5 seconds or less in some cases. Even the relatively heavyweight Windows Server 2022 VMs boots in just 1.4 seconds! Some build jobs require nearly 100 GB of content to be deployed, and the upload to the cloud is a problem in some customer environments. E.g.: due to security restrictions. Doing it locally is easier and faster.
Photography and video editing. I do both as a hobby. Lightroom will cheerfully use whatever number of CPUs (and GPUs!) you have to process images. My recent hardware upgrade made a very noticeable difference to the workflow. Similarly, photogrammetry and other advanced 3D modelling workflows can use up virtually any amount of compute.
Password cracking. I occasionally need to reverse a password hash for work, typically to fix lost access to some ancient piece of network gear. Sometimes it's for verifying password strength where the plain-text password is not known. I could distribute this across a pool of cloud VMs, but my lab subscriptions have hard limits and this is sensitive data.
Automated searches through code for a breaking change. E.g.: scripting the checkout, build, test, etc... in a loop. (Git bisect doesn't solve every problem)
VS Code now has ripgrep integrated, so "find in files" will use every CPU. I have about 100 GB of files on my laptop from various old projects, so this is a very helpful feature for digging up some code snippet that I only vaguely remember writing. It can find a match for a regular expression at 700 MB/s using 16 threads at 100%.
Defragmenting moderately large (50-500 GB) databases. I have some customers that have built up horrendously fragmented databases over decades, but can't afford the IOPS and/or downtime to fully rebuild all of the tables and indexes. I recently utilised the aforementioned Samsung SSD to defragment an 80 GB database in just minutes, that would have taken days or weeks on the production server. Again, transferring this out to a cloud is a no-go, and the problem was that typical server storage is much slower than a typical laptop. Locally I could get 200K IOPS sustained. SQL Server Developer Edition is the same as Enterprise, and will utilise all cores to rebuild indexes in parallel.
Etc, etc...
If you know what you are doing, you can make the CPUs go "brr...".
NB. "Trawling through 70 GB of web server log files" unless you partition your search you're going to be doing a lot of IO due to having only 64-GB of RAM. Chucking in extra mem to keep it all ram-resident will beat cores any day.
The query engine in Microsoft SQL Server is absurdly advanced, and simply beats the pants off just about everything else for complex, general-case queries. There are other engines that are better tuned for special cases, but if you've got "stuff" that's within the capacity of a single box, it's really hard to beat MSSQL.
Let's expand on my example above: Clustered ColumnStore.[1]
It's basically PowerBI or Vertica embedded within the traditional MSSQL engine. This extension enables the ability to have a per-column (vertical) on-disk storage format instead of the traditional row-store format most databases have.
You can have it as a secondary acceleration index, or as a clustered (primary) index. In the latter case, it'll compress your raw table data. For my log files it was 33-to-1 compression (3% of the original file size).
Second, even if the data doesn't fit into memory, it'll utilise the columnar storage to simply ignore (skip) columns it doesn't need for the query. I was computing performance statistics, so it was just reading in some numbers instead of the text. A further 100-to-1 optimisation can be gained here.
It then parallelises this across all cores/hyperthreads, so I get 100% utilisation on 16 cores.
All of this I/O is fully asynchronous with a huge queue depth. Expect to see 200K IOPS on a laptop, easily.
Once in memory, it uses Intel AVX SIMD instructions to chew through the data 8 rows per clock tick per core. With 8 cores going at once, this is 64 rows per clock, or up to 234 billion rows processed per second.
What's neat about it is that it's fully general: Columnstore can be mixed -- in a single query -- with in-memory tables, on-disk tables, traditional row-store, and external tables streamed in over the network. The engine will just... "figure this out". It'll build indexes on the fly if it has to, ideally in memory, but failing that... it'll stream disk-to-disk as required. I've seen "sort spills" reach 600K IOPS on my laptop.
If I have to scan through years of logs and be able to join it against additional tables for "enriching" it, then it's hard to beat. Sure, there's a scale where the are dedicated log analytics databases, but they have their quirks, limitations, and costs.
I take your incredulity that others can fully utilise their computers as a sign that general knowledge of just what is possible is poor. People have pointed out that some simple shell scripts can outperform Hadoop clusters up to a surprising scale. Similarly, knowledge of the capabilities of traditional database engines like MSSQL is oddly lacking, even amongst developers. Not to mention more esoteric ways of making your computer do your bidding, such as Mathematica, Julia, GPU codes, or whatever.
Your computer is a power tool for your squishy, manual-labour meat brain! A lever for the mind. Learn to utilise it better and you'll be better at thinking.
[1] https://docs.microsoft.com/en-us/sql/relational-databases/in...
> It then parallelises this across all cores/hyperthreads, so I get 100% utilisation on 16 cores.
that suggests your working set is indeed smaller than ram. Good.
However be aware that the parallelise feature of mssql is much better that it was but is still moderately crap. Disable it and see if you get a 16X slowdown. My guess is you won't, which means when enabled you won't be getting a 16X speedup (also hyperthreading can actually slow things down - disbale it and see what happens). Edit: I'd not be surprised if you were only getting 4X to 5X speedup (I may be wrong but I do have experience in this).
> With 8 cores going at once, this is 64 rows per clock, or up to 234 billion rows processed per second.
(giggle) Oh, I don't think so!
> I've seen "sort spills" reach 600K IOPS on my laptop.
Then you're hitting IO and yer cores are useless. It spills to disk because it's overflowed ram - you may have a problem.
> Learn to utilise it better and you'll be better at thinking.
... agreed, but the reverse is true also.
edit 2: Depending on what you're looking for, and if you do it often, check if MSSQL's free-text indexing can index it, if so you may get a far bigger speedup from that than anything else. Maybe. Depending.
So keep in mind that the SSD I'm referring to can read at 7 GB/s. Not seven gigabits... 7 gigabytes per second. I've benchmarked some not-so-old servers (the entire box, not a VM!) as capable of only 35 GB/s streaming reads from main memory! So these days "disk" is more like "5x slower memory". It used to be 100-1000x slower.
I told one customer to get a 16x PCIe gen 4 adapter card with 4 NVMe M.2 ports on it. This would -- in theory -- be able to pump out 28 GB/s of reads, equivalent to a 200 Gbps NIC. Total storage would be 32 TB with CoTS 8TB drives.
That's a mind-blowing amount of IO in a single box!
The hardware has the capability, it's the software and inability to utilise that is the bottleneck now..
> I've benchmarked some not-so-old servers (the entire box, not a VM!) as capable of only 35 GB/s streaming reads from main memory!
Then you have either a low-end server or something is very wrong (I think you benchmarked wrong, see below). You should be getting maybe 15 GB/sec per channel, so 4 channel = 60GB/sec, for a decent server you may well have 6 or 8 channels. (figures may be a bit off but 35 is puzzling me).
You may have measured on a single core, but each core has its own mem reading stuff so it should multiply up if your benchmark uses extra cores.
Anyway, I have to get back to work on my dual-core haswell. Good luck!
(Edit: 15 * 4 = 60, not 80, corrected)
So much so that I’m now trying to figure out whether we can somehow cheaply move to a dual Epyc 7742 setup.
At the risk of ill manners but i must ask, have you looked at all this because I've got in some cases orders of magnitude speedup by popping the hood and instrumenting.
We could sit there and optimize the approach, but the time that would take and the complexity added, it would be cheaper to just buy more powerful computers.
For example, on reasonably new 8 core 64gb ram, 7gbs HDD machine, I optimized a calculation for one of the features down from 2-3 days to about 15 minutes. Throwing cores or ram or disk at this algo before would’ve got nowhere. Now htop glows like a Christmas tree when it’s running. This was costly to do but it was the only way.
We could talk about python being slow and blah blah blah use C but once again, whenever we do tricky stuff with code that then gets shared amongst a bunch of python devs I need to think about the holistic cost including maintenance, ease of changes, bug fixing, resource that actually know c, client appetite, what the production load is going to look like etc
And finally, at some point a client will just come along with another million rows of data
I have a bunch of embarrassingly parallel workloads which I have to run regularly (text analysis, machine learning which is not easily accelerated on a GPU, etc.).
> consider buying a server.
I looked into it; not really an option. Those are vastly more expensive, really loud (I need a quiet workstation), have significantly lower boost clocks (I also need single-thread performance), and are either really hard or even impossible to actually buy in my country through normal retail channels.
If you’re using Rayon for anything you expect to benefit from a lot of cores, my experiences have been very negative in the past. Rayon is often extremely inefficient once you allow it to use more than a handful of cores.[0]
If your unit of work is absolutely massive, you might not be as affected by Rayon’s scheduler, but it makes me sad that this issue still hasn’t been resolved. My comment on that thread (documenting just one instance where I ran into this issue) was almost 4 years ago! And the issue itself is almost a year older than that. Sure, I could try to contribute a general solution, but it’s just easier to stop using Rayon.
I no longer consider Rayon to be an advantage for Rust. There are plenty of good ways to wire up a parallel work pool, but they do require more than a one line change.
[0]: https://github.com/rayon-rs/rayon/issues/394
It's 90% the process node. There is no magic design sauce that makes those chips better.
The big/little design change did seem to improve performance quite a bit for Intel, even using the same manufacturing node as before
The same MacBook Air can do hour long Skype video calls with ~9% charge consumption, with no discernible heat difference on the body too.
M1 is a different beast.
Working on HPC field and with generations and generations of Intel hardware, I can tell you that two node difference (alone) never gives you 25% real world performance in the realm of Intel, let alone theoretical.
Real world performance pretty much always lines up with these estimates. It's the main reason why we have exponential growth in computing power in the first place.
It would be a more fair comparison to compare the M1 to AMD's ZEN 4, which should be coming out next quarter.
[1] https://www.tsmc.com/english/dedicatedFoundry/technology/log...
This is jumping from Intel's heroically old 14nm+++++++++++++++ to a brand spanking new TSMC process. It might be on the order of 4 to 6 times denser.
We've seen how the latter went until AMD came back to the game.
Why is everyone insistent on comparing Apple’s latest CPUs to Intel’s parts from multiple generations back?
Intel is on 12xxx series now. AMD also has new generation laptop CPUs that perform very well. Let’s compare to those.
The M1 is power efficient but much of that is due to the TSMC process they used. I really don’t understand this desire to make arbitrary comparisons to old Intel CPUs.
Will these be meaningful for you?
The thing is, M1 on air is a relatively low clock speed CPU with passive cooling. 10200K is actively cooled, unlimited beast of its generation. The libraries I use are X86 optimized. Yet the code I’ve written is 25% faster on ARM.
This is significant.
Apple knew this so the M1 has an extremely fast memory system.
Intel’s memory architecture can keep up with the load until I utilize all cores. When I start to use HT cores, neither cache, nor memory can keep up.
I do the both tests (Intel and Apple) both with physical core counts to keep it fair.
So it’s not some run of the mill code.
Curious what you meant here since that part doesnt exist, and if it did it would be near the bottom of the stack for the generation (101xx and 103xx are the low core count i3 and i5 parts).
Written that comment on mobile on a plane, before take-off, so I either typo'd and didn't see, or auto-correct changed it deeming 10200K is more correct than 10700K.
If you look to the original comment where I sparked the discussion, I've mentioned it as 10700K already.
Being able to compete against a consumer high end desktop, even if it's an older one, it's a feat all by itself
> M1 is a different beast.
The M1 has other advantages which are decidedly not rooted in the hardware: software. Apple seems to do profoundly better than Windows and Linux at applications actually using hardware accelerators. This makes a gigantic difference in performance and power efficiency.
On a Windows or Linux system it is not at all a given that even very easily hardware-accelerated tasks, like video decoding, actually use the hardware that's there.
TSMC 5nm is better, but AMD is not there yet.
I find the different balances of performance and efficiency cores interesting. I’d love to have a lot more Icestorm cores on a Mac to play with background tasks that don’t need the firestorm ones.
On the Intel side, I’m also disappointed by not having the option to implement the missing instructions in software.
But even being one behind, it outperforms M1, with lower efficiency. Thus reasonable to speculate that a process node advancement would at least put Intel's chip on par with M1 on both perf and efficiency
We'll see, but I highly doubt that Intel-at-equivalent-of-TSMC-5nm will be performance/watt competitive with the M1. Apple gets that power efficiency via an extremely wide-and-deep design that intentionally trades off a surprisingly low max clock ceiling (3.2GHz) for power efficiency while maintaining an absurdly high IPC (something like 75% higher than Intel's IPC at only one process node smaller) that makes it much more competitive on performance than conventional industry wisdom ever expected (there is and has been for years a strong belief that a design as wide as Apple's would be grossly inefficient because you could not possibly extract enough instruction level parallelism to keep so many decoders/execution units saturated -- hence we've spent over a decade with 4-5 wide frontends being the "practical maximum" that allows them to still target ~5GHz turbo speeds, which was believed to be basically the best/only path to high performance within what was believed to be the limit of "achievable" IPC. The M1 blows these fundamental assumptions out of the water).
So Intel makes conventional design decisions that make up the performance of their lower clock-for-clock IPC by turboing well north of 5GHz, at the cost of consuming something like 7x the max power of the M1. A single node shrink isn't going to buy them a 75% increase in IPC (their last 4 or 5 node shrinks combined haven't bought them that much IPC increase), so Intel@equivalent-of-TSMC-5nm still going to have to burn power like crazy to stay competitive on performance, and that's going to cost them efficiency that the node shrink just can't paper over.
It will lead in absolute perf, but its not going to be nearly as efficient.
I have a list of instructions per clock from SPECINT results which I compiled from Web sources. (I didn’t verify the results myself.) I added in rumored future processors with their highest rumored performances:
Intel has had 256-bit dual-issue FMA since Haswell (2013); that's 32 flops/cycle/core. AVX-512 doubled the vector length for Xeon Phi and was generally available in 2017 Skylake.
[1] https://www.tomshardware.com/uk/news/ryzen-7-5800x3d-vs-core...
That is because there is a huge design gap. Intel architecture and design teams have always assumed on the inevitability of their own manufacturing superiority and process advantage and thus never had to really consider where to place transistors for the most benefit, or think really hard on better ways to improve single-thread performance instead of just turning a knob that was invented two decades ago.
In any case, your statement is just plain wrong. Dennard scaling has been dead for a decade now. Planar area decreases with new processes come with increased power density, i.e. new process nodes reduce die sizes but the transistor performance does not improve the way they used to.
source: worked at Intel for CPU design for a decade, then worked on the same things elsewhere using TSMC process
Nobody in CPU design cares about unconstrained power performance. Even at Intel, they never use the PL2 power level as the design landing zone. All modeling is done at much lower power target, even below PL1.
That is to say: unconstrained power performance is not an engineering effort, it is a marketing and SKU differentiation effort. Your “more powerful” statement is a matter of how much power Intel/Apple/AMD/whoever is willing to shove into their piece of silicon, which in turn dictates the form factors and customers they can try to satisfy.
Apple only needs a few watts per big core and still get the peak perf it needs, and put that chip in all the form factors it sells. Obviously if M1 sucked down the kind of power as any Intel chip, it cannot go into an iPad, or a fanless macbook air, without totally compromising performance.
Intel on the other hand needs grossly more watts to hit that same performance, and therefore is losing badly on form factors that care about power: laptops and servers. Some form factors/customers just don’t care, which is why Intel cooks up marketing targeting the gamer market, or the 8-pound laptop market. That’s all fine, just don’t pretend Intel is not getting absolutely hammered on the CPU technical specs for the money-making markets that matter.
BTW, the DIY gamer CPU market does not matter. The 8-pound laptop market does not matter.
The fanboyism runs so deep it's amazing. Let's compare PS5 to GameCube next and talk about Sony's great engineering advantage
Go compare the perf/power curves of the alder-lake cores (both P/E) versus the last-gen Apple cores from TSMC n7+. TSMC n7+ is roughly comparable to Intel 10nm ESF in transistor performance, so it is a fair comparison.
Do your research then get back to me about how the process is entirely responsible for every perf/power difference and we (the silicon design teams) are just identical robots doing the same thing across the board in every company.
I do like how you claim there is no design magic sauce, but then Apple will lose big when Intel/AMD come out with their 5nm equivalents, as in, they found some magic sauce? WTF?
They have a patent for something similar at least: https://wccftech.com/amd-big-little-processor-cores-show-up-...
Not saying x86 can’t ever replicate that, but the ISA makes it harder.
It also avoids scheduling conflict between housekeeping / background tasks and "user" stuff, which means more predictable and stable throughput and better cache utilisation (your cache doesn't get blown because the system needed some time to zero memory or update something).
What people need to realize is that a P cores takes around 4 times the die area/transistors of an E core, and use probably 3 to 4 times the power of an E core, but performance is only around double that of an E core. So with the same area, you get of the 8P+8E you can get 10P cores, not 12, and the question is if a 10P (rather than a 16P or even 12P) core Alder Lake is better than a 8P+8E. (Btw, these ratios are pretty similar as those for the Firestorm/Icestorm cores in Apple's M1.)
If your workload is very parallelizable you get more bang from your silicon and Watts with E cores than with P cores. If you have only embarrassingly paralleled workloads, you're probably better off with 32E cores than with a 8P cores.
If you watch a video while writing a document in word, that's trivially parrallelizable.
(Or it should - browsers are not always very good about this but the tools are there, they just don’t want to pay licensing fees to use them.)
I always cringe a bit when people insist they need 6 cores “so they can have discord and Spotify open while they game”. People don’t generally have a very good intuitive sense of just how little cpu time it takes to run those (a lot of times the “at the desktop” measurements are not even with the processor clocked up, it’s running in the lowest active power state), and there is almost always a little cpu headroom due to non-perfect game scaling. Context switching is cheap, your OS is running thousands of threads at any given time, but people don’t have a good intuitive sense of the overhead of context switching either.
Playing a video and writing a document is something that can be trivially done on a core2duo era processor, or the modern equivalent of a low-end dual-core nuc with e-cores (atom cores) like a J4005. You don’t even need performance cores for that.
The tradeoff Intel is making is that some tasks need performance cores because they can’t really be parallelized all that well, and embarrassingly parallel stuff is more efficient to implement with lots of smaller cores. In principle this is also more power efficient, like big.LITTLE in phones, but Intel actually seems to have fairly poor power scaling - they are space efficient but not all that power efficient. This is also partially due to the fact that the little cores share a core voltage with the big cores - the big cores need higher voltage to clock up and that kills efficiency on the little cores. At that point you may as well clock the little cores up too since they have to share the voltage anyway, and that’s an inefficient way to run them.
As Agner Fog mentions, the AVX-512 is also present in silicon yet disabled, which hugely blows up core size for almost no benefit (it’s dark silicon which does have some minor thermal benefits). Alder Lake is in many ways not that well-designed at a high level, but it is magnificently fast on those performance cores and the e-cores would be very nice if they weren’t conjoined awkwardly to the p-cores. Gracemont-based server and atom chips are going to be nifty when they finally arrive.
Apple is still way ahead with Icestorm though - those cores are absolutely absurdly tiny for the performance they give. They are 0.5mm2 each iirc, even considering the node shrink that is teeny-tiny, they are half as many transistors as a Zen3 core and almost the same performance (meaning much higher IPC) or something along those lines. Firestorm is neat but Icestorm didn’t get nearly the attention and it is really insanely fast for how few transistors it uses.
Playing a video and editing a document, sure. But change that to being on a video call and having a few moderately intensive web apps open (say Gmail, Google Docs and some kind of project management software) and your average dual core processor from a few years ago is going to start to struggle.
I do ALL my work on a t25 which is just about 5 years old now. This includes daily calls with dozens of video participants (ms teams and WebEx mostly) while I have chrome and Firefox open with multiple tabs including outlook and Gmail and slack, couple of MS words, couple of PowerPoints, Visio, etc. That's a dual core processor from more than couple of years ago and it's nowhere near capacity with that workload.
It does have lots of ram and Ethernet to residential Fibe 1gbps. But CPU just hasn't been an issue and I plan to work on this another 5 years
The real question is, are you one of the video participants too? Or are you just watching others but not sending any video of your own?
Heck I've run full PeopleSoft multi vm setups on my ancient t420 back in the day. A little CPU really goes a longer way than people realize.
I’m just using Nehalem and Conroe as a touchstone that people understand in terms of cpu performance. That is what you get in general cpu performance, but it’s also got better media support and crypto acceleration and so on, so it’s more usable today. Actually a closer look at the numbers puts it just below Sandy Bridge i5 actually.
AMD cpu users (non-apu) and RX 6500/6400 owners obviously need to look elsewhere however. NVIDIA’s 1030 chips are ancient pascal based crap but AMD dropping encoder support on a chip designed during the pandemic era of work-from-home is a bit tone-deaf.
(Yes, 6500 and 6400 were designed during the pandemic, the architecture took longer to develop but 6500 wasn’t the lead ship in its architecture, and taping out a new chip configuration takes much less time than designing a whole architecture from scratch… AMD definitely knew it would be used as a desktop chip too. It was a deliberate space-saving move in the face of their wafer shortages, not something they got blindsided by.)
My 2015 MacBook Pro with Broadwell i5 would indeed work "fine" for videoconferencing (in my case Google Meet), but it would also use most of a CPU core to do it. Which is fine if that's all you're doing. But if I wanted to share my screen and work on something else while doing so then it could quite quickly get to the point where I needed to start shutting down other programs, and even then it could cause lag when typing in less well-implemented apps.
My new MacBook with M1 Pro cpu it still uses most of a cpu core for videoconferencing, but it's a single E core and I have 9 others cores. Videoconferencing now makes zero difference to system responsiveness.
Most people won't need 8 cores, but I reckon the average office worker in 2022 would benefit from 4.
It browses the web perfectly fine, plays YouTube, and runs VS code, all that the same time. The only website I've notice it struggles on is Google maps, but Google maps has a 2D mode than runs great.
It did actually surprise me how well it runs and now I've been using it as my main machine for a few weeks.
Can you blame browser makers for this? When was the last time you saw someone pay for a browser? Pre-2005 Opera?
Chrome is free and youtube notoriously doesn't make a profit (unless this changed in 2021).
Their money comes mostly from search.
Even then it’s not like they’re paying per unit. All of the patent pools and licenses I know of have a cap and AMD, Intel, Nvidia, etc are easily already there.
I mean, they're not wrong. I have Discord sitting open at the login screen and it's burning 10% of a core. Run a few Electron apps and your core2duo suddenly becomes a core1uno.
> Playing a video and writing a document
Using what? VLC + notepad, or Chrome + YouTube + Google Docs? There's a huge difference.
Discord gets a core, Spotify gets a core, Chrome with a few webapps open gets a few cores. That leaves 1 or 2 for a game if you're lucky. God forbid you also try to stream to Twitch. I'll cut the rant short here. You're right though; you should cringe if they insist on 6 cores, instead of 8 or more.
Bloated software is the problem. The worst part is that efficient alternatives exist (Discord -> Ripcord, Spotify -> spotify-tui) but they're against ToS and can get you banned from both services. From what I hear Discord is a lot more likely to ban you. I've not heard of any Spotify bans. But as a user, even if you know the problem, your only choices are to risk an account ban, or buy an "overpowered" CPU to run your chat client.
EDIT: Am I wrong? If you're skeptical, go find a Core 2 Duo and try watching a YouTube video while you're in a Discord call, and let me know how it goes. Those are the kinds of apps people use these days, like it or not. Don't shoot the messenger.
That is what I said about “people look at the measurement at the desktop and freak out about single-digit utilization without even realizing their processor is sitting in the lowest frequency state”.
Is your processor even boosting all its cores when that 10% measurement is taken? In my experience usually it is not. Maybe different for Zen2/Zen3 but cpu measurement gets very complicated in general for those architectures since it can move between boost states so fast - it can boost so agilely that you can end up with garbage numbers because it’s moving between boost states faster than the task manager can enumerate the threads. It can move between boost states between the time you take the frequency measurement and the time you start enumerating the cpu load. This was a whole thing a couple years ago when people started looking at Zen2 and it turns out most third-party tools can’t handle this and only Ryzen Master gives you an accurate picture of what it’s doing.
In general it is such a small amount of cpu that it’s not worth caring about.
Also, typically while you game you’d have the overlay up and not the actual UI and that’s even faster - showing the login screen is likely slower than actually in game.
Load up a game and the cpu will be loaded up harder and will stay boosted and then take your measurement of discord.
People who are casually making observations about how intensive discord are never do that, it’s just “I see it using 6% at the desktop” [with the processor in the idle state rendering a full ui instead of just the overlay].
You're right that I'm probably not benchmarking rigorously enough. But people don't care about any of this stuff. They care about "my FPS gets lower when Discord is open, pls fix". And they don't want to micromanage CPU usage either. Make sure you minimize Discord after starting the call, make sure you close that gmail tab during the game then open it back up when you're done so you don't miss notifications? No thanks. I'd rather just pay for a few more cores.
I guess what triggered me is you saying you cringe at the end users. They don't deserve that. They just want the stuff they buy to work reliably. And if they need a bunch of extra headroom for that to happen, well, that's just the sorry state of the industry. If you must cringe at someone, direct your cringe at developers.
I posted this comment here because I also appreciate your name.
I have ADHD and I'm capable of doing work while focussing entirely on two tasks, often I'm looking at one screen where I'm writing code while reading an article on another screen.
Sometimes I feel like "reality" is too slow and I've already figured out the code for the unit test (or whatever I'm working on) in my head and I'm impatiently waiting for my hands to catch up while filling in a boilerplate template. I often focus on two things at once because the main work I'm doing isn't stimulating enough.
A lot of people question how I'm able to do this without making constant mistakes, but before I started on my medication my brain and the world around me was so incredibly noisy. Not necessarily sound, it's just that everything around me was a distraction. When I'm taking my medication, the distractions don't exist and there's almost so little happening that I find it unsettling.
I don't really know how to describe it properly so this probably made little sense. Just imagine you have a really angry baby screaming at you 24/7, it would probably effect your work? Now the baby is suddenly being quiet and you're able to work really effectively, but you got so good at coping with the baby distracting you that now you have all this spare CPU time in your brain which used to be devoted to denoising.
As an example: Mirtazapine is an antidepressant used to treat major depressive disorder, but that doesn't mean you can give it to an otherwise emotionally healthy person and it'll give the same effect delta to them.
If you put a crutch on a healthy leg, it doesn't suddenly become a better leg.
My medication allows me to experience the world without crippling anxiety and constant distraction, it just so happens that 30 years living with those issues means that a large portion of my attention and time was devoted to dealing with them.
So, yes, my ADHD medication is performance enhancing, but that's an incredibly reductive way to look at it. That doesn't mean that anyone can take it and perform better, in fact studies of studies have found that the results of stimulative medication for people without ADHD (such as those who are using it to try and improve cognition during studying) can be similarly triggered by placebo effects:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3489818/
I take my medication because I have a disability. I was diagnosed later in life and have a well tested / carefully controlled history of use with this medication where I'm frequently checked against diagnostic guidelines. In almost every metric of my life the disability had a negative effect, you can see a history of that here: https://news.ycombinator.com/item?id=28739990#28741432If my medication is performance enhancing, then literally any antidepressant, antianxiety, anticoagulant, anti-inflammatory, antihistamine, antibiotic, etc., is also performance enhancing by the same thought. What we need to consider is the baseline it lifts you from and the place it takes you to.
It's probably worth mentioning that I'm in the UK, not the USA, where access to stimulant medication is a _lot_ stricter and it can be very difficult to get a diagnosis of ADHD. I had to go through several hour long sessions with specialised psychiatrists to receive the diagnosis after spending years being tested for other conditions like Mixed-Anxiety Disorder.
An excerpt from my remission notice is here showing the clinically recognised reduction in symptoms, it mentions that the decision was based on both subjective and objective justifications: https://i.imgur.com/RJQCAs6.png
The diagnostic screening questions for the ASRS1.1 are here showing the areas where I subjectively improved: https:/...
I was worried at the start I would end up on 70mg, but it just seemed to work from day one onwards.
Do you get the reduced appetite? I basically don't eat from 11am until 8pm most days, which has worked quite well for weight loss. I wasn't super heavy (18st, now 14.5st) but I had put on quite a lot during COVID.
That's a great point. At any point in time there are already dozens of processes running, of which only a fraction are tied to a GUI. Even GUI apps are currently resorting to launching multiple processes. For example, Firefox announced last year their new site isolation security architecture where they load each site as an independent process.
https://hacks.mozilla.org/2021/05/introducing-firefox-new-si...
Also, here's a link to a recent HN discussion on Firefox's recent announcement.
https://news.ycombinator.com/item?id=31355857
https://security.googleblog.com/2018/07/mitigating-spectre-w...
If those two tasks aren't themselves multithreaded (which they arn't if you're doing them in your browser) then you need at most 3 cores.
All the other background processes in your browser and OS are not using any significant CPU time, and can all comfortably share a fraction of a single core.
[1] https://www.servethehome.com/intel-sierra-forest-the-e-core-...
Denverton is actually a very interesting platform for the wattage - that’s 16 Goldmont cores in a 32W package (despite being 14nm!) with full RDIMM support and QuickAssist acceleration. And as mentioned there is a new atom-based server platform with newer cores (I think they’re not the Gracemont but actually the next iteration?) coming in the next year or two.
https://en.wikichip.org/wiki/intel/cores/denverton
If you want something with a lower core count now, look for something with a J3405, J5005, or Tremont-based processor. Intel has some NUCs with those chips and there are also itx-form factor motherboards from other companies like Asrock and gigabyte.
Actually a lot of the 2-bay synology NASs come with those chips, I believe it’s the ones ending in J. They have fairly nice video encode/decode support for their price bracket which makes them nice for plex/etc.
Kind of like the "Vista debacle", I think Atoms were just unsuited to running Windows 7. Maybe I'm too forgiving or forgetful, but I used that netbook for many years, and it wasn't that slow with Linux. (7 was pretty sluggish, though)
...I do remember intensely disliking the introduction of early gnome 3, while running Fedora on that netbook. Not kind to the integrated graphics :-)
In that form factor, and costing what it did, I was massively impressed.
Lxde is great though and sure id believe that. I ran lxde on a single-core Phenom low power thing and with a SSD it was marginally usable even with that complete shit hardware.
I actually would not be surprised if, at the extreme low-end like the V140 or the Atom netbooks, that a SSD actually produced measurable increases in CPU performance. Not only do you have iowait which generally doesn't count as "idle" time according to linux (and probably windows) cpu utilization metrics, but even if you account for iowait, swapping off for another task while you iowait likely does have a performance hit in terms of caches that are no longer hot and so on, so performance is likely diminished when you come back to it too. And with no SMT, and only a single thread, context switching does become comparatively much more expensive in that fashion.
1st gen Atoms with the chipset (945?) that was bigger and hotter than the processors was kind of amusing.
That Intel can't help itself from making confusing product names is pretty inexcusable. If a 'Pentium' is an Atom or not makes a big difference in what the specifications mean, and it's not clear to regular people that a dual-core mainstream Intel is about performance equal to a quad-core Atom (assuming similar age chips).
Laptop makers love to combine atom CPUs with amazingly slow spinning hard drives, and they're usually pretty small too. Windows 10 performance on a spinning drive is trash, and it's even worse when it's a low rpm laptop drive. Doesn't help that they're often as little ram as possible and soldered down (although even with 8GB ram, Windows 10 will flog a spinning drive just sitting at the desktop; who knows what it's doing).
It would be interesting to see a 4p+24e or 6p+16e setup for parallelized loads though. Not sure there's currently a need for that on the desktop though.
1. It has roughly the IPC of Skylake. 2. It’s the first Atom with AVX2. Even in 2021 Intel released brand new Atom cores without AVX2, which was introduced with Haswell (Haswell New Instructions). And in most tests, merely enabling AVX2 will yield 10% higher performance. That’s why Clear Linux has higher performance as they’re all tuned for Skylake. That’s also the reason why Red Hat couldn’t target x86-64-v3 with AVX2 for their new releases.
All previous Atom cores are slower than you imply:
Tremont (Atom 4th gen) has the IPC of Sandy Bridge. Goldmont (Atom 3rd gen) has the IPC of Core. It’s not exceeding Core like you imply. And Atom first gen is dog slow. It is like running a Pentium II/III CPU or a first generation Raspberry Pi. I find them an unsuccessful attempt of Intel of trying to create a mobile chip. In essence they just brought back a Pentium era chip.
This new Atom is fantastic though.
There is obviously a dearth of actual practical benchmarks of real tasks apart from STH and a few others, but if you look at Passmark or other generic benchmarks, like I said, it’s certainly faster than core2quad enthusiast desktop, which is completely reasonable and even impressive given its power budget. Actually in Passmark it’s even faster than a full Nehalem Core i7 with SMT, it actually works out just a bit under a Sandy Bridge i5 of similar clock - so I was a bit conservative there. And Passmark corroborates my “3104 is about 50% faster” guesstimate.
(UserBenchmark’s actual measurements - not the speed rating - is likely more accurate than Passmark but I know I’ll get my head bitten off for it!)
https://www.servethehome.com/intel-pentium-silver-j5005-benc...
https://www.cpubenchmark.net/compare/Intel-Pentium-Silver-J5...
(Do remember that Intel progress through that era wasn’t really as bad as people say… I’ve seen people say “5% a generation” and the only generation close to that low was Broadwell. Skylake is a ton faster than Sandy Bridge clock for clock, and J5005 still does fall a bit below Sandy Bridge. There’s your two generations of progress since Goldmont - Atom went from below Sandy Bridge to matching Skylake during the Tremont/Gracemont generations.)
The commentary UserBenchmark provides is terrible and the “effective speed” composite rating is terrible. The actual int/fp benchmarks are perfectly fine and actually tend to be more reflective of actual benchmarks like spec than Passmark has been, in my experience. Passmark does sometimes have oddities, UserBenchmark hasn’t.
But bandwagoners can’t accept that, they just see “UserBenchmark” and their vision goes red and they shake as they struggle to type out “nobody should EVER use UserBenchmark”. I do apologize for being colorful but that’s how it is.
Like I said, there’s a reason I didn’t link it, no matter what you say, there’s a significant number of people who just can’t act mature enough to handle separating the data from the editorial positions.
It would be nice if we had good Spec2017 benchmarks for everything. I would prefer that. If you want broad comparisons of completely obscure hardware “what does this ultra-budget Xeon Bronze look like against this atom vs a desktop processor from 2007” the options are limited. You have Passmark, you have UserBenchmark, and you have Geekbench. And people still lose their shit over Geekbench too.
Even Passmark has only seen that xeon a total of 14 times.
The situation is even worse for GPUs where UserBenchmark is the only reasonably broad benchmark database available for truly ancient stuff that 3Dmark won’t run on. What if you want to compare a GTX 9800 to a Vega 7 and a Intel HD 605? It’s UserBenchmark or nothing. It sucks but it is what it is, and people need to just grow up. Including the owner of UserBenchmark. But the effective speed is just a composite benchmark and he’s never actually tampered the underlying int/fp benchmarks.
Phoronix is also not particularly good and has some serious methodological problems too. But nobody else does linux benchmarks apart from STH and Phoronix, and Phoronix pumps them out like crazy and just survives on volume even if his testing is kinda shit. Look at his “Linux games” test and he’s mixing multiple resolutions into a single result set, including the same game multiple times, etc, and then pulling out averages, everything is framerates and not frametimes (meaning, averages and not minimums), etc. From what I’ve heard his application testing isn’t any better but I don’t remember specifics.
Just goes to show: you can be scum and put out good data, and you can be a pillar of the community and put out bad data. Reddit tone police don’t change that.
There is a awful lot of incredibly useful data in the world that came from people who are extremely disagreeable on a personal level - or much, much worse.
That's pretty neat, but I'm thinking about, say, a 60W-95W package that would make sense in a normal desktop.
How many E-cores could you fit in such a package? For highly-parallel workloads, would it stomp all over anything else on the market? (I'd guess yes)
If this works that way in practice is anyone's guess though.
It's clear that we need something in between regular quad core CPUs, and GPUs connected through PCIe. Some kind of parallel compute unit should be standard in pretty much all CPUs (or SoCs) eventually.
So, an iGPU accessing unified RAM?
Apple's strategy with their in house chips has always involved optimizing for performance and power draw, and not so much for die area. They throw execution hardware at the problem and run the clocks slower for power efficiency.
For instance, going back to the 2012 A5x:
>the A5X is approximately 2x the size of NVIDIA's Tegra 3, and even larger than a dual-core Sandy Bridge desktop CPU
While it's technically possible for Apple to have ramped up CPU clocks in pursuit of higher performance, Apple has traditionally been very conservative on CPU clock frequency. Higher clocks require higher voltages (especially on the same process node), which result in an exponential increase in power consumption.
https://www.anandtech.com/show/5688/apple-ipad-2012-review/1...
Caring less about die area makes sense when you are selling the whole widget and not just the chip itself.
What about massively reduced energy consumption and associated loss of head dissipation/noise and increased battery life? Isn't that why Apple and mobile phone SoCs introduced that design in the first place?
I hope their drivers would work properly with the OS. Else, the user would be locked on either the performance cores or the efficiency cores - it happens with laptops with dual graphics card (intel inbuilt and high performance Nvidia or AMD graphics)
- some LAPACK-style libraries select slow routines for AMD cpus
- better understanding of / support for performance counters and suchlike
Once Intel is matching AMD for density, Intel will be able to jam far more small cores into a multi chip architecture than AMD will with just "big" cores.
Big/Little is a very important step forward towards core density (and idle power consumption, switching off big cores under load is a huge saver in laptop power usage).
Oh well.
So the low power high power cores do seem to be inspired by the M1. If they can implement that easily in x86 is another thing but if anyone can Intel can I guess.
The half prescion fps are obviously aimed at ML but how on earth are the hoping to compete with gpus on this front? Where is the use case on this (genuine question).
The complex numbers bit makes me think of quantum circuit simulation (but that could be bias on my part as that's what I'm looking at). But this is a relatively small field at the moment, and just through normal use nvideas backend simulator blows cpu performance out of the water (both for speed and number of qubits simulated).
I assume I'm missing the point here and it is heavily used in compression or something but again if someone had an idea of the usecase for complex number instructions that would be useful! Thank you
> The half prescion fps are obviously aimed at ML but how on earth are the hoping to compete with gpus on this front? Where is the use case on this (genuine question).
Sure, it's not GPU competition, but still helpful, after all a fair chunk of the computers on the planet to do not have discrete GPUs.
(This is my layman understanding.)
I solidly believe AMD is the king for efficiency, but I wish I could find better benchmarks showing idle power use for AMD vs Intel (not peak power use). My understanding is that Intel has deeper power states its processors settle into.
I'm surprised someone doesn't artificially limit or undervolt their Intel proc to approach or surpass what we're seeing from AMD. Would it significantly lengthen the total execution time of the program? Would there still be a significant difference in "performance"?
Performance-per-watt is important, and the total amount of power to execute the program. Would this be in Kilowatt/hours?
I want to see how many Kilowatt/hours something like Cinebench consumes on similar AMD & Intel processors, so we can derive the "real computing efficiency".
So the provider can save energy and make sense of this, but customers won't notice the energy savings in their pocket and they only notice that they can only use 8 cores.
When I tested this a few months ago under Linux every request of our server application would land on a different CPU leading to funny performance characteristics. Even Ubuntu 22.4 does not seem to have the kernel included 5.16+ that fully supports this CPU (https://www.phoronix.com/scan.php?page=article&item=adl-linu...)
As we use the JVM the only interesting use case I can imagine would be to assign the slower efficiency cores for the garbage collector.
It seems to me this falls firmly in the category of caveat emptor.
Generally speaking trying to run code that shares memory between P and E cores only results in pain. It could work but the software needs to be designed for it.
We're now getting from symmetrical to heterogeneous computing becoming the mains-stream on all platforms.
Just a few weeks ago there was here the issue with measuring efficiency which is flawed on Apple's Activity Monitor.
As the multi-thread code paradigms thrived, asymmetrical-cores will also become part of the toolchain eventually.
That fixed the crashing but obviously came with a loss of performance.
glibc's `__clear_cache` would cache the cacheline size on first call, so if the program was started on a big core then migrated onto a little core it would only flush every other cacheline. Which was an issue for any program needing to explicitely clear the caches, like most any JIT.
And the mitigation was not to "change its allocation", it was to bypass libgcc and handroll cache clearing: https://github.com/mono/mono/pull/3549
Source: https://www.mono-project.com/news/2016/09/12/arm64-icache/
This issue didn't only affect Mono e.g. dolphin (https://github.com/dolphin-emu/dolphin/pull/4204) and ppsspp (https://github.com/hrydgard/ppsspp/pull/8965) had been hitting the same issue and adopted mono's fix.
But fundamentally this is the 8890 being broken: as the Mono post notes, technically nothing precludes core migration in the middle of clearing the cache, which would also lead to broken behaviour, with no mitigation.
Nonetheless, I appreciate the corrections and extra context you have provided.
https://www.phoronix.com/scan.php?page=article&item=linux-51...
Did you have this kernel, or an earlier one? (Perhaps there're still many problems to fix)
Big little is fundamentally the correct architecture. It's not Intel's fault that the software guys making the Linux Kernel haven't fully supported heterogeneous yet.
In the long term, it'll be the optimal call for so many workloads. Thousands of server and laptop workloads care only about performance per watt, not peak single thread.
With these architectures I think often what the management should really look at is: "We currently employ 100 hardware engineers, and 30 software engineers, with this architecture, we're going to need 50 hardware engineers and 500 software engineers. Do we still think this is the right call?"
Because this problem is incredibly well solved (I believe?).
Scheduling priorities in operating systems an enormously well explored topic.
Put simply: you must incorporate heterogenous cores into the OS thread scheduler.
There's three actors here:
Hardware developers.
OS developers.
End user software engineers.
The hardware developers are running into physical constraints.
The fix is to shift the prioritisation between the different core types, big little, to the operating system scheduler.
Even the performance cores of today have circuitry that's there to make certain workloads faster (video decoding, crypto operations, etc.). Other stuff has mostly been offset to the GPUs (actual grahphics-related stuff, machine learning).
In an architecture where you can mix/match different kinds of cores, it doesn't sound crazy to have cores that are specially designed for those tasks. Processors could ship with one "crypto-core", a "video-decoding-core", etc.. These should provide similar performance to full performance cores at _much_ lower cost, energy consumption and die area.
If (and that's a big if) they can get to a point where:
a) Designing/implementing new specific-purpose small cores is not outrageously expensive because the whole coordination/integration part is already solved
b) They have a combination of cores that provides similar performance to current p-cores-only cpu's at similar processor cost and power budget.
That would be a huge win for processor companies. They would be able to open lines of specialized processors commanding huge premiums for certain industries (think about processors with 128 ml-cores, 128 video-encoding-cores, etc.).
Or maybe I'm just delusional :)
It does provide power advantages more than performance. The downside is that it’s much harder to improve the algorithms after they ship.
Intel's Quicksync is best-in-class and even available on low power Celeron/Atoms
> a thing that is hoped or wished for but in fact is illusory or impossible to achieve. "the economic sovereignty you claim to defend is a chimera"
That said Alder Lake does open up a bunch of new possibilities and isn't an illusion
> It is completely unrealistic that an application program can handle this situation in a reasonable manner and optimally allocate different threads to the different cores.
A suggestion I saw here a lot was “pretend to support it, but detect attempts to use the instructions and pin the task to the P cores.” This would have issues with libraries like libc that would attempt to vectorize themselves and end up forcing an E core appropriate process onto a P core.
Gracemont is faster, clock-for-clock than Broadwell and nearly as fast as Skylake running ChaChaPoly-1305, as used in WireGuard, largely due to avx2.
Intel’s benchmarking shows 80% performance peak vs peak of 4C of Gracemont vs 2C4T of Skylake.