84 comments

[ 3.1 ms ] story [ 141 ms ] thread
It is also in the comments, but the gaming benchmarks are really strange here. Every new Ryzen processor beats even the fastest Intel cpu in each and every game they benchmarked? That's highly unlikely, and completely conflicts with the results of other, more gaming focused reviewers like GamersNexus, see https://www.gamersnexus.net/hwreviews/3287-amd-r7-2700-and-2.... What is anandtech doing different?
On the comments: "Intel CPUs were tested with Meltdown/Spectre patches, that's probably the discrepancy you're seeing."
So every other reviewer is not using those patches (isn't Windows installing them automatically?), and the performance impact is that huge that it completely negates every performance advantage Intel had for now almost a decade?
I'm not 100% sure about every other reviewer, but many times tests will be done with a base windows 10 install (as to avoid the need to look at comparisons between patches).

Regarding performance impact, short answer is - yes. Basically, it removes most of the advantage of the cache for Intel.

> many times tests will be done with a base windows 10 install [without any updates/patches]

So, even without Spectre & friends, these benchmarks are essentially meaningless?

It's a difficult problem. It's not practical to re-test all of the old CPUs or GPUs every time you review a new part. Either you test all components against a baseline OS and driver set, or you test components with whatever OS and driver releases are contemporaneous. Neither case is ideal.

Reviewers have generally settled on the former option; OS and driver updates tend to increase performance, so testing with contemporaneous software might artificially inflate the apparent performance deltas in favour of newer components. Nobody wants to buy a new processor thinking that it's 40% faster than their old one, only to discover that most of the difference was just OS optimisations. You'd rather have your readers be pleasantly surprised than bitterly disappointed, so it makes sense to choose a benchmarking methodology that errs on the side of favouring older parts.

Spectre is a weird exception to business as usual, because we saw a huge performance decrease in a single update that affects one particular optimisation in one particular processor generation. In this case, it probably makes sense for reviewers to re-run all of their old benchmarks and set a new baseline.

> isn't Windows installing them automatically?

I thought the OS update was only part of the patch and that OEMs also needed to update their BIOS, which means users have to manually update their BIOS.

Microcode is indeed distributed through Windows Update as well. And packages on other OSes. And applied at boot time.
It takes a long time to run a comprehensive test suite against a single processor. Reviewers often have to ship the hardware back to the company afterward. The result is that they very frequently re-use the old benchmarks in newer tests.

New software doesn't affect existing benchmarks.

I could be mistaken, but I recall seeing benchmarks that came out during the whole meltdown/spectre fiasco showed that the effects on games were pretty negligible.

I thought it was mainly workloads relating to virtualization etc... that were effected significantly

There have been many more updates since (for instance the first article you linked was from 01/10/18):

January 29th Microsoft had to revert the Intel patch discribed:

https://www.zdnet.com/article/windows-emergency-patch-micros...

There's also implementations in March for the Spectre variant 2:

https://www.digitaltrends.com/computing/microsoft-windows-pa...

There's much more in here.

Which is wrong. Video Games don't actually make very many kernel system calls. Meltdown / Spectre patches have virtually no effect on video games.

The majority of DirectX stuff in the innermost-loops automatically DMAs to the video card without even needing a kernel call. Video games care a lot about performance and did their best to never touch kernel-space anyway.

Compilers, Web Servers, Databases, etc. etc. were affected very strongly. Compilers open files. Web Servers open Sockets. Databases communicate with mutexes / semaphores. All of these are OS-level features and require a kernel-mode switch. Meltdown means that every kernel-mode switch clears out the TLB-cache.

So each time a compiler opens a file (MMap or otherwise), it basically loses its entire TLB cache. Every. Single. System call. That's why its a big deal for servers, but not a big deal for video games.

" Meltdown / Spectre patches have virtually no effect on video games."

The amount of speculative-execution that games rely upon for many of their components is insane. With the patch applied, I went from a solid 60FPS at 1440x900 in GTAV down to about 48 FPS (as measured by the Steam FPS counter) on a 7th-gen i3.

I wonder if moving to a different file format for compilation might minimize the impact of this? For example, what if source code were stored as a single SQLite file? What impact would that have?
If anything, it'd be slower. SQLite locks the database file (which would be another OS-level system call).

Simple files are more straightforward. You do one MMap and then there wouldn't be any other system calls made. I'd bet that compilers are "slow" because they have to make many, many, many mmaps to do anything. Each #include turns into another MMap, which Meltdown mitigations cause a full TLB flush each time.

Mmap uses page faults instead of syscalls to read more data. Both these mechanisms require switching to the kernel and back, so it's not clear to me why it should be less affected by Meltdown patches.
For reading many blobs of approx 10KB each, it is often faster to read them out of an SQLite database than from separate files on disk. See https://sqlite.org/fasterthanfs.html for details and to download code that you can use to test this yourself on your system.

I don't know if SQLite would be faster or slower in the context suggested here. But I'd like to suggest that perhaps it is worth running the experiment before reaching a conclusion.

In the benchmark you sent (https://www.gamersnexus.net/images/media/2018/cpus/2700x/rev...) the Intel only beats the Ryzen on the Civilization game overclocked to 5GHz. In Stock clock it have the same results as Anandtech
No, the 8700K also beats the AMD cpus in every other game they benchmarked, even without overclocking. Including the benchmark image you linked to?

I got word on reddit that pc-per said they did apply the spectre/meltdown patches for their benchmarks. https://www.pcper.com/reviews/Processors/Ryzen-7-2700X-and-R..., the results are as one would expect.

There was some speculation back when the first Ryzen came out that Anandtech (among other reviewers) were pressured to not do the gaming benchmarks at all, or to do them last, since the gaming performance was so abysmal. I think they delayed those numbers for a few months.

It's very suspicious when Ryzen 2 numbers are that far off and whilst they keep getting all this amazing access to AMD for trips and CEO interviews and new toys.

Anandtech has nothing to gain if they're the only ones with invalid benchmarks.

Edit: If the comments on the AT article can be believed, then Toms Hardware's Intel benchmarks didn't include the latest Meltdown/Spectre firmware patches because they didn't know that the X470 platform had them to begin with.

> What is anandtech doing different?

AMD's officially supported memory is 2933 MT/s. Intel's officially supported memory is 2600 MT/s.

Other review sites overclock memory and run both systems at 3200 MT/s RAM (closer to reality: you may have to spend $20 on RAM but the faster speeds seem to be worth it these days). It seems like Anandtech prefers to run the system on default settings however.

Its definitely a discrepancy. But I don't necessarily think Anandtech is in the wrong. It just goes to show how much these little decisions matter in benchmarking, and why its good to have multiple benchmarks from different sites.

So, do you go for the "obvious overclock" with 3200 MT/s RAM, or do you actually follow the official recommendations from the manufacturers? Regardless, its good to see that different websites have tested the two different cases and leaves it up to the reader.

See this line: https://www.anandtech.com/show/12625/amd-second-generation-r...

> As per our processor testing policy, we take a premium category motherboard suitable for the socket, and equip the system with a suitable amount of memory running at the manufacturer's maximum supported frequency

Personally speaking, I think running RAM at an overclock at 3200 MT/s is more fair to both systems. Both systems achieve 3200 MT/s easily, with Intel overclockers often hitting 4000MT/s or more (so really, Intel has the better memory controller). So its just... odd... that AMD would be equipped with better RAM.

But Anandtech's approach is reasonable (although they probably should list the timings of their RAM in their setup page). After all, most computer builders probably don't know the finer overclocking details between the systems, and may instead just go for the officially supported numbers.

> Personally speaking, I think running RAM at an overclock at 3200 MT/s is more fair to both systems

at that point you're comparing a system that AMD doesn't sell to a system Intel doesn't sell

I like that they went stock.

I build fast machines but I don't think I've ever overclocked one, I certainly can't remember.

I value stability and longevity, my 2500K lasted til end of last year (it still works fine, it's just my laptop is faster).

A lot of modern RAM can hit 3200 MT/s within the specified voltage settings. RAM literally has gotten better over the past few years, but "official" JEDEC timings lag behind.

Its why Intel has XMP: so that your sticks of RAM will talk to the motherborad and automatically overclock themselves to factory-tested values these days.

These "easy overclocks" I'm talking about are very simple! The factory gives you an XMP profile of the RAM under specified conditions, and then the motherboard enables a checkbox for XMP-profiles in the BIOS.

No need to tweak timings or anything. These days, overclocking RAM is a question of "Do you want to run your RAM at the speed tested at the factory" ?? Or would you rather run RAM at speeds specified by JEDEC from 2014?

Every RAM manufacturer worth a darn has an XMP setting, where they've thoroughly tested the RAM at these higher speeds. You might as well spend a minute, click the button in the BIOS/UEFI prompt and accept the free performance benefit.

FWIW, SDRAM transfer rate isn't really a big performance driver. The bus is idle almost all the time on modern parts, which are dominated by the precharge/read-write cycle and not transfer. When the 512 bits of data actually arrives, it gets blasted out very fast. Shrinking that transfer time has a measurable effect, but it's not remotely linear in the clock frequency.
Its certainly non-linear, and you're right that its dominated by precharge / read-write cycles.

But DDR4 RAM has support for 16 banks per stick! A CPU with two memory controllers can have 64 outstanding requests to DDR4 RAM simultaneously (16 per stick, x2 from the chip select, x2 for the 2nd memory controller).

And since modern cores are highly multithreaded, and also have prefetchers and whatnot, I'm sure the modern DDR4 bus is quite active.

Perhaps it doesn't matter too much for single-threaded situations (single-threads only benefit from prefetch and out-of-order executions). But a Ryzen 2700x is a 8-core / 16-threaded beast, which will surely eat up all of those outstanding DDR4 requests. Intel's 8700k is 6-core/12-threads as well, so they'll rack up those memory requests and keep the bus busy.

And if each of your Assembly commands are AVX (256-bit requests per instruction)... well... that's going to be a busy bus.

RYZEN seems to like fast memory more than Itel one review I just looked at said that the new RYZENS are targeting 3400
This is because Ryzen's "infinity fabric" or internal interconnect between the cores is tied to the RAM speed.

This is kind of like the old days when CPUs used a Front-side Bus (FSB). If you increased the FSB frequency it would increase the CPU frequency, the RAM frequency, and even PCIe depending on how it was wired up. It made things simpler to have the FSB be the sole clock-generator.

So despite the increased bandwidth moving from 2600 to 3200mhz RAM not having much impact on performance (in most applications), the increased frequency of the bus between the cores allows for greater performance beyond that.

IIRC Intel uses a ring-style bus with it's own clock, hence why it this effect only applies to AMD.

Yeah, up until 3200 MT/s the IF has a lot of impact.

> the old days when CPUs used a Front-side Bus (FSB)

They still kinda do. It's not the bus, but there is one clock called BCLK (base clock) now and almost everything is derived from it. It's not adjustable on cheaper boards, but with a good board you can absolutely do the same adjustment :)

Yeah I have an older box running an i7-920 which is overclocked from 2.6 to 3.6ghz with the base clock.

Sandy Bridge and newer get extremely unstable when you toy with the base clock more than a few MHz though. You're basically stuck shelling out an extra few bucks for a "K" chip if you want to overclock on Intel now.

Interesting for developers is the improvement in compiling Chrome, and also the big hit from the Meltdown/Spectre patches on Intel on the same benchmark.
I held off upgrading my personal desktop last year (and built Ryzen 1700 for work) then meltdown/spectre hit and I decided to wait for the 2700X, I'm glad I did more performance at less money and without the hit that Intel took, I'll likely build an R2700X in a month or two when any teething troubles have had time to hit the wire.
Generally raising a processor's clock frequency will decrease its IPC as it spends more time waiting for main memory, so it isn't surprising that the overall IPC only went up 3% despite the various cache optimizations. I wouldn't have actually been surprised if it had stayed the same. But I'm frankly sort of amazed that they've managed to go to having consistently lower cache latencies than Intel has. I always thought that Intel's greater willingness to throw engineer-hours at optimizing layouts would mean they'd always take less clock cycles to hit their caches.
I don't think IPC tests involve main memory I/O.

Edit: I missed that the benchmark methodology is on a different page from the results.

An IPC test is basically running a benchmark while locking the processor's Frequency (through overclocking tools). That way, you get to test the architecture without dealing with all the Turbo-boost stuff on modern chips.

Memory latency / bandwidth is a piece of any benchmark. Some benchmarks are purely CPU (Linpack), but others are better overall tests of the system (ex: Cinebench or Blender)

IPC is always going to be a number that is dependent on your particular workload. Searching through a huge linked list laid out randomly in memory is going to measure just your memory latency. An optimized media codec might tend to just measure your execution resources. But really you want to measure all sorts of additional things like the out of order engine depth, the strength of the branch predictor and prefetcher, and the latencies and sizes of all the different cache levels. And it's hard to get a good measure of all the stuff within a chip and especially stress the branch predictor and prefetcher without having a working set that sometimes spills over into main memory. And really you want to include the efficiency of your memory controller in your rating of the chip too.

But if you have a particular workload in mind some of these aspects might not important to you and you'd be best served by just benchmarking your particular application on a given chip.

(comment deleted)
So it seems for now that everything is consistent with the benchmarks published a month ago in a paper magazine in France: https://www.cpchardware.com/cpc-hardware-36-debarque-en-kios... (french)

What did they do differently ? They didn't ask AMD for permission or sign an NDA for exclusive "early" access, and just sourced the parts on ebay from one of the numerous samples that were given to ODMs/OEMs.

Mind providing a little info on what it was they were saying that was different than the rest (which is what I inferred you are trying to express)?
I might not have been clear: everything they benchmarked is identical to benchmarks that are being published now.

They just had a big head start because they acted independently and didn't wait for vendor approval to publish their tests.

Edit: here's an english source with the magazine contents: http://www.guru3d.com/news-story/cpc-hardware-amd-ryzen-7-27...

Ah. To me it sounded like you were saying their benchmarks were more "honest" and didn't contain some bias the others did because they bypassed the NDAs. Thanks for clarifying!
Have the meltdown/spectre patches really had a roughly 30% effect on compilation performance on Intel CPUs? The i7-8700K went from 22 compiles/day to 14.

https://www.anandtech.com/show/11859/the-anandtech-coffee-la...

https://www.anandtech.com/show/12625/amd-second-generation-r...

The fact that the i7-8700K shows up as slower than the i7-8700 now is a bit weird too.

> The fact that the i7-8700K shows up as slower than the i7-8700 now is a bit weird too.

It's impossible. The 8700K and the 8700 are the same chip, just that the 8700K has an unlocked multiplier and a higher default clock[0] - better binning. The 8700 will never be faster in any application.

[0]: http://www.cpu-world.com/Compare_CPUs/Intel_CM8068403358220,...

It is improbable, but there are many reasons why a specific 8700 might outperform a specific 8700k. There is a factor luck of the draw involved in the specific chips they test with.
Can you name some reasons? I thought concepts like luck of the draw in this context normally translate to silicon lottery and are meant to describe overclocking potential. But there is no reason why a chip sold as one of those cpus should not be able to reach the specified turbo clock, and in that case the 8700 just has to be slower in such a test.
The only thing I can think of is maybe the 8700 has better binning and is able to hit its turbo speed while putting out less heat and so is able to maintain its turbo speeds for longer than the 8700k.

I think it is more likely to just be testing margin of error

One reason might be that manufacturer can sell chips that would qualify as 8700k as 8700 simply to meet the 8700 demand if they have more good chips than they can sell at the higher price point.
I remember this happening for chips from the first step revision of Nehalem.

When the demand for the i7 920 was way up, there were a bunch of better chips sold as such.

Savvy overclockers knew which batch numbers they were and sought the particular runs.

Unlocked multipliers don't mean you've chosen the optimal multiplier. And something binned for higher clock rates doesn't necessarily mean it runs cooler at the same clock rate as another part. As others have mentioned, thermal management is a real issue here. A throughput benchmark doesn't care about brief bursts of speed, but rather sustainable rate.

Finally, even with identical parts, the assembly can affect performance. The same heat-sink installed slightly more effectively in terms of the mechanical interface (no contamination, just the right amount of heat-sink compound, no air bubbles) will allow the chip to run at higher power without any thermal throttling.

I honestly would have thought that the binning/testing done on those processors would make sure that the difference always go into the same direction, as in more performance for the higher specced part. Because I assume that in a benchmark just as this the cooling is sufficient to keep the turbo clock indefinitely, meaning that it makes no difference if the 8700 without limitations would beat the 8700K. But what you and the other say sounds convincing, maybe scenarios like this can happen in practice more easily than I thought.
It's a common real-world issue that someone buys the "faster" CPU option in a laptop and finds it runs slower than their coworker's because the thermal design and/or quality of assembly causes it to overheat and throttle.

With many systems, turbo is only a temporary burst benefit for a lightly loaded system. Many server or HPC systems would intentionally disable turbo to have consistent sustained throughput.

Definitely. But this is not a laptop with cooling issues, in this case we are talking about a professional benchmark setup. If cooling was an issue during this, it would just add an other negative point to the bunch of irregularities in this review.
There'd be something like a 2% error margin in their timings. They can only report what they measure. It's actually a good sign such discrepancies show up in their data; it means it is likely real data.
Here's a different benchmark (video rendering) that still places all the 8th gen Intel chips above Ryzen 2:

https://www.pugetsystems.com/labs/articles/After-Effects-CC-...

I assume these tests are unpatched Intels though. Since my use case for CPU is video rendering and not gaming, I still don't have a reason to switch to Ryzen, as much as I want one. Threadrippers are still way out of my price range (~$1K) and are about the same performance as top tier i7 8th gen.

Isn't 3D rendering a scene like this something everyone would just use a GPU for?
No, you can't always do this (e.g. Blender Cycles does support CUDA and OpenCL, but only if you don't use their shading language thing), and it's not always better (e.g. according to Gamers Nexus, a Threadripper 1950X beats a GTX 1080 Ti significantly on their test scene)
Adobe does a terrible job with multithreaded chips in general, it seems. Instead of investing in faster chips, is there faster software you can get for a couple hundred? Ex-Lightroom user here; got way more performance by changing tools than I ever could have with better hardware.
Ultimately if all you need to use the CPU for is After Effects CC then follow the results of that benchmark (or rather, follow the performance/$ for that specific benchmark). Otherwise, in general the AMD CPUs give more multi-core performance/$ than Intel, for example, in any multi-core x264 based test I've seen. And it makes sense why that is so, the AMD CPU has more cores and more cache for the same $. And that's only comparing CPU prices and performance, if we consider the motherboard prices then the needle moves further into AMD's favor.

As to why After Effects CC seems to prefer Intel so much would indicate some specific optimizations Adobe performs, optimizations that seem to favor Intel for now, not general raw multi-core performance of the Intel CPU.

For single core performance the situation is reversed, the higher clocked (but lower core count) Intel CPUs will obviously have much better single core performance which is important for games as even today they tend to be coded relying on a few threads.

> As to why After Effects CC seems to prefer Intel so much would indicate some specific optimizations Adobe performs, optimizations that seem to favor Intel for now, not general raw multi-core performance of the Intel CPU.

I bet its AVX-512. Which is "Intel Specific" but that's the entire point of AVX-512, a new Intel specific instruction set that has high performance benefits.

Overall, AMD is betting on general purpose cores with at most 128-bits per operation. AMD supports the AVX 256-bit instruction set, but its "emulated", as the ALU pipelines are 128-bit (granted: AMD can gang together two ALUs in a 2x128-bit configuration to execute the 256-bit instructions)

Intel supports 3x256-bits per operation on their CPUs, with the 512-bit AVX512 instructions using two execution ports at a time.

So for any code that's written to be SIMD-heavy, Intel machines will likely be faster. Not only because of AVX512, but also because Intel's AVX-256 bit instructions are better supported compared to AMD (3x256-bit for Intel, vs 2x128-bit for AMD)

------------

With that being said: the benefits of SIMD approach Ahmdal's law just like anything else. In particular, AVX512 uses so much power that Intel cores are forced to downclock.

So AMD can run more cores, at higher frequencies. But Intel's cores do more work per clock tick and per unit energy. It seems like AMD's 16-cores (each doing 2x128-bit) are only slightly slower than Intel's 10-cores (each doing 3x256-bits) in those SIMD-heavy tests.

> (3x256-bit for Intel, vs 2x128-bit for AMD)

It's not quite that harsh for workloads that aren't entirely FMA. Zen has 2x128-bit adders plus 2x128-bit multipliers.

Fair enough. The full AMD pipeline contains 4x 128-bit execution ports. But this is a bit complicated, so I was ignoring that part of the core.

IMO, the more important bottleneck is the load/store unit. Intel's system has 2x 256-bit loads + 1x 256-bit writes to the L1 cache.

While AMD's system has 2x128-bit load+store units for its L1 cache.

Thus my numbers above: 3x256-bit for Intel (2 reads + 1 write), while 2x128-bits for AMD (2x read-or-write).

would it be worth it starting on $99 and upgrading to Ryzen 7 later on.

I was going to do that with AM3 but then they dropped the socket. and the upgrade path is quite big here if it's all on the same socket.

well I actually have a ryzen 1500x and I'm happy with it. I mean I actually looked into 2400g and maybe upgrade/downgrade to it because vega is built-in and probably good enough for my use case (and not as loud as my R9 280)
Careful that gpu support is shaky at best under Linux (things should improve with kernel 4.17)
well my mainboard is shaky with linux support also... somehow the chipset does not love it :( (b350m)
I've been hearing that for a decade or so.
AM3 and AM3+ lasted quite a while - used from new CPU's 2009-2014, which were the high end desktop sockets from AMD until 2017.

In the same time Intel went through 4 sockets: LGA 1156 -> 1155 -> 1150 -> 1151

5 actually, there's two versions of the 1151 and they aren't compatible.
It's super weird that there weren't any RTL changes in the core. Like even in the process part of tick/tock, you can normally sneak in some fixes. And it's not like you can put those RTL folks on optimizing mask layouts; it's a totally different skillset.

Are the uarch people at AMD running around with their hair on fire trying to fix all their Spectre/et al. bugs, and skipped a release to focus?

I think it was speed boost2 that was / is the main change and also its a process shrink to 12nm
Sure, but the question is, what are the other half of their staff doing of not working on this release?
Playstation5, Navi GPUs, XBox's next processor, Ryzen3, 7nm Vega. There's rumors for DDR5 memory controller and MCM reaching the GPU space somehow.

AMD's plate is rather full. They're juggling a lot of cool projects despite having way less money than Intel or NVidia. Zen+ was always going to be a "low effort" minor update.

I'm frankly surprised that this minor update got much of a boost at all. Its a good sign for Zen2 next year.

I've been very happy with my 1700. There were some initial POST problems from a memory mismatch (they are very picky, it seems) but once that was sorted out, it's been terrific. Video encoding goes much faster than it used to. No cooling problems with the included AMD heatsink & fan.

The questions for me are: Is it worthwhile to upgrade? And will my old memory work in with the new CPU? (my guess would be "no")

It is not worthwhile to upgrade. The difference between the Ryzen 7 1700 and the Ryzen 7 2700 is small, the slightly higher clock and performance improvement does not warrant an upgrade. The new processors are just a new nice option if you are not already on a current cpu, like an old FX or a dual core intel.

Your old memory will work with your new cpu, to 99%. AMD improved memory compatibility with software updates, and the memory support in the 2000 line is supposed to be better again. If your ram is slower than DDR4-2666 you lose some performance compared to what you could have, but that was already true for the 1700 - DDR4-3000 is a good minimal ram speed target, as Ryzen profits a lot from being combined with faster ram.

Just a note these chips don't have hardware mitigations for Spectre and Meltdown. The first Intel and AMD chips that will have had the time for them to be implemented to offset the performance losses will be coming out next year (Ice Lake and Zen 2 respectively). That being said, like Ars mentions in the article, AMD chips are in general hit less by the OS patches than Intel ones.
For start AMD chips are nor vulnerable to Meltdown, so the impact of mitigations is smaller as it only addresses Spectre.
Is there any info if linux compilation segfault (kill-ryzen) has been fixed?

Edit: and what is ECC status? Still not officially supported?