I sometimes wonder if it would be possible for CPUs to have, in addition to their full speed cores, a small part dedicated to parallel tasks with drastically lower speed, but a very high number of cores. Like a tiny graphics card, but with an x86 architecture and less driver problems.
Something like that, but with more cores and cheaper.
It looks like there are up to 64 cores in Xeon Phi at 1.5Ghz (price > $1000). A Nvidia 1030 is still better with more than 300 cuda cores at 1.4GHz (price ~= $150). Not that I think that cuda cores and Xeon Phi cores can be compared, but 64 is not that much at that price... What we'd need is dumber cores in very high number. We don't need all fancy things that normal CPU cores have, like virtualization.
Note that Nvidia's core counts are misleading. To apply Nvidia's logic to Xeon Phi, a 64-core Xeon Phi is really 512 cuda cores--each SIMD lane is counted as an independent "Cuda core".
I wish we could talk about GPUs and similar architectures in terms of their total FPU width—how many 32b or 64b floating point operations can be done per clock— and as a secondary measure the number of independent instruction streams that can be worked on in parallel. One provides a upper bound on the arithmetic power of the chip and the other predicts how easy it will be to get close to full utilization of those compute resources. Distilling an architecture down to just one core count only guarantees that you can't compare it against dissimilar architectures.
Integrated gpus are almost exactly that, minus the x86 part. A very high number of x86 cores would take a ton of chips space and generate a lot of heat.
I've wondered the reverse, why GPU manufacturers don't replace the whole current motherboard, CPU and memory setup and create their own all in 1 PC that runs on the graphics card.
In a similar vein to de-noising realtime path tracing, Christoph Shied has an interesting paper that describes an algorithm that steps around the problem:
https://cg.ivd.kit.edu/atf.php
Intel's graphics libraries are well crafted and pleasant to integrate. In fact, Embree is currently being integrated into Blender's Cycles as an alternative ray engine.
I'd guess we'll see an effort at integrating the Intel denoiser fairly soon as a plugin at least.
Someone recently released a Blender plugin using the NVIDIA Optix denoiser, which I think couldn't be directly integrated due to licensing issues:
For the example picture, it would be nice to have it also compared to a fully raytraced image. I'd love to know how many rays it would take to be equivalent to the denoised version.
What is the difference that makes the algorithm not be applicable? Is it that path tracing noise is "pure monte carlo noise" (I'm sure that has a better term) so has nicer properties from photographic sensor noise (thermal noise with various patterns etc)?
There are two huge factors, the first even more potent than the second.
First, there is much more information per sample and per pixel. Normals, depth, separate diffuse color, separate diffuse texture, first bounce normal, first bounce position, etc. There is a technique that just uses color, but uses histograms of all the samples within a pixel.
Second, for a video game you can accumulate samples temporally. Many of these demos will use one sample per pixel, but filtering information can be kept on every frame and reused to some degree.
This is trained on monte carlo based rendering noise, and optionally uses additional available rendering data like albedo and normal passes. So while the idea of using deep learning for noise reduction in general works, this codebase in particular isn't suitable for that task.
One issue is also building large enough training data set for clean vs noisy images. Approaches like https://arxiv.org/abs/1803.04189 try to work around that
Other approaches (like Google's pixel cameras and astrophotography) additionally take high speed burst photos and average pixels to reduce noise, this can be used together with deep learning based noise reduction (or other noise reduction techniques) for better results
32 comments
[ 5.1 ms ] story [ 68.4 ms ] threadIt looks like there are up to 64 cores in Xeon Phi at 1.5Ghz (price > $1000). A Nvidia 1030 is still better with more than 300 cuda cores at 1.4GHz (price ~= $150). Not that I think that cuda cores and Xeon Phi cores can be compared, but 64 is not that much at that price... What we'd need is dumber cores in very high number. We don't need all fancy things that normal CPU cores have, like virtualization.
I don't think it's likely, but I wouldn't be overly surprised if they end up doing something like that.
https://hackaday.com/2018/02/17/catching-the-pcie-bus/
https://en.wikipedia.org/wiki/Larrabee_(microarchitecture)
and his Quake2 demos (which are amazing) have gotten a lot of attention on youtube recently: https://www.youtube.com/channel/UCdyHpbF9SgV8uMIWu3oHgdg/vid...
It should work on any CPU with SSE4.2 support, which includes all modern AMD CPUs.
I'd guess we'll see an effort at integrating the Intel denoiser fairly soon as a plugin at least.
Someone recently released a Blender plugin using the NVIDIA Optix denoiser, which I think couldn't be directly integrated due to licensing issues:
https://remingtongraphics.net/tools/d-noise/
[0] https://github.com/troopy28/OptiX-Denoiser-for-Blender
First, there is much more information per sample and per pixel. Normals, depth, separate diffuse color, separate diffuse texture, first bounce normal, first bounce position, etc. There is a technique that just uses color, but uses histograms of all the samples within a pixel.
Second, for a video game you can accumulate samples temporally. Many of these demos will use one sample per pixel, but filtering information can be kept on every frame and reused to some degree.
If it can’t be used on a still I rendered 10 years ago with code that isn’t around anymore then it’s clear it can be used on a photo either.
One issue is also building large enough training data set for clean vs noisy images. Approaches like https://arxiv.org/abs/1803.04189 try to work around that
Other approaches (like Google's pixel cameras and astrophotography) additionally take high speed burst photos and average pixels to reduce noise, this can be used together with deep learning based noise reduction (or other noise reduction techniques) for better results