I wonder if you could use this to do extremely low bandwidth video conferencing. Send a network configuration at connect time (relatively expensive) but for each frame, send the convolution network parameters, and deconvolve at the receiving side. Presumably this would result in far less data over time than sending the whole image. Sort of face-specific compression.
Possibly, but one thing the article doesn't address is the computational costs of doing the convolution. Because you have to store an intermediate representation of the image for every convolution kernel that you use, you start to eat up memory in the GPU. I think processing should be quick enough that you could generate animations in real-time, however.
What the OP described would be extremely low bandwidth -- you only need three parameters to describe what to draw per keyframe (maybe one or two more if you want to describe mixtures of emotions or identities or more complex things), whereas you'd need more to describe 3D animation.
It's not a huge benefit over what you're suggesting (realistically, when are your bandwidth constraints ever that strict?) but it's an interesting idea.
Ha, very similar to the original intention of the vocoder!
That Kraftwerk/Daft Punk voice, the original idea developed by Bell Labs was to come with ways to compress voice signals maximally by representing them only by the envelope necessary to modulate a bank of oscillators. Like any codec/compression scheme, the basic idea is to come up with a simple set of basis vectors that can represent the same data in a minimal fashion as long as it can be reconstructed on the other end.
"The method specifies the speech signal in terms of its short-time amplitude and phase spectra."
In the case of machine learning, the idea is simply to come up with those basis vectors empirically and automatically instead of reducing them to a mathematical model such as "set of sinusoidal waves".
I'm sure I've read sci fi that uses the idea - they have extremely limited bandwidth so they make up for it with aggressive compression and reconstruction that more or less reduces people to a handful of parameters and uses those to animate an avatar. It relies on computation being a LOT cheaper than bandwidth.
I think Verner Vinge's A Fire Upon the Deep describes a teleconferencing system based on avatar models combined with the receiving side's video image history of the sender. It really had nothing to do with the story, other than a technical detour about ship-to-ship communication. The novel also had a galaxy-spanning Usenet-like system that was part of the story. :)
Just happened to have read some fiction set in WH40K universe and... now I know what exactly a mutant possessed by Chaos looks like. This makes a perfect illustration.
There are companies trying to use this type of thing for game content generation. I recall some guy at tech crunch disrupt won an award last year (two years ago?) for something along these lines?
Very nice. Scanner Darkly.. and also reminiscent of the images derived from the thought reading experiments where they blend youtube video frames that are most similar to the activation states of the subject thinking about that image... and more like the sgd images, not the agd ones.. interesting.
23 comments
[ 0.20 ms ] story [ 53.1 ms ] threadIt's not a huge benefit over what you're suggesting (realistically, when are your bandwidth constraints ever that strict?) but it's an interesting idea.
That Kraftwerk/Daft Punk voice, the original idea developed by Bell Labs was to come with ways to compress voice signals maximally by representing them only by the envelope necessary to modulate a bank of oscillators. Like any codec/compression scheme, the basic idea is to come up with a simple set of basis vectors that can represent the same data in a minimal fashion as long as it can be reconstructed on the other end.
http://www.bell-labs.com/newsroom/publications/290623/
"The method specifies the speech signal in terms of its short-time amplitude and phase spectra."
In the case of machine learning, the idea is simply to come up with those basis vectors empirically and automatically instead of reducing them to a mathematical model such as "set of sinusoidal waves".
It's almost too easy, really...