This is awesome. I think the title is misleading though; it's about reproducing digitized speech. This converts the already digitally represented sound to actual analog sound, audible by humans. The output device is a by cleverly (and mechanically) played classical piano.
The resulting voice sounds like a bad special effect, but is really cool since it isn't. :)
There doesn't seem be any fitting involved. I think he just took the speech spectrogram and used it as a piano score. Thus the acoustic output of the piano is not meant to approximate the acoustics of talking. We hear speech nonetheless because there's enough of a speech-like signal for our auditory functions to hold on to.
I find it particularly interesting that words that end in hard consonants (like T or D) are represented by very high-pitched keys to create the percussive sound (especially noticeable at the end of the video). This is too cool.
On another subject, I want some kind of 'piano player' with pistons like this piano, and you attach it to a non-MIDI instrument and it plays the keys via MIDI data, without having to retrofit it electronically.
However, if you don't look at the subtitles it's quite hard to recognize the words. Showing the subtitles of what you're supposed to hear is called 'Prompting', an effect which is also used by backward-speech advocates.
If you read up a bit on how mp3 compression and GSM work then you'll find that in essence they rebuild the sound from a very limited number of frequencies.
Very nice find, and extremely impressive the speed with which the keys are manipulated.
I wonder if it is possible to have a human learn the key sequences of the dominant frequency bands and make a piano speak a word or two.
In neuromancer there is a head in villa straylight that speaks with organ pipes iirc.
> I wonder if it is possible to have a human learn the key sequences of the dominant frequency bands and make a piano speak a word or two.
I was wondering that too. Imagine if a composer in pre-digital history had figured out something like that and as a final "chord" to a piece had let the piano speak a word.
Seeing as it is on MAKEzine I am upset there are not plans to build your own... Seriously that would be one hell of a weekend(s) project. Writing the software/midi stuff sounds fun too!
It's fascinating to listen to but I wonder about extending it to other languages. Does American English lend itself well to this sort of thing or are other languages and accents reproducible. I imagine that languages with much harsher stops like German or Russian would be more difficult and those that are easily broken down, like Japanese, would be even easier than English.
30 comments
[ 3.0 ms ] story [ 74.8 ms ] threadThe resulting voice sounds like a bad special effect, but is really cool since it isn't. :)
For example we can:
Hear a song we recognize, playing softly in the background, in a crowded noisy room
Talk and understand conversations over the telephone even with it's amazingly small bandwidth
Understand speech with something like 70% signal loss
Hear the cry of a baby in the middle of a battlefield over the sound of guns
and on and on and on
On another subject, I want some kind of 'piano player' with pistons like this piano, and you attach it to a non-MIDI instrument and it plays the keys via MIDI data, without having to retrofit it electronically.
Your dog reads books!? - Yea, but only fiction.
Your piano can talk!? - Yea, but not very well.
http://www.scholarpedia.org/article/Sine-wave_speech
Very nice find, and extremely impressive the speed with which the keys are manipulated.
I wonder if it is possible to have a human learn the key sequences of the dominant frequency bands and make a piano speak a word or two.
In neuromancer there is a head in villa straylight that speaks with organ pipes iirc.
I was wondering that too. Imagine if a composer in pre-digital history had figured out something like that and as a final "chord" to a piece had let the piano speak a word.
Literally awesome.
Anyone?
http://ablinger.mur.at/docu11.html#qu3