Launch HN: Lyrebird (YC S17) – Create a digital copy of your voice
We are the co-founders of Lyrebird (https://lyrebird.ai/) and PhD students in AI at University of Montreal. We are building speech synthesis technologies to improve the way we communicate with computers. Right now, our key innovation is that we can copy the voice of someone else and make it say anything. The tech is still at its early stage but we believe that it is eventually going to make possible a wide range of new applications such as:
- reading loud text messages with the voice of the sender,
- reading audiobooks with the voice of your choice,
- giving a personalized digital voice to people who lost their voice due to a disease,
- allowing video game makers to have more customized dialogs generated on the fly, or avatars of their players,
- allowing movie makers to freeze the voice of their actors so that they can still use it if the actor ages or dies.
Yesterday we launched a beta version of our voice-cloning software: anyone can record one minute of audio and get a digital voice that sounds like them.
We know that many on HN are concerned about potential misuses surrounding these technologies and we share your concern. We write further on our ethical stance on this page: https://lyrebird.ai/ethics/.
Our blogpost about the launch: https://lyrebird.ai/blog/create-your-voice-avatar that features the first video combining generated audio and generated elements of the video.
There was a thread about us on HN when we launched our website four months ago (https://news.ycombinator.com/item?id=14182262) but at that time, no one could test our software yet and we did not really answer any question of the community. So this time we are ready for questions and would love some feedback!
98 comments
[ 2.7 ms ] story [ 155 ms ] thread>> reading audiobooks with the voice of your choice, AND the speed of my choice.
It's not even necessary to copy anyone's voice, as long as there's a selection of the most comprehensible and human-sounding ones.
Then, you could even automatically generate slideshow presentation from a few illustrations and headlines, and that would make "rendering" articles into videos very fast and easy. I'm sure a lot of people would pay for such service.
----
By the way, recently I've encountered Deep Voice 2, a similar research project by baidu:
http://research.baidu.com/deep-voice-2-multi-speaker-neural-...
Results are very impressive.
It would have been an order of magnitude better if I could just generate arbitrary phrases in his voice.
(Or maybe not; maybe the constraint made the video better.)
https://www.youtube.com/watch?v=gXTrrTX7YuY
are you open for beta? would like to try out your api on education content.
Our upcoming versions should be more robust to different accents and we also plan to extend it to other languages.
Vanguard allows voice authentication (https://investor.vanguard.com/account-conveniences/voice-ver...) - and who knows who else will roll something similar out in the future. Yeah, its really really dumb, but it's happening in production now. I wouldn't use this product if I were you, but honestly you should also not use voice verification/authentication for anything.
Quick q's (purely out of curiosity):
1) > We are [...] PhD students in AI at University of Montreal
Are you doing the startup on the side/planning on going back to school?
2) I don't recall reading about you guys in articles about YC S17 demo days. What are reasons why some companies might not participate in demo day or remain off-the-record? In your case, you seem to have had a working product long before demo day
1) The research of the PhD and the startup are quite complementary at the end, so we hope we can continue doing both.
2) We didn't do demo day because we raised our seed round just before YC and did not want to raise again.
What would be your use case?
I see lyrebird api being very helpful in helping my users practice listening skills and add a level of creative fun! If we had 10-20 different voices, the flashcards will be read a little differently each time. Right now (since our flashcards is dynamic), our audio feels very monotone. We would love to help you beta test your API and work something out.
Our current production process requires a group consisting of editors, readers and technicians to get together every Friday morning from 7am to record an hour or more of news which is then mastered onto CD, duplicated hundreds of times and mailed out by 11am.
We usually have four readers each week (from a pool of 30 or so) who take it in turns to read the items. Some readers are better than others and sometimes readers don't turn up. Sometimes there are interruptions or disturbances to recording such as another reader in the studio coughing, rustling of papers, etc.
If we had the ability to digitise the voices of our readers it would enable our new (in development) totally digital production and distribution system (podcasts, streaming, etc.) to be produced at any convenient time and to allow our listeners to choose their preferred reader's voice(s).
The studio software side is using FL/OSS software, with Ardour as the digital audio workstation attached to a Delta 1010 digital input system and an Evolution UC33e control surface.
Being able to program the pre-production phase to generate the audio recordings using favourite readers voices which are then fed to the (automated) studio mastering process would give us some amazing functionality and flexibility to produce programmes on-demand with no studio presence required.
The development experience and final package will be documented and published for other talking services to adopt and adapt.
Perhaps you should increase the minimum from 30 recordings to 100?
One thing: I found that I was in such a hurry to record that I probably spoke faster than normal. It'd be nice if there was a way to tune a few parameters manually (tempo, pitch, etc).
If I ever lose my voice and have to have a TTS appliance speak for me, I'll be contacting you all to get my voice profile!
EDIT: For those interested, pretty impressive that it figured out the appropriate cadence for this: https://lyrebird.ai/g/v7MpYaUA
> It'd be nice if there was a way to tune a few parameters manually (tempo, pitch, etc).
Yes we are currently exploring ways to control the generation: volume, pitch, tempo, speed but also intonation and emotion.
Also, https://lyrebird.ai/g/D3Fw328D
Some other people shared their voices on twitter if you want to compare: https://twitter.com/LyrebirdAi
500 internal server error Sorry, something went wrong at Lyrebird!
If this error persists, please contact us.
> Q: Will I be able to copy another person's voice?
> A: Yes but only if you have the authorization of the person whose voice is being copied.
Perhaps you can unpack that answer a bit? What's the authorization process?
There will be two scenarii:
- you want to use the voice of someone that has a Lyrebird account: he or she has to give you their authorization.
- you want to use the voice of someone who does not have an account. We have specific contracts for that. Say you want to copy the voice of Morgan Freeman, the contract will be between him/her, you and Lyrebird. We will also probably explore alternative ways for that.
Just to be clear, the license of the voice/digital voice is revoked upon deletion of the recordings? I understand it is subject to the biometric agreement, but the words perpetual and irrevocable still worried me. Thanks!
We delete all the recordings when you click delete, so we can't recreate the voice anymore. However, this is still necessary in case we share some generated sentences in social media or so (like we're doing on twitter now).
This is something that you should only do with the permission of the user who provided the voice. You don't need generalized permission to do that for every user, and given the nature of the technology, you shouldn't ask for such permission.
> This is what our lawyers suggested to protect ourselves.
Generally speaking, a lawyer's advice is going to be optimized for maximum protection in possibly unforeseeable circumstances, not for what might actually be needed or even reasonable to request of every user.
Generally speaking, companies aren't going to go out of their way to rein in their lawyer. Most people won't even read that fine print, unfortunately.
Random question: it's said that people think their own voices sound weird when they hear recordings of themselves played back. Do you have a way to measure that phenomenon? Have you seen people complaining about the accuracy when in fact it was just that effect making people sound "weird" (to themselves)?
Other interesting observation are the sentences that people generate for the first time with their digital voice...
The end result of this is that your own voice, when recorded and played back to you, will generally sound less bassy and more harmonically rich than you expect it to.
And thanks, we are going to update the instructions to make them more clear.
To recap:
- we want to start by raising public awareness about the technology and we did demos with the voices of Trump/Obama for that,
- your digital voice is yours, people can not use it without your authorization.
- hacking voice-controlled interfaces
- generating fake news
FTFY
don't @ me saying "sure any technology can be used for good and bad stop being a ludite" yeah I know that just messing with you
We write more about it here: https://lyrebird.ai/ethics/
First uses that come to mind are players adding themselves to a VR world - or maybe celebrities / public figures.