That's a pretty cool showcase of modal [1]. From a marketing perspective I have to congratulate, this is a really well done way to get people to check out your platform.
Been using textgen and downloading tons of models, the models are all over the place. The problems of accuracy and short term memory are major issues that people are trying to implement work arounds.
Check out textgen, it has voice in/out, graphics in/out, memory plugin, api, plugins, etc, all running locally.
Nice to see Tortoise being used - I still think it's the best TTS system out there now. Generation time is slow, but quality is incredible. I wonder if the code can be optimised to speed up the generation, but I don't think the author is maintaining it any longer.[0]
Tortoise looks really nice! The output is very "polished" and audiobook-like. It's a contrast to Bark[0] which is far more expressive but unpredictable.
If you're generally interested in TTS check out Bondsynth AI, it's a product I've been working on for long-form text-to-speech (think ebook to audiobook or website to audio). It's still in beta, so it's free, but I'm looking for feedback.
The "API server" only runs when requests come in. Otherwise Modal scales everything down to 0, and you don't pay for anything. It's similar to putting Lambda behind an API gateway.
Is this the first time you've heard the term serverless? It means your code is plug & play, you don't have to manage the infra (I.e. a server) yourself.
I understand the concept of serverless as in AWS Lambda, but my first impression reading the title is serverless as in no backend. That would be a lot more amazing.
"serverless" is a throwback to 2013 a decade ago when it was a novelty to not know where your servers were or what they were made of, some people never move on. but in this case, this is a company advertising their serverless architecture for that specific crowd.
this is distinct from "client side" where no servers are involved, aside from maybe something that serves a static web page.
This is so good. I tried playing around with whisper.cpp last week and got absolutely terrible performance. I played around with the "step size" as well as "n" (not sure what these do, I should probably read the docs seriously).
I tried a lot of tweaks but this one is definitely the best I've seen.
How did it know how to spell my name correctly when I just spoke into my microphone??? The two L's usually trips up the transcription models. What????
This link is the most worthwhile hijack of an HN thread that I've ever seen.
How is tiny.en so damned accurate?
How much faster does this run natively?
Do the NEON instructions work for arm devices like an RPI or is it just tuned for Apple?
With this and Alpaca 13B you could probably replace an entire window manager.
Edit: it seems like it was less than a year ago that local speech recognition was a slow slog through a swamp of crufty, complicated research projects with most the high quality training data hidden behind walled gardens. Now I can stutter-step over a word and a demo in my browser correctly transcribes what I meant sans stutter. What happened?
Sorry if this is off-topic, but those are some really good book recommendations in the demonstration image! If those are coming from Vicuna, that speaks well of it.
I pitched this on a recently thread, but it was 12+ hours after it was posted, so I'll try again here.
What I really want is a program to waste the time of phone calls making unsolicited sales pitches.
It would do voice to text, run a simple language model to generate responses, then synthesize the voice back. It doesn't need to be a sophisticated model, not much more sophisticated than the classic "Eliza" program. A few years back someone did this with a canned loop of vague responses and it fooled the sales people for surprisingly long:
It seems like it could all run locally for low latency. Probably the most important part to get right would be a TTS system that isn't immediately pegged as a robot.
Yes, that was the youtube video I linked to. If that can be successful with some telemarketers, I'm sure a chat-gpt aided program could be even more successful at frustrating telemarketers.
Off tangent but can someone at Apple please just replace the Siri word recognition with whisper. We can finally have multi language support and not dogshit recognition.
64 comments
[ 8.5 ms ] story [ 169 ms ] thread1: https://modal.com/
What’s the state of container based ML deployments?
Can I take a container orchestration if these services and just put them on a vps w a GPU and run this?
Is there secret or just special sauce in ML infra?
It was a bit laggy, but for a free demo from an open source project, I should be the one being shamed!
Well done.
Check out textgen, it has voice in/out, graphics in/out, memory plugin, api, plugins, etc, all running locally.
https://github.com/oobabooga/text-generation-webui
Do you know how to get this working? I looked through the read-me and didn't see any options for it.
You need to enable the extensions.
I only did voice out locally with silero_tts, it also supports voice out with eleven labs api.
Voice input is via whisper tts.
[0]https://github.com/neonbjb/tortoise-tts
https://mycroft.ai/mimic-3/
[0]: https://github.com/suno-ai/bark
https://github.com/coqui-ai/TTS
Support for it was recently added to vocode:
https://github.com/vocodedev/vocode-python/pull/56
Demo at: https://www.youtube.com/watch?v=OmQup3kst5s
Signup at: https://signup.bondsynth.ai
https://en.wikipedia.org/wiki/Serverless_computing
It’s a cost-saving measure that costs you mortal wounds in integration expense, and eventually building out a real infrastructure plan.
I presumed the same as the person you're replying to.
As Michael Bolton said in Office Space: "Why should I change my name? He's the one who sucks."
this is distinct from "client side" where no servers are involved, aside from maybe something that serves a static web page.
https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_...
Compare it yourself.
It does all sorts of things better like upgrading words into proper nouns, and doing it based on context.
Honestly it's the first time speech recognition finally feels "production ready" after years of Siri/Alexa.
I tried a lot of tweaks but this one is definitely the best I've seen.
How did it know how to spell my name correctly when I just spoke into my microphone??? The two L's usually trips up the transcription models. What????
if the variations are pronounced the same? luck, probably.
This is the first time I've ever seen it get the spelling actually right (missing the accent is forgivable) ;)
How is tiny.en so damned accurate?
How much faster does this run natively?
Do the NEON instructions work for arm devices like an RPI or is it just tuned for Apple?
With this and Alpaca 13B you could probably replace an entire window manager.
Edit: it seems like it was less than a year ago that local speech recognition was a slow slog through a swamp of crufty, complicated research projects with most the high quality training data hidden behind walled gardens. Now I can stutter-step over a word and a demo in my browser correctly transcribes what I meant sans stutter. What happened?
What I really want is a program to waste the time of phone calls making unsolicited sales pitches.
It would do voice to text, run a simple language model to generate responses, then synthesize the voice back. It doesn't need to be a sophisticated model, not much more sophisticated than the classic "Eliza" program. A few years back someone did this with a canned loop of vague responses and it fooled the sales people for surprisingly long:
https://www.youtube.com/watch?v=XSoOrlh5i1k
It seems like it could all run locally for low latency. Probably the most important part to get right would be a TTS system that isn't immediately pegged as a robot.
https://lennytroll.com/