This API is the first API from https://www.booste.io, where my mission is to make the ML stack as simple as Stripe made the payments stack. One-line API clients.
Lots on the roadmap (Node client, using larger GPT-2 model, PPLM for long-content generation, running on GPUs, hosting other models such as BERT), but there's no time like the present to put something out there.
I'm super bullish on CV in fitness. Would love to hear what you're thinking.
For CV, it's not on the near roadmap. I've looked into hosting OpenPose and YoloV3. OpenPose could work for a fitness project but Yolo definitely needs custom training to be novel. In the future with this project I plan to do custom training as well, which would make using novel CV models a possibility. For now, the plan is to stay focused on NLP since pretrained models can handle a broader range of novel use cases.
I have a form on the docs site (and here: https://forms.gle/yoENXh4tLU5cTW2P8) for requesting new models, so down the line I know who to be talking to and which models to prioritize.
It says on the website "Predict the next word(s) from a given sequence of words." Speaking as someone who has very little programming knowledge, would it be possible to use this to create a gmail like auto-complete with this? If so, how can I make it so that the words generated resemble my own writing style (like gmail auto-complete does?).
Working on a fine-tuning add-on currently :) The one I'm working with is called PPLM by Uber eng, and it takes a "bag of words" to keep generated text on topic. It's not exactly what you're asking for (style transfer), but you'd be able to load it with your common email phrases, and the postprocessing will bias similar phrases, keeping it somewhat stylistically consistent.
Working on a feature that allows you to define your common phrases, and it'd bias the output toward that. See my response to knicholas in this thread. It's not a perfect fix but would work.
Other APIs I launch in the future should help. BERT comes to mind. You could generate multiple options using GPT2, then have BERT assess if they're in your style.
I'd like to keep pretrained models free, so this current experience should stay free.
Add-ons such as style transfer logic or any form of personalization on the model would have a cost, haven't done the math yet so I couldn't reasonably ballpark it.
UPDATE: First time launching this to larger audience, and hitting infra limits. It's self healing, but please be patient if an error throws; try back in 3 minutes. I'm working to remedy it now.
UPDATE: (as of 24 upvotes on main post). I scaled up my infra. It should be more stable at current load. Please continue to be patient; y'all are great.
UPDATE: (as of 36 upvotes): Thanks everyone for trying it out and hitting me with the traffic. Great Show HN.
Found a memory leak. Luckily it self heals on crash, so downtime is minimal. You can expect this api to be live 55/60 minutes. Will be fixing tomorrow - for now, it's bedtime.
If you want to wait for something with guaranteed uptime, jump on the waitlist at https://www.booste.io and I'll be reaching out when appropriate.
UPDATE: (a week later): I've gutted my infrastructure and moved to TF Serving. Haven't had a downtime event in the 30 hrs it's been live. Jump on the waitlist for updates :)
Related, Fabrice Bellard developed a C program for GPT-2 inference. It's not open source, but the binary is free to download, and can use any pretrained GPT-2 model:
I tried it using a GPT-2 model I'd finetuned, and it worked well.
I started making a simple chatbot with it using Python: at each step, I fed in the conversation so far, and a speaker prompt (e.g. PERSON2:), and read the output until I saw PERSON1: appear. Rinse and repeat.
Unfortunately, I got stuck with some text encoding issue between python and the CLI binary. I tried obvious things like forcing the decode to use UTF-8, but after a couple of steps I always ended up receiving some characters that weren't valid.
I didn't have this problem when running the binary from the command line, so it must have been something to do with how I was using python popen.
20 comments
[ 2.7 ms ] story [ 52.8 ms ] threadThe goal is simplicity, so you hit the endpoint with a one-line call from the Python client.
Here's a demo of pip install, code, and use. https://www.loom.com/share/c09c9ca228644345852544808decd864
This API is the first API from https://www.booste.io, where my mission is to make the ML stack as simple as Stripe made the payments stack. One-line API clients.
Lots on the roadmap (Node client, using larger GPT-2 model, PPLM for long-content generation, running on GPUs, hosting other models such as BERT), but there's no time like the present to put something out there.
Let me know your thoughts :)
(edit: fixed link)
For CV, it's not on the near roadmap. I've looked into hosting OpenPose and YoloV3. OpenPose could work for a fitness project but Yolo definitely needs custom training to be novel. In the future with this project I plan to do custom training as well, which would make using novel CV models a possibility. For now, the plan is to stay focused on NLP since pretrained models can handle a broader range of novel use cases.
I have a form on the docs site (and here: https://forms.gle/yoENXh4tLU5cTW2P8) for requesting new models, so down the line I know who to be talking to and which models to prioritize.
It says on the website "Predict the next word(s) from a given sequence of words." Speaking as someone who has very little programming knowledge, would it be possible to use this to create a gmail like auto-complete with this? If so, how can I make it so that the words generated resemble my own writing style (like gmail auto-complete does?).
Other APIs I launch in the future should help. BERT comes to mind. You could generate multiple options using GPT2, then have BERT assess if they're in your style.
Add-ons such as style transfer logic or any form of personalization on the model would have a cost, haven't done the math yet so I couldn't reasonably ballpark it.
Found a memory leak. Luckily it self heals on crash, so downtime is minimal. You can expect this api to be live 55/60 minutes. Will be fixing tomorrow - for now, it's bedtime.
If you want to wait for something with guaranteed uptime, jump on the waitlist at https://www.booste.io and I'll be reaching out when appropriate.
https://bellard.org/nncp/gpt2tc.html
I tried it using a GPT-2 model I'd finetuned, and it worked well.
I started making a simple chatbot with it using Python: at each step, I fed in the conversation so far, and a speaker prompt (e.g. PERSON2:), and read the output until I saw PERSON1: appear. Rinse and repeat.
Unfortunately, I got stuck with some text encoding issue between python and the CLI binary. I tried obvious things like forcing the decode to use UTF-8, but after a couple of steps I always ended up receiving some characters that weren't valid.
I didn't have this problem when running the binary from the command line, so it must have been something to do with how I was using python popen.