We're working on the mechanics of deploying and scaling all the amazing prompt-based backends everyone is building --- chaining, prompt-search-prompt, image & audio generation.
This link is a simplified hosting harness we built building & launching a Python + Prompt API endpoint.
The thing which could be a great offer is ability to supply data. I’m not sure if it involves up training or something else. Imagine I have a bunch of documents and I want answer to be based on my data.
That "Ask the Bible" service that went viral the other day on Twitter had me thinking the same thing.
Especially for the set of problems with "Step 1: Hosting embeddings for some large corpus", it feels like there's a really useful role for offering static query/AKNN search atop popular datasets.
How do third-party API services (me) earn? Otherwise as a developer, I'm providing a free service to users, and yet I'm paying steamship's hosting costs.
Please provide an alternative to ShipQL or just drop it entirely. Most developers I know are very allergic to learning new company-specific DSLs.
Re: earning -- 100%. We'd love to talk to you if you've got thoughts / requests (crew@steamship.com).
Re: DSLs -- what's the famous saying? "All services that survive eventually grow large enough to re-implement email"? :) We hear you though; if there's a way to support a standard QL atop the domain we're serving, we'll give it a good look.
The query problem is hard. If we provide a DSL, someone has to learn the DSL, which no one likes. If we provide you some standard QL, we still have to explain what parts of it are supported / unsupported, and how it applies to our specific data model. We started with a small DSL and will have to see how people want to use it. Thanks for the feedback!
> Do I have to pay? Our free introduction has a limit of 500 API calls and 200,000 characters / 40 MB of plugin (LLM) usage. Reach out to support@steamship.com to increase you limit.
And I'm out. I've easily made 500 calls to OpenAI in a day of playing with it. There is no pricing page. I'm not going to email someone to get a quote. It's 2023.
[Steamshipper here] This particular Prompt API template is a simplified starting place to what we're working on. So you're right to ask that question -- it's not so different than a lambda in this form.
More broadly: there are a lot of moving parts to a production AI backend: vectors, LLM chaining, long-running async tasks, persistent state, data conversion, etc.
We're working toward a framework that lets you snap together those pieces at hackathon speed... and then translate it into a multi-tenant, auto-managed infra stack out of the box.
I tried this out using ship try. I'm not a python dev, so installed pip and Jupyter, then tried the ticket tagging one and the audio transcripts one, but neither worked - one referenced a json file of ticket examples that didn't exist, and the audio one couldn't import utils.
Not a python person, so not sure if this is me or not, but just very unclear what this is doing/what it's meant to do.
Idea sounds useful, but I can't tell what this is meant to do/what I can do with this.
No pricing also means that hard to understand if this is something I can use in production or not.
I'm usually a typescript/lambda kinda guy for reference, and have already used openai in production on that stack.
Really appreciate you giving it a go in the face of those errors.
In retrospect, that initial `ship try` CLI & those initial packages had too many moving parts for a first launch.
Part of what we're trying to do with this Prompt API template is strip things down to bare bones so that we can nail the basic "clone -> modify -> deploy" flow before layering back on bells & whistles.
If you're ever up for another go, give us a holler and we'll jump in if you hit snags like that again!
24 comments
[ 3.5 ms ] story [ 62.6 ms ] threadWe're working on the mechanics of deploying and scaling all the amazing prompt-based backends everyone is building --- chaining, prompt-search-prompt, image & audio generation.
This link is a simplified hosting harness we built building & launching a Python + Prompt API endpoint.
Right now our examples are pretty simple. But you can chain prompts manually and/or pull in services like DALL-E or voice gen.
We're working on the rails to do that in a way that'll scale well cleanly across API users of your app.
Especially for the set of problems with "Step 1: Hosting embeddings for some large corpus", it feels like there's a really useful role for offering static query/AKNN search atop popular datasets.
Please provide an alternative to ShipQL or just drop it entirely. Most developers I know are very allergic to learning new company-specific DSLs.
Re: DSLs -- what's the famous saying? "All services that survive eventually grow large enough to re-implement email"? :) We hear you though; if there's a way to support a standard QL atop the domain we're serving, we'll give it a good look.
And I'm out. I've easily made 500 calls to OpenAI in a day of playing with it. There is no pricing page. I'm not going to email someone to get a quote. It's 2023.
Totally fair! In our case we're just trying hard to maximize product iteration cycles before we build out billing, etc.
If we just provided a way to pass through your own OpenAI key, does that work better for you?
That would be my ideal, thanks!
Well done!
I've talked to them before, they seem great, check them out.
More broadly: there are a lot of moving parts to a production AI backend: vectors, LLM chaining, long-running async tasks, persistent state, data conversion, etc.
We're working toward a framework that lets you snap together those pieces at hackathon speed... and then translate it into a multi-tenant, auto-managed infra stack out of the box.
>it's not so different than a lambda in this form.
Here?
> More broadly: there are a lot of moving parts ...
Not a python person, so not sure if this is me or not, but just very unclear what this is doing/what it's meant to do.
Idea sounds useful, but I can't tell what this is meant to do/what I can do with this.
No pricing also means that hard to understand if this is something I can use in production or not.
I'm usually a typescript/lambda kinda guy for reference, and have already used openai in production on that stack.
In retrospect, that initial `ship try` CLI & those initial packages had too many moving parts for a first launch.
Part of what we're trying to do with this Prompt API template is strip things down to bare bones so that we can nail the basic "clone -> modify -> deploy" flow before layering back on bells & whistles.
If you're ever up for another go, give us a holler and we'll jump in if you hit snags like that again!