HN Explorer

18 comments

[ 0.26 ms ] story [ 56.2 ms ] thread
neha_n
This is much needed. For someone looking to quickly implement, having a simple interface goes a long way.
yding
Very cool Ishaan!
hardware2win
>completion(..., azure=True)

Why like this?

detente18
azure uses the openai python sdk, just remaps certain components. The models are also user-named. This makes it hard to detect if a model passed in is an azure model
ij23
azure models have custom names - eg I call mine 'chat-gpt-test1', I require some flag to know if it's an azure model
d4rkp4ttern
Great start! Are you planning to add to following:

Retries w exponential backoff, Caching, Streaming output, Function-calling support

detente18
You can use tenacity for retries and wouldn't you want to cache the request / response around the endpoint instead of the gpt call -> that's what we ended up doing.

Streaming output and function-calling support is interesting

d4rkp4ttern
Could you elaborate on this — “wouldn’t want to cache around the endpoint instead of GPT call”. Just want to see if I’m missing an important consideration here
detente18
You're probably calling openai from a backend server. If you're making the call because a user asked a question -> your endpoint received it -> did some processing -> called openai -> returned the response,

why would you cache the openai call instead of the endpoint that's receiving the user call?

ij23
good points, probably going to add streaming output, function calling support. As for retries tenacity does a great job already
kaushik92
This is amazing. Really needed something like this to standardize all my different AI APIs!

On a side note - I love how quickly your team is shipping! Do keep it going!