Show HN: ReliableGPT run 200 GPT-4 requests in parallel (colab.research.google.com)

14 points by ij23 3y ago ↗ HN

reliableGPT handles GPT-4 429 rate limit errors

it optimally allocate jobs for incoming requests to maximize usage of Requests/min and Token/min limits

3 comments

[ 1.4 ms ] story [ 21.4 ms ] thread

nomadness 3y ago ↗

Why not using just a fetch with retry function ?

huac 3y ago ↗

It is, in fact, a try-catch. https://github.com/BerriAI/reliableGPT/blob/main/reliablegpt...

derwiki 3y ago ↗

Looks like this will only be effective for short prompts/responses, eg if you have 4k tokens in your prompt, you can only fire 10 requests/minute with 40k token/minute rate limit