Ask HN: How do you version GPT prompts

1 points by okoma 3y ago ↗ HN

This is for the purpose of tracking improvements of a code completion or a chatGPT prompt.

Are you using an existing solution, rolling your own or a mix?

5 comments

[ 4.5 ms ] story [ 25.5 ms ] thread

verdverm 3y ago ↗

Git

okoma 3y ago ↗

What structure do you use if you wanted to check the accuracy of an older prompt with an updated or extended data set?

verdverm 3y ago ↗

This question is slightly different because you have to track a testing set and the metrics. I would look to the many ML workflow tools

okoma 3y ago ↗

Thanks for pointing towards the right direction. I'll edit the original question.

To rephrase, I am looking for a tool to do model lifecycle management https://github.com/kelvins/awesome-mlops#model-lifecycle and wonder if there is any one in particular that you'd think is better suited for prompts, i.e. an array of objects with templated text

okoma 3y ago ↗

I found a bunch of resources here: https://alphasec.io/prompt-engineering-playgrounds-marketpla...

Good candidates for what I am looking are OpenAI Playground (once it supports versioning and variables for chat), PromptLayer (once it supports chat prompts), EveryPrompt (chat prompts missing). More dev oriented, LangChain https://js.langchain.com/docs/getting-started/guide-chat which can then be easily versioned.