Cool approach! The only thing I would change or allow customization for is the prompt you're injecting. Most users will probably want to change this:
"Output a json object or array generated with random content fitting this schema, based on the PORMPT section below." (sic)
My findings from one week of forcing chatgpt to produce structured output:
- it prefers a certain kind of format (the one it uses by default when you ask it without json/markdown), and if you deviate too far, it won't listen. E.g. I tried adding underscores to certain words, which it only did half the time.
- regex parsing the output is quite robust. Even if it adds sections you don't want like intro or conclusion, the parser will just discard these
I started porting Auto-GPT to NextJS for fun yesterday. The python implementation requires output to be a predictable json schema, so this might be useful in the TypeScript version.
7 comments
[ 2.7 ms ] story [ 27.1 ms ] thread"Output a json object or array generated with random content fitting this schema, based on the PORMPT section below." (sic)
My findings from one week of forcing chatgpt to produce structured output:
- it prefers a certain kind of format (the one it uses by default when you ask it without json/markdown), and if you deviate too far, it won't listen. E.g. I tried adding underscores to certain words, which it only did half the time.
- regex parsing the output is quite robust. Even if it adds sections you don't want like intro or conclusion, the parser will just discard these
> based on the PORMPT