Show HN: GPT-JSON – Structured and typehinted GPT responses in Python (github.com)

What we've found useful in practice in dealing with similar problems:

- Use json5 instead of json when parsing. It allows trailing commas.

- Don't let it respond in true/false. Instead, ask it for a short sentence explaining whether it is true or false. Afterwards, use a small embedding model such as sbert to extract true/false from the sentence. We've found that GPT is able to reason better in this case, and it is much more robust.

- For numerical scores, do a similar thing by asking GPT for a description, then with the small embedding model write a few examples matching your score scale, and for each response use the score of the best matched example. If you let GPT give you scores directly without explanation, 20% of the time it will give you nonsense.

https://github.com/jiggy-ai/pydantic-chatcompletion/blob/mas...

Someone had to come along and spoil the fun.

Next it'll be ChatGPT/XML.

However I've found it to be generally unreliable and adds a lot of text to each call.

I suspect this can be improved by: - Only writing the parts of the spec which are needed, rather than the full JSON schema spec - Including relevant examples rather than arbitrary JSON schema examples

It takes the expected return type of a function, translates it into a JSON Schema, queries OpenAI, validates the response, and then magically returns the response as typed output.

[0]https://github.com/jumploops/magic

"Yes, integrating the CUE language with GPT-JSON can potentially lead to improvements in its capability. CUE, short for Configuration, Unification, and Environment, is an open-source data constraint language designed to simplify the definition, generation, and validation of data in structured formats, like JSON, YAML, and others.

Integrating CUE with GPT-JSON can bring several benefits:

1) Validation: By using CUE, you can define schemas for the input and output JSON, which helps in validating the data structure and ensuring it adheres to the expected format. This can help reduce errors and improve the quality of the interaction with the GPT model.

2) Simplification: CUE can help simplify the process of defining and generating JSON inputs and outputs for GPT-JSON. By providing a more concise and expressive syntax, CUE can make it easier to craft structured inputs and outputs, which can be especially helpful when working with complex data structures.

3) Interoperability: Integrating CUE can improve the interoperability between GPT-JSON and other systems or applications. With CUE's ability to work with multiple data formats (such as JSON, YAML, or HCL), it becomes easier to integrate GPT-JSON with various tools and platforms.

To integrate CUE with GPT-JSON, you would need to follow these general steps:

1) Define CUE schemas for the input and output JSON structures used by GPT-JSON.

2) Use the CUE language to validate and generate input JSON for GPT-JSON.

3) Process the output JSON generated by GPT-JSON using CUE to ensure it adheres to the defined schema and transform it as needed.

Integrating CUE with GPT-JSON can improve the overall robustness and ease of use of the library, making it more accessible and useful for a wider range of applications."

        messages=[
            GPTMessage(
                role=GPTMessageRole.SYSTEM,
                content=SYSTEM_PROMPT,
            ),
            GPTMessage(
                role=GPTMessageRole.USER,
                content="Text: I love this product. It's the best thing ever!",
            )
        ]

        messages=(
            ("system", SYSTEM_PROMPT),
            ("user", "Text: I love this product. It's the best thing ever!")
        )

        messages=(
            SystemMsg(SYSTEM_PROMPT),
            UserMsg("Text: I love this product. It's the best thing ever!")
        )

I've had luck doing chain-of-thought prompting in the JSON payload as you've described, too. Cheers, really validating to see someone taking a similar approach.

  ```
  [TOML]
  key="""
  value
  """
  """

      -
         column name: salary_max
         format: number
         example: 150,000
         description: Salary Maximum
      -
         column name: keywords
         format: string
         example: engineer, python, docker, remote
         description: Relevant Keywords (Comma separated keywords used for filtering and matching jobs to candidates)

I see a ton of code in this project, and I don't know what most of it does. As far as GPT troubles with JSON, I'll add a couple: sometimes it likes to throw comments in there as if it was JS. And sometimes it'll triple-quote the JSON string as if it was Python.

My approach to solve these problems was via prompt engineering - using the system message part of the API call. Asking it to "return valid json, do not wrap it in text, do not preface it with text, do not include follow-up explanations, make sure it's valid json, do not include comments" - seems to work 99% of the time. For the remainder, a try-and-catch block with some fallback code that "extracts" json (via dumb REs) from whatever text was returned. Hasn't failed yet.

It's fascinating to watch the new paradigm arrive, and people using old habits to deal with it. This entire project is kind of pointless, you can just ask GPT to return the right kind of thing.

JSON is admittedly way easier to work with up front, but markdown seems to be a more scaleable choice.

Of course, this is all very much an opinion and highly anecdotal at the moment.

"Do not include any explanations, only provide a RFC8259 compliant JSON response following this format without deviation.

{

  "author": "string describing the author full name",

  "year": "number describing the year the book was written",

  "isFiction": "boolean describing if the book is a work of fiction"

  ...

https://github.com/knowsuchagency/struct-gpt

I tried to make the API as intuitive as possible and added the ability to provide examples to improve the reliability and quality of the LLM's output.