It's a good thing people were enamored of how inexpensive GPT-5 is, given that the system prompt is (allegedly) 54kb. I don't know how many tokens that is offhand, but what a lot of them to burn just on setup of the thing
How does a prompt this long affect resource usage?
Does inference need to process this whole thing from scratch at the start of every chat?
Or is there some way to cache the state of the LLM after processing this prompt, before the first user token is received, and every request starts from this cached state?
4 comments
[ 2.5 ms ] story [ 23.4 ms ] thread> Place rich UI elements within tables, lists, or other markdown elements when appropriate.
Does inference need to process this whole thing from scratch at the start of every chat?
Or is there some way to cache the state of the LLM after processing this prompt, before the first user token is received, and every request starts from this cached state?