[–] nithiink 4mo ago ↗ How do you handle prompt caching? A lot of cost savings for a single model chat come from cache hits on the conversation context, and switching models invalidates that cache — the new model has to reprocess everything at full input price. [–] [dead] FrancescoMassa 3mo ago ↗ [flagged]
4 comments
[ 4.4 ms ] story [ 17.8 ms ] thread