Cutting LLM Batch Inference Time by Half with Dynamic Prefix Bucketing (daft.ai) 2 points by DISCURSIVE 7mo ago ↗ HN
0 comments
[ 2.5 ms ] story [ 9.8 ms ] threadNo comments yet.