Retrieval-augmentation can significantly boost the performance of both 4K short context LLMs and 16K/32K long context LLMs.
The 4K context LLMs with simple retrieval-augmentation can perform comparable to 16K long context LLMs, while being more efficient at inference.
After context window extension and retrieval-augmentation, the best model LLaMA2-70B-32k-ret can outperform GPT-3.5-turbo-16k and Davinci003 in terms of average score on a suit of downstream tasks with informative queries.
2 comments
[ 2.9 ms ] story [ 34.8 ms ] threadRetrieval-augmentation can significantly boost the performance of both 4K short context LLMs and 16K/32K long context LLMs.
The 4K context LLMs with simple retrieval-augmentation can perform comparable to 16K long context LLMs, while being more efficient at inference.
After context window extension and retrieval-augmentation, the best model LLaMA2-70B-32k-ret can outperform GPT-3.5-turbo-16k and Davinci003 in terms of average score on a suit of downstream tasks with informative queries.