Baidu's Improving Retrieval Augmented Language Model with Self-Reasoning (arxiv.org) 66 points by a-s-k-af 1y ago ↗ HN
[–] kaspermarstal 1y ago ↗ Can anyone explain what is gained by training a model? Why not use the foundational LLM for the relevance, evidence, and trajectory processes? [–] a-s-k-af 1y ago ↗ I assume you are referring to fine tuning a model here? [–] Tostino 1y ago ↗ You could also just continue pre-training of an existing foundation model. Would still be cheaper by not starting from zero. [–] a-s-k-af 1y ago ↗ The amount of accuracy while doing fine tuning or distillation is usually better than pre-training an existing model, not to mention the graph against the cost.
[–] a-s-k-af 1y ago ↗ I assume you are referring to fine tuning a model here? [–] Tostino 1y ago ↗ You could also just continue pre-training of an existing foundation model. Would still be cheaper by not starting from zero. [–] a-s-k-af 1y ago ↗ The amount of accuracy while doing fine tuning or distillation is usually better than pre-training an existing model, not to mention the graph against the cost.
[–] Tostino 1y ago ↗ You could also just continue pre-training of an existing foundation model. Would still be cheaper by not starting from zero. [–] a-s-k-af 1y ago ↗ The amount of accuracy while doing fine tuning or distillation is usually better than pre-training an existing model, not to mention the graph against the cost.
[–] a-s-k-af 1y ago ↗ The amount of accuracy while doing fine tuning or distillation is usually better than pre-training an existing model, not to mention the graph against the cost.
5 comments
[ 0.20 ms ] story [ 65.8 ms ] thread