Recently, I conducted a few fine-tuning experiments on a multitask instruction dataset on 1b7 LLM, primarily related to open-ended long story generation. However, it was observed that after generating nearly 300 tokens, the quality of the generated text started to decline, becoming less fluent.
I am curious to know if there exists a relationship between the scale of the model and the quality of long text generation. For example, is it possible to achieve fluent long text after generating a certain number of tokens for models like 1b7?
1 comment
[ 3.8 ms ] story [ 13.9 ms ] threadI am curious to know if there exists a relationship between the scale of the model and the quality of long text generation. For example, is it possible to achieve fluent long text after generating a certain number of tokens for models like 1b7?