For the application: being able to prompt anywhere in the sequence can be of interest. For what we've seen in the experiment, the rejection sampling leads to similar generation than the autoregressive one, we did not…
Hey! I'm Arnaud, first author of the paper. XLNet also shuffles the data during training, but they use a masking mechanism instead of the causal + double positional encoding. The application differs, XLNet is not AFAIK…
Hey, I'm Arnaud, first author of the paper. The answer is a bit mixed. We actually started looking into this because of a repetition problem that appeared in a low-data regime for a sequence generation task. Basically,…
For the application: being able to prompt anywhere in the sequence can be of interest. For what we've seen in the experiment, the rejection sampling leads to similar generation than the autoregressive one, we did not…
Hey! I'm Arnaud, first author of the paper. XLNet also shuffles the data during training, but they use a masking mechanism instead of the causal + double positional encoding. The application differs, XLNet is not AFAIK…
Hey, I'm Arnaud, first author of the paper. The answer is a bit mixed. We actually started looking into this because of a repetition problem that appeared in a low-data regime for a sequence generation task. Basically,…