[–] ArtWomb 7y ago ↗ Just came to post this link as well. Progress in machine translation has truly been amazing. Also check out the recent work from Google on Transformer-XLhttps://ai.googleblog.com/2019/01/transformer-xl-unleashing-...And for deep background, these recent sets of notes from Stanford's Deep Learning NLP class:http://web.stanford.edu/class/cs224n/https://www.youtube.com/playlist?list=PL3FW7Lu3i5Jsnh1rnUwq_...
[–] yoquan 7y ago ↗ Hmm, the article's summary table showed "only the top task layer(s)" for BERT fine-tuning. Is that correct?The original paper [1] wrote:"All of the parameters of BERT and W are fine-tuned jointly to maximize the log-probability of the correct label."[1]: https://arxiv.org/abs/1810.04805 [–] lilianweng 7y ago ↗ thanks! my text should be correct but the table is wrong. just fixed.
3 comments
[ 9.4 ms ] story [ 29.2 ms ] threadhttps://ai.googleblog.com/2019/01/transformer-xl-unleashing-...
And for deep background, these recent sets of notes from Stanford's Deep Learning NLP class:
http://web.stanford.edu/class/cs224n/
https://www.youtube.com/playlist?list=PL3FW7Lu3i5Jsnh1rnUwq_...
The original paper [1] wrote:
"All of the parameters of BERT and W are fine-tuned jointly to maximize the log-probability of the correct label."
[1]: https://arxiv.org/abs/1810.04805