Generating Lyrics Using Deep (Multi-Layer) LSTM (hackernoon.com)

As usual LSTMs are shit at generating text. But attention models (like BERT) are a whole different game.

The results shouldn't be that bad. There may be a bug in the implementation.