The author primarily talks about the compression–prediction equivalence and also provides some working code linked in Github https://github.com/nathan-barry/gzipt
Every prediction model is inherently a compressor, and all compression algorithms are prediction models.
Reference:
Language Modeling Is Compression — Delétang et al., DeepMind, 2023. The prediction-compression equivalence, with the Chinchilla-beats-PNG result.
1 comment
[ 3.6 ms ] story [ 17.1 ms ] threadEvery prediction model is inherently a compressor, and all compression algorithms are prediction models.
Reference: Language Modeling Is Compression — Delétang et al., DeepMind, 2023. The prediction-compression equivalence, with the Chinchilla-beats-PNG result.