How Diffusion Models Can Achieve Seemingly Arbitrarily Large Compression Ratios (medium.com)

Pertaining to recent discussions around AI art ethics and the ability for image diffusion models to "copy" training samples, this post provides intuition for how generative models such as diffusion models compress information from a dataset. It provides intuition showing that the amount of information that is compressed during learning is not determined by the dataset’s memory footprint, but by the nature and complexity of the underlying distribution that generated the data in the first place.