Acronymy (Can we define every word as an acronym?) (acronymy.net)

this is pretty cool, here's another question, how much language compression would we get if we collapse all related words to a single synonymous word? Here's what chatgps came up with:

Assume an English-like active vocabulary V = 50,000 word types (a rough stand-in for “distinct words” seen commonly). We could get a realistic guess of: ~30% reduction for a less modest, more aggressive embedding-style collapse in typical English text. I.e. Collapse words with similar meaning directions in vector space... happy, glad, pleased, delighted → happy

But lots of words have multiple definitions

congress [0]

> collection of non governors really exhibiting self service

0: https://acronymy.net/define/congress

this is the type of important work that transformer LLM’s are actually really good at, I think

Wilkins. The Analytical Wilkins. https://en.wikipedia.org/wiki/The_Analytical_Language_of_Joh...