I tried to love mc, but its ergonomics felt slightly off. Maybe it's just hard to rewire my Norton / Volkov Commander / FAR Manager muscle memory, I don't know I ended up being on a Linux fork of Far Manager, which…
March 2024
This video has a live coding part which implements a masked diffusion generation process: https://www.youtube.com/watch?v=oot4O9wMohw
You can absolutely do it, and I think it's a nice idea to try.
Correct, diffusion LMs can edit their intermediate predictions, so "final" tokens aren't necessarily final. This is an exciting property because it allows models to correct errors in what's generated so far -- something…
Despite the name, diffusion LMs have little to do with image diffusion and are much closer to BERT and old good masked language modeling. Recall how BERT is trained: 1. Take a full sentence ("the cat sat on the mat") 2.…
I'm also very skeptical of the significance of this "aha moment". Even if they didn't include chain-of-thoughts to the base model's training data (unlikely), there are still plenty of it on the modern Internet. OpenAI…
The whole point of NPU-enabled devices is to run models locally, so they your data never leaves your device. This is a huge privacy win.
I use tmux when SSH'ing to remote boxes, but when working locally I find native terminal panes and tabs to be a better experience. Does tmux provide anything extra to what wezterm/kitty/iterm2 do?
A lot. POS taggers used to be linear classifiers + features. In 2018 they switched to BERT and similar encoder-only models. In 2023, POS tagging is largely irrelevant, because it was used as a part of a larger pipeline,…
I tried to love mc, but its ergonomics felt slightly off. Maybe it's just hard to rewire my Norton / Volkov Commander / FAR Manager muscle memory, I don't know I ended up being on a Linux fork of Far Manager, which…
March 2024
This video has a live coding part which implements a masked diffusion generation process: https://www.youtube.com/watch?v=oot4O9wMohw
You can absolutely do it, and I think it's a nice idea to try.
Correct, diffusion LMs can edit their intermediate predictions, so "final" tokens aren't necessarily final. This is an exciting property because it allows models to correct errors in what's generated so far -- something…
Despite the name, diffusion LMs have little to do with image diffusion and are much closer to BERT and old good masked language modeling. Recall how BERT is trained: 1. Take a full sentence ("the cat sat on the mat") 2.…
I'm also very skeptical of the significance of this "aha moment". Even if they didn't include chain-of-thoughts to the base model's training data (unlikely), there are still plenty of it on the modern Internet. OpenAI…
The whole point of NPU-enabled devices is to run models locally, so they your data never leaves your device. This is a huge privacy win.
I use tmux when SSH'ing to remote boxes, but when working locally I find native terminal panes and tabs to be a better experience. Does tmux provide anything extra to what wezterm/kitty/iterm2 do?
A lot. POS taggers used to be linear classifiers + features. In 2018 they switched to BERT and similar encoder-only models. In 2023, POS tagging is largely irrelevant, because it was used as a part of a larger pipeline,…