HN Explorer

Jevgpt – reinventing the wheel with another wheel

1 points by kunggaochicken ↗ HN
Felt extra autistic today and made jev an autoregressive chatbot by classifier small token vocabularies.

https://github.com/giga-james/jevgpt

had some trouble getting it to explore enough token options so i did a small spec decoding pass to expand token vocab (jev as drafter and verifier lol)

it aint much and it aint honest work either

4 comments

[ 2.6 ms ] story [ 16.1 ms ] thread
nico
This is amazing. I don't think I fully understand how it works, but the core of having Jev (or some classifier), generate the words/tokens it's going to output by picking/classifying the words/tokens, is very cool

Where do you see the biggest potential improvement gains?

adityamishra241
Really interesting approach. I’m curious how much the verifier pass improves the output quality compared to just increasing the token vocabulary.
kunggaochicken
great question! the verifier pass actually operates more as an "assistant" gate so we can expand the token vocabulary.

jev allows 255 decisions (token vocabulary here), so if we want to support a token vocabulary of 1000, we have to parallelize into multiple jev calls to allow that expansion. the verifier pass simply chooses the best out of the parallel decisions so we can exceed the 255 decision limit. in theory this should allow for up to 255^2 token limit, but if you increase the verifier depth you can increase that exponential factor as much as you want (it'll be slow though)