It's on the page, if you click the little info icon in the upper-right. Here's the text but there's some nice graphics there too:
Snake Game, training entirely in the browser. Built on tinygrad: the rollout / targets / train graphs are TinyJits authored in Python, then compiled once to WGSL and replayed here under WebGPU.
Observation: flat 10×10 board (100) + 4-dim prev-action one-hot = 104 dims. fc_pi.weight is zero-init so the opening policy is uniform over the legal actions; fc_v uses tinygrad's default Kaiming init.
Per rollout: T=24 × N=384 parallel snakes (9,216 transitions), then K=3 epochs × 4 mini-batches of PPO updates. GAE γ=0.99, λ=0.95; AdamW wd=0.01; ratio clip ε=0.1; grad-norm 0.5; Huber value β=1, val_coef=1; entropy bonus 0.008333333333333333.
Action mask + value clip + KL early stop. The 4-dim prev_a obs tail lets fc_pi zero the U-turn logit (the env silently overrides same-axis reversals anyway). Value loss is max(huber(v_new−td), huber(v_clip−td)) at ε=0.2. Approx-KL is sampled after each epoch and breaks the loop at 1.5·kl_target.
My average eventually made it to about 3900, and then stagnated between 3600-3900. I'm curious if this is universal behavior or not. I'm up to about 5k steps.
my training on a 10x10 just randomly broke. i got to like 3600 then the graph went flat, the viewer on the left just showed it constantly restarting the game, and the scores in the negative. my average is now -10.
Really cool! But right as it was nearing 4,000, it seems to have corrupted itself and no longer got any scores above 0. Not sure if that's a code bug or a neural net issue.
I noticed snake gets penalized for not getting to the apple early, is that what you really want? Snake is about how long it gets not about the balance between length and wall clock time
Poorly programmed, it doesn't learn from its mistakes, the games get stuck in a loop because the snake doesn't capture a piece but the piece remains and there's a gap, constantly moving the snake along the same path with negative scores in an infinite loop leaving an unaltered yin and yang ;) there's a repetitive pattern in these infinite games between the position of the gap and the piece
FYI this website sets off a bunch of Bitdefender alerts as being a suspicious web page. I assume probably false positives or something but still something you might want to look into.
"The page https://ppo.gradexp.xyz/ has been detected with suspicious activity. It is not recommended to continue browsing this website."
This is seriously impressive. Running PPO training directly in the browser through WebGPU feels like a glimpse into where lightweight AI experimentation is headed.
31 comments
[ 2.1 ms ] story [ 80.7 ms ] threadI noticed that if you go from training to watch and then back, the training temporarily drop significantly in score.
avg500 -4.6 last 500 episodes
peak 3959.3 best window
roll/s 20.68 20-step avg
progress 4388 562749 episodes
Looks like this is for Linux and Windows, on NetBSD I get this issue :(
trained and made a viz for the model and then made it displace text.
should probably do a proper write-up:https://x.com/i/status/2038367016969724259
"The page https://ppo.gradexp.xyz/ has been detected with suspicious activity. It is not recommended to continue browsing this website."
Same for:
https://ppo.gradexp.xyz/version.js
https://ppo.gradexp.xyz/dist/sizes.js
https://ppo.gradexp.xyz/dist/size_6/manifest.j
https://ppo.gradexp.xyz/dist/size_6/weights.safetens
https://ppo.gradexp.xyz/dist/sokol/demo.wa