Follow live the open training of a 535B (23B activated) LLM (twitter.com) 1 points by ggcr 22d ago ↗ HN
[–] ggcr 22d ago ↗ Announcement tweet by Stanford prof Percy Liang and Marin: https://x.com/percyliang/status/2090918065634684997 https://xcancel.com/percyliang/status/2090918065634684997It will include Pre-training and Mid-training. ETA 3 months.Live wandb dashboard: https://wandb.ai/marin-community/marin_moe/reports/535B-A23B...GitHub issue tracking progress: https://github.com/marin-community/marin/issues/843523T tokens (not deduped) directly in S3 for anyone to see: s3://marin-us-east-02a/marin/datakit/store_4d2e363dIntermediate checkpoints also in S3
1 comment
[ 0.21 ms ] story [ 11.1 ms ] threadIt will include Pre-training and Mid-training. ETA 3 months.
Live wandb dashboard: https://wandb.ai/marin-community/marin_moe/reports/535B-A23B...
GitHub issue tracking progress: https://github.com/marin-community/marin/issues/8435
23T tokens (not deduped) directly in S3 for anyone to see: s3://marin-us-east-02a/marin/datakit/store_4d2e363d
Intermediate checkpoints also in S3