HN Explorer

Show HN: Llama.go – port of llama.cpp to pure Go (github.com)

25 points by Ambix ↗ HN
It's April 12 - The First Man in Space* Day - and I'm releasing #ML framework I've been building for the last month :)

It's written in #Go and allows #LLaMA #GPT inference having just regular PC - so no monster GPU cluster is needed to start experiment with:

https://github.com/gotzmann/llama.go

The V1 is using FP32 math only, but will work with AVX2 data types and INT8 quantisation soon.

* The first man in space was Yuri Gagarin from USSR

5 comments

[ 3.1 ms ] story [ 26.4 ms ] thread
smcleod
"Pure Go" ... looks at repo - 16.9% python.
barelysapient
The op only claimed to rewrite the llama.cpp in go, not the entire project. If look at llama/llama.go, it’s all written in go.
Ambix
Python needed only to converting PyTorch models into more accessible bin format :) All the inference and tensor math code is 100% Golang. I'm going to inject AVX2 assembler soon, but that's too very specific to Go ecosystem.
ipaddr
Do we have a php version yet
tough
very cool, waiting for llama.rust any day now