HN Explorer

Show HN: I ported Tree-sitter to Go (github.com)

222 points by odvcencio ↗ HN
This started as a hard requirement for my TUI-based editor application, it ended up going in a few different directions.

A suite of tools that help with semantic code entities: https://github.com/odvcencio/gts-suite

A next-gen version control system called Got: https://github.com/odvcencio/got

I think this has some pretty big potential! I think there's many classes of application (particularly legacy architecture) that can benefit from these kinds of analysis tooling. My next post will be about composing all these together, an exciting project I call GotHub. Thanks!

22 comments

[ 3.0 ms ] story [ 41.9 ms ] thread
sluongng
Oh this is really neat for the Bazel community, as depending on tree-sitter to build a gazelle language extension, with Gazelle written in Go, requires you to use CGO.

Now perhaps we can get rid of the CGO dependency and make it pure Go instead. I have pinged some folks to take a look at it.

gritzko
That is very very interesting. I work on a similar project https://replicated.wiki/blog/partII.html

I use CRDT merge though, cause 3-way metadata-less merges only provide very incremental improvements over e.g. git+mergiraf.

How do you see got's main improvement over git?

skybrian
How about making 'got' compatible with git repos like jujutsu? It would be a lot easier to try out.
irishcoffee
Is it a go-ism that source for implementation and test code lives in the root of the repo or is this an LLM thing?
jbreckmckye
Interesting. I have a similar usecase but intended to use CGo tree-sitter with Zig

Are these pretty up-to-date grammars? I'm awfully tempted to switch to your project

How large are your binaries getting? I was concerned about the size of some of the grammars

conartist6
It looks like porting the custom C lexers is a big part of the trouble you had to go to do this.
acedTrex
Claude attempted a treesitter to go port

Better title

AlexeyBelov
Thank you, I flagged the submission. I doubt this project will have activity in 6 months, but I'd love to be proven wrong.
shayief
This is great, I was looking for something like this, thanks for making this!

I imagine this can very useful for Go-based forges that need syntax highlighting (i.e. Gitea, Forgejo).

I have a strict no-cgo requirement, so I might use it in my project, which is Git+JJ forge https://gitncoffee.com.

silverwind
Gitea is definitely watching this one. Initialy tests show a 20x increase in syntax highlighting speed compared to the previous regexp-based approach.
trickypr
Do you have an equivalent of TreeCursors or tree-sitter-generate?

There are at least some use cases where neither queries nor walks are suitable. And I have run into cases where being able to regenerate and compile grammars on the fly is immeasurably helpful.

At least for my use cases, this would be unusable.

Also, what the hell is this:

> partial [..] missing external scanner

Why do you have a parsing mode that guarantees incorrect outputs on some grammars (html comes to mind) and then use it as your “90x faster” benchmark figure?

up2isomorphism
"rewrite" a nice code base without mentioning it is vibe coded is not great.

Essentially you use AI to somehow re-implement the original code base in a different language, made it somehow work, and claim it is xx times faster. It is misleading.

silverwind
I agree it's misleading. Ideally the author would disclose how much of the tree-sitter test suite passes.
kopirgan
Can someone please explain what's the connection between this and LSP? For example in Helix can one use this instead of various language servers?
nnx
This looks very interesting, but I wonder how's the rewrite approach gonna impact the long-term maintenance and porting changes _back_ from Tree Sitter.

As you mention WASM-readiness, did you consider using the official Tree Sitter WASM builds nicely packaged with wazero (pure Go WASM runtime) ?

It may help staying sync with upstream for the long term and, while probably a bit slower, has nice security and GC advantages too.

CodeCompost
I've seen a lot of "I've ported X to Go/Rust" posts lately. Is it the expectation that we're all supposed to abandon the original projects in favor of the ported versions which use newer and shinier programming languages? Is development going to continue on those new "Go/Rust" ports or are they just one-off karma farming projects?
monster_truck
Was excited to try this in my project but it doesn't seem like it's truly a complete port.
brodouevencode
Neat, but it really bothers me when projects don't use standard layouts.