Yes, function calls across the boundary are slow. It's fine if the functions do lots of work, it's very expensive for function calls that should be cheap.
For sqlite it might not matter too much, since any query does quite a bit of work.
Agreed. I wouldn't be surprised if a pure-Go alternative would be slower. Someone remarked upthread that the same author wrote a C-to-Go transpiler specifically to make a pure-Go sqlite and it was (somewhat unsurprisingly) considerably slower than the CGo version. It's possible (and to an extent, likely) that the transpolar was generating suboptimal Go code, but I would wager that a hand-written version would also be slower.
zombiezen/go-sqlite uses cznic's pure Go converted version of SQLite, so avoids CGo. It's explicitly stated to be "a fork of crawshaw.io/sqlite that uses modernc.org/sqlite, a CGo-free SQLite package. It aims to be a mostly drop-in replacement for crawshaw.io/sqlite."
The same author ("cznic") has also built a C-to-Go compiler/transpiler and used it to convert SQLite to pure Go source, so that it doesn't require CGo: https://gitlab.com/cznic/sqlite ... the converted Go code is pretty much unreadable and it's somewhat slower than SQLite via CGo. But a pretty neat idea! I use it in one of my projects and it works well.
I saw that recently. I would be interested in using it but it wasn't clear what level of compatibility it has or how many tests it's passing/not passing.
Hehe. Well, if you look at the previous release it shows 99.7% percent of (a smaller number of) the tests passing. So you've got the best of both worlds. :-)
Nice. I tried to cross-compile a Go app using SQLite for darwin/arm64 a couple months back and couldn’t figure it out for the life of me. Ended up just shipping a darwin/amd64 binary and letting Rosetta handle it. Will give this a spin when I have time.
Not sure why this is downvoted. SQLite is great and all, but not depending on FFI is really nice for a lot of things (e.g., cross-compiling is a lot simpler). I'll trade quite a bit to avoid even the very highest quality C dependencies.
What are the use cases for this? Is this a good fit for Memory based testing for db stores? Also, I can’t seem to find documentation for how this compares to SQLite
If you take into account the fact that Richard Hipp is not a billionaire whilst his software is used in almost all devices on (and not-on) the planet, his stuff is used by countless of megacorporations, etc, etc, then by my calculations: not nearly enough.
His paycheck alone should be tens of millions of dollars.
He should be making so much money that if he'd drop a 100 dollar bill it wouldn't be worth his time to pick it up.
Development on this project has started in 2013 and scc shows "Estimated Cost to Develop (organic)" as $1.2M so for whatever that's worth it could be said to check those "5-7 years" and the "millions of dollars" boxes.
Keep in mind that this project isn’t creating a new SQL database from scratch. It’s literally ingesting SQLite’s C source code and spitting out Go. cznic’s done an absolutely incredible amount of work here. It’s not ready for prod yet, but it works and passes all of SQLite’s tests.
This modernc.org/ql is an embedded database accessed through QL, which is a sub-SQL language "less complex and less powerful than SQL" (according to their homepage). I fail to see when this would be useful compared to the de facto standard SQLite.
BTW, modernc.org/sqlite (by the same author) is a port of SQLite to native Go. I've used it with great success for a few personal projects. Before that, I used the go-sqlite package, which needs CGO, and makes it hard to build static executables.
50 comments
[ 2.8 ms ] story [ 124 ms ] threadhttps://pkg.go.dev/modernc.org/ql?utm_source=godoc
https://gitlab.com/cznic/ql/-/blob/master/design/doc.go
[0]: https://www.arp242.net/static-go.html
For sqlite it might not matter too much, since any query does quite a bit of work.
https://github.com/crawshaw/sqlite https://github.com/zombiezen/go-sqlite
zombiezen/go-sqlite uses cznic's pure Go converted version of SQLite, so avoids CGo. It's explicitly stated to be "a fork of crawshaw.io/sqlite that uses modernc.org/sqlite, a CGo-free SQLite package. It aims to be a mostly drop-in replacement for crawshaw.io/sqlite."
[1]: https://sqlite.org/th3.html
[2]: http://www.hwaci.com/sw/sqlite/prosupport.html
https://github.com/etcd-io/bbolt is another pure go option.
cznic seems like an alternative to bbolt. nice to have some options.
https://github.com/genjidb/genji
as far as sqlite3 is concerned, pure-go option we indeed only have this cznic available, which is nice.
Its pretty clear you dont know what youre talking about. It literally says its SQL in the title, you know that big bar at the top of your screen:
> Document-oriented, embedded SQL database
There's of course safety issues with C, but sqlite is one of the most rigorously tested sw packages out there.
for go-sqlite3 you still need native sqlite3 preinstalled on the OS, which is OK for developers but not for many ordinary users.
Ignoring that leads to long postmortems about how your embedded db failed in a way you didn’t expect: https://blog.roblox.com/2022/01/roblox-return-to-service-10-...
SQLite is good
Rocksdb is good
You don’t need pure go.
His paycheck alone should be tens of millions of dollars.
He should be making so much money that if he'd drop a 100 dollar bill it wouldn't be worth his time to pick it up.
For one, are you ready to write tests the way SQLite does?
see https://www.sqlite.org/testing.html
BTW, modernc.org/sqlite (by the same author) is a port of SQLite to native Go. I've used it with great success for a few personal projects. Before that, I used the go-sqlite package, which needs CGO, and makes it hard to build static executables.