The benchmark doesn't accurately represent the real-world database performance because the dataset is too small (roughly half a gigabyte based on [1]?), meaning it fits into the page cache bypassing disk I/O.
I made a local change so that all Put benchmarks ran O(n) O(1) sized transactions and the results were quite different: Void was the slowest, followed by LMDB, Bold, LevelDB, then Badger.
I'd also wager the LMDB author would also (lovingly!) tell us we're holding it wrong
Go doesn't use the C calling convention, but has its own growable stack system and goroutine scheduler that maps to goroutines to threads. So a goroutine can't just call a C function directly.
In order to interface with C code safely, Go's runtime has to jump to the system stack and do some additional setup, make the call, and then switch back. (Adding to that, if the call takes too long, this prevents other goroutines on the same OS thread from running, so the scheduler must jump in and move those goroutines to a different thread.)
All of this is expensive, though we are talking about nanoseconds, not milliseconds. Performance is mostly a problem when doing lots of very quick calls (e.g. you're writing a game engine interacting with something like OpenGL) or lots of slow calls (causing scheduler trashing).
No, my understanding is that Rust uses normal stacks, and it uses a classic threading model, so aside from async, calling C doesn't need to any runtime stuff.
// voidDB is a cherished toy, a journey into the Unknown, a heroic struggle,
// and a work of love. It is the “Twee!” of a bird; a tree falling in the
// forest; yet another programmer pouring their drop into the proverbial [bit]
// bucket. Above all, it is a shrine unto simple, readable, and functional
// code; an assertion that the dichotomy between such aesthetics and practical
// performance is mere illusion.
I admire the author who took up the challenge of writing a KV DB and cherishing self's efforts. I hope someday I could do something similar and feel accomplished.
Love the poetry in the comment prose! Endeavour so much needed in the industry. It can only come out of the joy of coding, how else could it be born...
Thank you for highlighting. These contain references to some philosophies I relate to and find comforting, including those most prominently articulated by Ernest Becker and Alan Watts.
~/go/src/github.com/voidDB/voidDB git:(master)
$ go run github.com/tailscale/depaware@latest
github.com/voidDB/voidDB dependencies: (generated by github.com/tailscale/depaware)
github.com/voidDB/voidDB/common from github.com/voidDB/voidDB+
github.com/voidDB/voidDB/cursor from github.com/voidDB/voidDB
github.com/voidDB/voidDB/free from github.com/voidDB/voidDB
github.com/voidDB/voidDB/node from github.com/voidDB/voidDB+
github.com/voidDB/voidDB/reader from github.com/voidDB/voidDB
golang.org/x/sys/unix from github.com/voidDB/voidDB+
bytes from github.com/voidDB/voidDB+
cmp from internal/fmtsort+
encoding/binary from github.com/voidDB/voidDB/common+
errors from bytes+
D fmt from golang.org/x/sys/unix
hash from github.com/voidDB/voidDB+
hash/fnv from github.com/voidDB/voidDB
io from bytes+
io/fs from internal/filepathlite+
iter from reflect+
math from encoding/binary+
math/bits from golang.org/x/sys/unix+
os from fmt+
path from io/fs
reflect from encoding/binary+
slices from encoding/binary+
LD sort from golang.org/x/sys/unix
strconv from fmt+
LD strings from golang.org/x/sys/unix
sync from encoding/binary+
sync/atomic from internal/bisect+
syscall from github.com/voidDB/voidDB/cursor+
time from github.com/voidDB/voidDB+
unicode from bytes+
W unicode/utf16 from internal/poll+
unicode/utf8 from bytes+
It doesn't take much to split your go.mod file require blocks into needed dependencies, and test dependencies, with a few comments explaining the why of each dependency.
Go tooling will preserve your go.mod comments and structure.
You can also consider moving integration tests and benchmarks to another module in the same repo.
31 comments
[ 2.9 ms ] story [ 70.7 ms ] thread[1]: https://github.com/voidDB/voidDB/blob/master/test/bench_test...
For example:
- BenchmarkVoidPut runs a single O(n) sized transaction
- BenchmarkLMDBPut runs a single O(n) sized transaction
- BenchmarkBadgetPut runs O(n) O(n) sized transactions (!!!)
I made a local change so that all Put benchmarks ran O(n) O(1) sized transactions and the results were quite different: Void was the slowest, followed by LMDB, Bold, LevelDB, then Badger.
I'd also wager the LMDB author would also (lovingly!) tell us we're holding it wrong
In order to interface with C code safely, Go's runtime has to jump to the system stack and do some additional setup, make the call, and then switch back. (Adding to that, if the call takes too long, this prevents other goroutines on the same OS thread from running, so the scheduler must jump in and move those goroutines to a different thread.)
All of this is expensive, though we are talking about nanoseconds, not milliseconds. Performance is mostly a problem when doing lots of very quick calls (e.g. you're writing a game engine interacting with something like OpenGL) or lots of slow calls (causing scheduler trashing).
Huginn: thought, perception, comprehension, ...
Muninn: care, urge, wonder, curiosity, interest, ...
https://en.wikipedia.org/wiki/Huginn_and_Muninn#Etymology
https://github.com/voidDB/voidDB/blob/master/go.sum
... and this: https://go.googlesource.com/proposal/+/refs/heads/master/des...
Go tooling will preserve your go.mod comments and structure.
You can also consider moving integration tests and benchmarks to another module in the same repo.