It's a defragmenting stack, garbage collected programming language, and even mentioned as a memory-safe programming language in NSA's "Memory Safe Programming Languages Recommendations".
Not the glib root-commenter, but the two big caveats is that Go is memory-safe provided you import neither `unsafe` nor `c`.
That can end up being a pretty restrictive constraint, if it's not something your dependencies cared about. If it is a concern of yours, it's pretty easy to check for at build time.
Well, unsafe says it all. It's "unsafe". On the other hand, I never had to import "C" in any of my projects, yet.
However, if I'm going to wander into CGo realm, I can always pivot to C++, if speed or things implemented in C is that important for me.
No programming language, incl. a particular one which has an orange mascot, is perfect. I also find trying to solve social problems with technology (i.e. let's make programming languages foolproof to enable lazier-er programmers), pointless, to put it mildly.
Honestly, in my eyes Go is a very nice, little language, and I like how it handles.
In my experience, a typical project in orange-mascot language is also more likely to bring in a C dependency through a -sys dep than a typical project in blue-mascot language. (That might be changing with more RIIR, though.)
WASM is a funny case and I don't agree that it's safe.
When people say WASM is "safe" in this sense what they mean is that it won't corrupt your browser process (or other wasm runtime). That's a useful guarantee! But that's sandboxing, not memory-safety as a language property. You can sandbox C too, that doesn't make C a memory-safe language.
As far as I know you can still trigger unsafe access to the WASM heap, and therefore many kinds of of attacks still work. They don't "break out" but also, lots of valuable data or controllable user behavior is in-heap anyway.
Or, for an even more analogous example, see Google's pNaCl which was a native compilation target that provided many of the same security guarantees as wasm does while avoiding the need for complex JITs. pNaCl utilized a set of rules about control flow transitions and address formation which were enforced by a verifier to allow direct native code execution.
Having tools to avoid data races don't make it a memory-safe language, unless you strip that phrase of all meaning.
The more interesting question is, would it be worth it to be memory-safe? Rust says yes, and puts a lot of work into statically preventing data races at the cost of (at least) some language ergonomics. Java also says yes, but put a lot of work into making data race impacts well-defined, and on this tends to incur cost (runtime and language development time) with unclear benefits towards actually reasoning about the programs with such races.
Data races can lead to logic bugs that are potentially security sensitive, but memory safety generally refers to getting rid of all control flow hijacking security vulnerabilities which, in many large C/C++ projects, seem to be never-ending. Avoiding data races is more in the category of preventing bugs/crashes while memory safety is about avoiding an entire class of security vulnerabilities.
I've never seen a single control flow hijack PoC from go data races even in a code golf scenario. I've always wondered if this is possible but I don't think it is. Care to share an example?
The current implementation is not memory safe. Go uses 128 bit structs for interfaces and slices and assumes that they are updated atomically (when they are not).
It's possible to exploit this to read/write arbitrary memory and to execute arbitrary code. It appears that in practice this is very difficult so the issue is ignored.
The showcase of projects is (a bit unintuitively) listed under Available packages and is helpful if you want to get a picture of what you can build on top of gokrazy: https://gokrazy.org/packages/showcase/
I am very fond of the idea of a from-scratch userspace written entirely in Go. I've been dreaming of building something like gokrazy for servers. Or possibly something like Bottlerocket OS in Go.
I experimented a few weeks ago with packaging a custom kernel and minimal Go init binary into a disk image bootable on GCE. No bootloader needed, just EFI loading the kernel directly.
Same - I've been playing around with Unikernels for the same reason. Running my web server as the only process on the machine really appeals. Attack surface is tiny, basically non-existent. If the web server falls over, reboot the machine (which should be fast because it only has to load the kernel & the Go program).
It's basically the opposite of containers, which I like hehe.
That's fascinating, thanks for the link. But I'm coding in Go, and it looks like (from skimming the docs) that MirageOS only works with OCaml. I'll have to look further.
Another project that aims to deliver this is Linuxkit (https://github.com/linuxkit/linuxkit). All the components they ship are written in memory safe languages (usually Go) and run as containers under containerd. You can build a custom image very easily, fully defined as a YAML file.
There is also u-root which is "A fully Go userland with Linux bootloaders! u-root can create a one-binary root file system (initramfs) containing a busybox-like set of tools written in Go." https://github.com/u-root/u-root
Go simply can't really do well unless it has globs of memory or properly set kernel parameters (that affect the whole global machine). To get it to run reliably in low-memory conditions also requires careful tuning of GOGC + GOMEMLIMIT + MADV_DONTNEED. Go is geared more towards servers with tons of available memory, not IoT devices. Further, it's networking stack does really well on low-latency connections (ie, a datacenter) and starts to do poorly if you throw a in link with lots of data loss (this includes clients across the internet on crappy wifi).
I love that they basically reimplemented libc to get more performance, but they also threw away decades of edge cases that it doesn't (yet) handle very well. I'm confident they'll get there, eventually, especially when people do things like this and report issues.
This looks like a really interesting project. Thanks for sharing it! I'm going to have to try this out with a couple of projects. Just a cursory experiment reveals that it's GC seems to be much better than Go's GC and I'm very curious as to why that is.
Interesting that it says it is slower. It may actually be slower, but tends to not suffer as much degradation as the Go's GC. In other words, on my test setup using Go, I get ~1k rps once I hit memory limits, but with tinygo I get 7k rps once I hit memory limits. So it might be "slower" but it looks like it may be more efficient as well.
TinyGo is not Go, it's more accurate to see it as a dialect rather than a separate Go implementation imo. Many packages (even the stdlib) don't compile under it, and as I understand it, certain language features involving runtime reflection are likely to never be supported.
Just like what many people call C is not ISO C, and similarly many ISO C libraries don't compile with "C" compilers for the same environments as TinyGo.
The targets of this project are RPis (with 512MB+ RAM) or similar. That’s tons of RAM compared to “IoT devices” you have in mind, and not an issue at all for Go.
2Gi of ram is nearly the bare minimum to run a single Go server without suffering (huge) performance penalties -- we're talking a 50-99% degradation of performance while GC sucks up all the CPU power to gain more memory.
It's not a matter of allocation since there isn't a way to manually free memory in Go. The GC basically doesn't do anything if there isn't memory pressure. Once your applications starts feeling some memory pressure, the GC will start spending CPU to free up memory. This is a non-disputable fact of how Go works, in general.
It's absolutely a matter of allocation, since the more you allocate, the more work the GC will have to do. If you write clean, memory-friendly code (e.g. reusing buffers, favouring stack allocation), then you'll generate a lot less garbage and the GC will spend less time collecting it.
Without talking about your use case, this is such an absurd claim that doesn’t deserve a response. The rest of us can easily run a dozen or more typical light load HTTP services within 512MB.
Light load as in serves a couple rps persistently without problem (complete with SQLite database operations), and I’m not talking about highly optimized code. Enough for most home or small business intranet use cases. You’re not going to serve google.com in 50MB, obviously. If your code has to allocate ~100 megs for a request without an obvious reason that requires that much allocation, that’s on your code, not the language.
I don't know how you'd define "light load", but I have a few services that average about 100rps. Only 1 active container most of the time, they're allocated 128MiB RAM, with a p99 utilization of 55%. They don't have any GC config or special optimization - just operating with regular JSON POST requests, making MongoDB queries
Go just does incredibly well in these kinds of conditions. The only reason we allocate 128MiB is because that's the minimum Google Cloud Run allows - we'd probably go for 64MiB on most systems otherwise
No, request rate doesn't really matter. Increased request rate will increase the rate of the GC running, but it doesn't intrinsically increase the amount of RAM used. (More simultaneous requests can, of course, but that isn't necessarily implied by an increased request rate.)
Look, let me blunt, since you have insistently posted several times: You don't know what you're talking about. The code is open source, the algorithms publicly available, and it simply is not the case that Go is programmed to simply run up to gigabytes of memory usage before going "hurr durr I guess I should free some stuff". Like many other posters here, I also in possession of numerous 24/7 Go servers that run all sorts of things through them without needing gigabytes of RAM.
I can tell you what happened to you with high probability. You wrote a memory leak, and mistook the result for Go's error rather than yours. Hit your leaking code with the built-in memory profiler and figure out what you're leaking and fix it. It's OK, it happens to all of us, I've written many such leaks in many different languages, with many different allocation schemes.
RPis are abundantly capable of running Go. They can run a lot of Go. They're way, way above minimum spec. Of course Go won't let you magically hold a decompressed video stream in memory any more than anything else does or work other magic, but it's a heck of a lot more efficient than Python, and can run an awful lot of server in the resources a web browser takes just to display a blank window.
I’m bemused by these sorts of disparaging comments that come from people who seem to have no real world experience running Go applications at scale.
It’s entirely possible to run a server that will handle tens of thousands of req/sec on hardware with comparatively small amounts (by todays standards) of RAM e.g. 512MB
I don't see how you think my comments are "disparaging," this is just code doing what it is programmed to do.
> It’s entirely possible to run a server that will handle tens of thousands of req/sec
Indeed, you can do this. I'm talking about what happens once the Go starts getting memory pressure and the GC starts dominating the CPU. This is what it is designed to do. If your code can prevent that from happening, you are correct then, as it won't be an issue.
Even Java 17 years ago is ironic, considering the hardware requirements of J2ME, Java Embedded, Blue Ray players, Ricoh and Xerox copiers, Cisco phones, and so forth.
Prior to GOMEMLIMIT being a thing, people would get themselves into this situation occasionally. Their steady state working set size is something like 2.1GB, but they want to fit into a 4GB container limit, so GOGC=100 led to OOMs occasionally. They would bring GOGC down to 10 or something (why not 90!?), and then they are running GC every time they allocated 10% or 200MB, and then their CPU profiles would show they were burning a lot on GC, because it's pretty easy for an RPC server to generate 200MB of garbage.
GOMEMLIMIT definitely improves things, taking the very blunt instrument GOGC out of people's hands, and does what they actually want instead, which is to prioritize GC only when you're about to run out of memory.
People dislike tunables like this, and rightfully so, but a similar problem exists somewhat insidiously in non-garbage collected languages. Calling malloc() and free() is not a zero-cost operation, and you can certainly waste CPU by calling free() when nothing actually needs the memory you are freeing. GC lets cases like "this is a CLI app, the system has 128GB of RAM, the working set size is 2GB, and it's over in 1 second" basically never do any memory freeing. You allocate as much as you need, GC never runs, and the app exits. That kind of optimization is actually pretty neat. But, for steady state servers that run for an unbounded period of time, it is inevitable that you are going to spend time trying to reuse memory that you no longer need, because the system only has a finite amount of memory. Then the compromises start to balance throughput with efficiency. gc + GOMEMLIMIT is a very nice compromise that is excellent for many applications, but no heuristic is perfect for every possible computer program.
I have written a bunch of tiny Go apps, and agree with you. In general, 64MB is where I set the limit on these things and there are no problems. I have some big applications that literally allocate 1GB at once and they still run OK within 2GB memory limits. (Obviously, you can't do two of these operations at once, though.)
That's bollocks. I've got a 4GB server which runs 5 production services, and (maximally) 20 test services, and only one of those ever goes over 40MB, and then drops below that again. The production services handle about 65k requests per day. Not much, but it shows that you can easily run go applications in less than 2GB.
It's not an exact comparison, but at work we have tons of Go apps running in Docker containers with a 50MB memory limit. For lighter workloads, it works just fine.
This project always seemed to be swimming against the current by using Go but for the longest time no one would state the obvious. Finally. Now I know I'm not krazy.
I'm running gokrazy on a raspberry pi zero 2 w with 512MB of ram. I'm using it as a record-on-motion security camera[0]. The actual recording happens with ffmpeg, but the Go server does motion detection, hls streaming, and cloud upload of video chunks. It also does this all in-memory, the sdcard in the pi is mounted read-only.
It works great! No memory pressure/GC issues at all. Its really not that hard to write go that can perform well in moderately constrained environments.
I've seen fairly naive Go servers that do 10,000+ RPS that issue database requests on every request that consistently use less than 50MB of RAM without issue. Several of them barely ever break 30MB of RAM.
However, I don't like to have a Go compiler on the production machine for security reasons, as well as to reduce even more the system base.
I think instead that the compiler should be needed only to deploy binaries and should stay on the syadmin's machine. The Go toolchain is very good at cross-compiling, so there should not be a need for the compiler on the target machine.
I don’t know where you got the impression that the Go compiler would run on “the production machine”.
gokrazy images are built whereever you like (typically your PC, or some more automated setup) and then deployed to the target device. There is no Go toolchain on the target device.
Oh this seems great, I actually just wrote a small Go program that I was planning on running on a Raspberry Pi as the only thing running on the device, looks like this could make it a bit simpler to deploy, very nice!
The idea is not bad. Execution is always hard with this stuff. I've done a pure Go app platform with this sort of desire for a native tool chain but it's very hacky. I do wonder whether wasm will solve some of this but sometimes it's a solution in search of a problem. Still the simple idea of a server that will pull and run your code on demand with a simple "run x" is always appealing.
Go is pretty performant, faster than Nerves for sure, I have an old RPB+ from 2014 and some go code on it, works just fine. The Go compiler can target most popular arm platforms by default.
They are different beasts. Nerves does OTA updates (blue/green deploys), and a lot of other cool stuff. Needs beefier hardware but I started building a pretty wicked little tool on rpi4 with nerves to try and accurately track motorcycle lap times using gps and sensor fusion. It was pretty snappy in my experience.
103 comments
[ 4.6 ms ] story [ 165 ms ] threadWhy disagree? That's true.
That can end up being a pretty restrictive constraint, if it's not something your dependencies cared about. If it is a concern of yours, it's pretty easy to check for at build time.
[1] https://go-proverbs.github.io
However, if I'm going to wander into CGo realm, I can always pivot to C++, if speed or things implemented in C is that important for me.
No programming language, incl. a particular one which has an orange mascot, is perfect. I also find trying to solve social problems with technology (i.e. let's make programming languages foolproof to enable lazier-er programmers), pointless, to put it mildly.
Honestly, in my eyes Go is a very nice, little language, and I like how it handles.
(I rather suspect triyambakam was referring to undefined behavior on data races.)
[0] https://news.ycombinator.com/item?id=38613031
When people say WASM is "safe" in this sense what they mean is that it won't corrupt your browser process (or other wasm runtime). That's a useful guarantee! But that's sandboxing, not memory-safety as a language property. You can sandbox C too, that doesn't make C a memory-safe language.
As far as I know you can still trigger unsafe access to the WASM heap, and therefore many kinds of of attacks still work. They don't "break out" but also, lots of valuable data or controllable user behavior is in-heap anyway.
So, I don't see a problem here.
The more interesting question is, would it be worth it to be memory-safe? Rust says yes, and puts a lot of work into statically preventing data races at the cost of (at least) some language ergonomics. Java also says yes, but put a lot of work into making data race impacts well-defined, and on this tends to incur cost (runtime and language development time) with unclear benefits towards actually reasoning about the programs with such races.
Go makes this much harder to do than C/C++, especially as it relates to e.g. mishandling user input, but not impossible, even without CGo or unsafe.
It's possible to exploit this to read/write arbitrary memory and to execute arbitrary code. It appears that in practice this is very difficult so the issue is ignored.
https://blog.stalkr.net/2015/04/golang-data-races-to-break-m...
I experimented a few weeks ago with packaging a custom kernel and minimal Go init binary into a disk image bootable on GCE. No bootloader needed, just EFI loading the kernel directly.
For a low-powered device, would a Go usespace bring particular benefits? (Though, even the cheapest phones today aren't really "low-powered" anymore)
Edit: In order to preempt the argument others are having: Safer than C, anyways, while being much easier than rust.
I have used this in the past:
https://github.com/abice/go-enum
But mostly use protos as much as possible these days. (The enum is going to be in the API, might as well use the same object inside the app.)
https://www.withsecure.com/en/solutions/innovative-security-...
Look for TamaGo on the page.
It's basically the opposite of containers, which I like hehe.
https://github.com/cloudius-systems/osv
https://ops.city/ (also nanovms) - this is one that I actually got working to at least demo state
I didn't know about those. I kind of thought you may have used MirageOS, which I had read about earlier. It is done in OCaml.
https://mirage.io/
https://www.talos.dev/ https://github.com/siderolabs/talos
Or TinyGo, https://tinygo.org/docs/reference/microcontrollers/
I love that they basically reimplemented libc to get more performance, but they also threw away decades of edge cases that it doesn't (yet) handle very well. I'm confident they'll get there, eventually, especially when people do things like this and report issues.
It's absolutely a matter of allocation, since the more you allocate, the more work the GC will have to do. If you write clean, memory-friendly code (e.g. reusing buffers, favouring stack allocation), then you'll generate a lot less garbage and the GC will spend less time collecting it.
> Once your applications starts feeling some memory pressure
I rest my case.
It will run every 2 minutes, if it hasn't run otherwise
https://github.com/golang/go/blob/9b4b3e5acca2dabe107fa2c3ed...
Go just does incredibly well in these kinds of conditions. The only reason we allocate 128MiB is because that's the minimum Google Cloud Run allows - we'd probably go for 64MiB on most systems otherwise
Look, let me blunt, since you have insistently posted several times: You don't know what you're talking about. The code is open source, the algorithms publicly available, and it simply is not the case that Go is programmed to simply run up to gigabytes of memory usage before going "hurr durr I guess I should free some stuff". Like many other posters here, I also in possession of numerous 24/7 Go servers that run all sorts of things through them without needing gigabytes of RAM.
I can tell you what happened to you with high probability. You wrote a memory leak, and mistook the result for Go's error rather than yours. Hit your leaking code with the built-in memory profiler and figure out what you're leaking and fix it. It's OK, it happens to all of us, I've written many such leaks in many different languages, with many different allocation schemes.
RPis are abundantly capable of running Go. They can run a lot of Go. They're way, way above minimum spec. Of course Go won't let you magically hold a decompressed video stream in memory any more than anything else does or work other magic, but it's a heck of a lot more efficient than Python, and can run an awful lot of server in the resources a web browser takes just to display a blank window.
It’s entirely possible to run a server that will handle tens of thousands of req/sec on hardware with comparatively small amounts (by todays standards) of RAM e.g. 512MB
> It’s entirely possible to run a server that will handle tens of thousands of req/sec
Indeed, you can do this. I'm talking about what happens once the Go starts getting memory pressure and the GC starts dominating the CPU. This is what it is designed to do. If your code can prevent that from happening, you are correct then, as it won't be an issue.
That does not happen. You must be thinking of Java 15 years ago.
GOMEMLIMIT definitely improves things, taking the very blunt instrument GOGC out of people's hands, and does what they actually want instead, which is to prioritize GC only when you're about to run out of memory.
People dislike tunables like this, and rightfully so, but a similar problem exists somewhat insidiously in non-garbage collected languages. Calling malloc() and free() is not a zero-cost operation, and you can certainly waste CPU by calling free() when nothing actually needs the memory you are freeing. GC lets cases like "this is a CLI app, the system has 128GB of RAM, the working set size is 2GB, and it's over in 1 second" basically never do any memory freeing. You allocate as much as you need, GC never runs, and the app exits. That kind of optimization is actually pretty neat. But, for steady state servers that run for an unbounded period of time, it is inevitable that you are going to spend time trying to reuse memory that you no longer need, because the system only has a finite amount of memory. Then the compromises start to balance throughput with efficiency. gc + GOMEMLIMIT is a very nice compromise that is excellent for many applications, but no heuristic is perfect for every possible computer program.
Still even with limitations, there was a ton of stuff we could do in MS-DOS, let alone a more powerful system with dual core and built-in networking.
Go does not need me much memory, it's in the same ballpark as a C++ server, using way way less memory than Java / C#.
It works great! No memory pressure/GC issues at all. Its really not that hard to write go that can perform well in moderately constrained environments.
[0]: https://github.com/psanford/rom-cam
However, I don't like to have a Go compiler on the production machine for security reasons, as well as to reduce even more the system base.
I think instead that the compiler should be needed only to deploy binaries and should stay on the syadmin's machine. The Go toolchain is very good at cross-compiling, so there should not be a need for the compiler on the target machine.
gokrazy images are built whereever you like (typically your PC, or some more automated setup) and then deployed to the target device. There is no Go toolchain on the target device.
How does Go fare on single core machines like older RPi0 models?
* https://nerves-project.org/