Predictable finalisation. To build say, an operating system, you need to know exactly when memory is freed. If you don’t know this at a fundamental level, how can you schedule work? This IMO is fundamentally why no one has built a modern OS in C# alone.
Zero stdlib comes with no GC and seems to be viable for bare metal, so pretty much what you're describing.
Although you don't need to remove GC for everything. IDisposable already covers explicit destruction and the memory mapping can be handled with native functions where necessary. (I don't know if that's possible in bflat, I'm talking in general - but pinvoke should be enough)
Hence why system programming languages with GC also have other ways to manage resources (including memory), just because a GC is available doesn't mean it has to be used for everything.
See the whole linage of Mesa/Cedar, Modula-2+, Modula-3, Oberon, Midori,... OSes.
C#, like those languages, allows for manual memory management, unsafe code, stack allocation. MSIL was designed to also support languages like C++, and since C# 7, many of those capabilities are no longer only available to C++/CLI, rather being incremently exposed to C# as well.
Besides being a GC algorithm anyway, RC has performance issues, which can only be solved by using multiple optimization techniques from tracing GC, like background threads for cascading deletion of complex graphs, deferred counter manipulation, lock free data structures,...
Adding features to languages by anyone but the language designer doesn't work. It's unsupported, it affects too many other parts of the stack and leads to incredibly hard to spot bugs. You, or any project, can't do this.
Also: doing this means as soon as you start using predictable finalisation to close DB connections, or anything, your code stops being C# code. It can no longer be used except in a highly nontrivial environment. It depends, in an undeclared manner, on a nonstandard runtime.
So for open source code, or anything you're hoping to reuse: please don't do this.
If you want this, switch to C++, Swift or Rust, or some other language that has it built in.
Exactly. Idiomatic C# code does not depend on finalizers to release resources. Finalizers are typically only used as a last resort to prevent handle leaks, it is only memory deallocation that is nondeterministic.
Also for hard real time systems (which do not necessarily include operating systems). You need to have a deterministic computation time each cycle and never go over the time budget allocated. With a GC that isn't reference-counted it's nondeterministic. It could also be nondeterministic with reference counting if run e.g. into a loop that allocates a lot of objects and then they are freed, but you can measure and control and fix this behavior more easily than when trying to work against a non-resource-counting GC.
Like others said here, it could certainly be done in C#, but you have to fight the language on some level. Events/delegates, strings and almost all collection classes are heap based and therefore can trigger the GC. It's difficult to do useful stuff without them.
But you can do what you'd do in typical hard real-time applications anyway anyway, that is, preallocate everything you think you'll need. The APIs are there.
The most problematic data type is the string, it's almost impossible to do anything in C# without it, and it's in the heap, therefore can trigger the GC. Also it's immutable, so almost all useful operations on it produce another string(s). See Joe Duffy's old blog post here [0].
However Using the new Span<char> or ReadOnlySpan<char> which are stack based can alleviate that. Almost all string actions can be done on them. There's also a library called ZString which can be useful [1].
Another problem is the jit, which in "regular" dotnet works lazily, only on the first call to a function or even to string literals, so again you don't get deterministic execution. It is very quick but calling a new function chain in succession can add up. However using an AOT compilation like bflat does eliminates that problem.
Even without using AOT, you can approximate it by using reflection and calling RuntimeHelpers.PrepareMethod() on all methods in all referenced assemblies (except the system ones which should already be precompiled).
Is there still a point to this now that .NET 7 has NativeAOT and self contained EXEs? does this just give native trimming? Lack of macOS support seems like a non starter for a lot of cross platform cli tooling.
Regular .NET can’t do AOT for asp.net core. Can bflat? If so, that would be really helpful for example for lambdas. The cold start time is quite poor without AOT.
Except that isn't NativeAOT as it ships on the box using dotnet publish workflow, rather a curated workflow which requires special steps as shown on the efi-runtime example.
My remark was regarding the support you get with dotnet publish for NativeAOT, which by the way still doesn't support the GUI/EF scenarios from .NET Native.
ZeroSharp is done by the same person as bflat (me). These ZeroSharp samples get broken all the time because they bend the MSBuild targets that ship with .NET to an extreme. Bflat is a more supportable route and has a couple extra customizations to the compiler specifically for these scenarios.
Any thoughts on OSX support? I'm naive here, but DotNet 6 seems to work just fine (I haven't tried building bflat yet), and p/invoke seems straightforward enough:
I want bflat to be able to crosscompile from anything to anything.
Apple's platforms are difficult to crosscompile for - most instructions I've seen that need to reach dependencies beyond what libc offers require one to copy files from an Xcode installation and pray one didn't break EULAs. I don't even know where to start if I want to build from Linux or Windows to target macOS without a mac.
Then there's a philosophical issue that I have where Apple essentially would like everyone to pay up if they want to distribute software that targets their platforms (required code signing, etc.), and breaking backwards compat all the time (essentially treating developers like a resource to be exploited). This practice needs to go away. I know bflat not being available for macOS won't change that, but we need to start somewhere...
I agree with what you're saying, and it's of course entirely your call to make.
However, given that a significant percentage of developers are on Macs, this makes bflat a non-option for a large number of use cases. Even if I'm writing a library (and not an app or a service), I can't pick bflat because sooner or later someone will want to target Macs and iOS.
Also, will .Net V8 NativeAOT being able to target Apple Silicon help with the effort?
The crosscompilation on Go to mac is pretty limited. That's why I wrote "need to reach dependencies beyond what libc offers". If you crosscompile Go from Linux to mac, you'll actually get a different binary than if you compile on mac natively (DNS resolution will use Go's home baked version instead of the one provided by the OS). CGo also doesn't work if you crosscompile.
It's a bunch of work to even get a hobbled version of crosscompilation to macOS running.
Is there a simpler way you could achieve Try/Catch semantics and throw-like flow control, without having to check return values after each function call? eg. Even though VB6 lacks exceptions, I built such a framework for it out of primitives like "On Error Goto" (and helper tooling that would automatically wrap and instrument functions to propogate the Err details). It sounds a bit heavy but was in fact really slick in practice.
The way exceptions are done in .NET is by keeping track of extra metadata to be able to unwind methods (remove them from the execution stack) and execute handlers. That way the overhead of exception handling is close to zero if no exception is ever thrown. This requires quite a bit of infrastructure.
Propagating exceptions as error codes would come with execution time costs even if no exception is thrown and would have to be written from scratch (since .NET doesn't do it this way).
Either approach requires more investment than I'd want to put into it right now.
Note that .net supports exception filters (and the stdlib uses them), so replacing exceptions with error codes would observably change behavior in a big way.
Hi. I think this is an exciting development. It will be much work to replicate all dotnet standard libraries. Is there anything existing that can be ported/reused? For instance the original work on mono? Or is any of microsoft’s work licenced permissible enough to use?
Would you be able to provide support for dotnet 8 alpha? Super interested in this, but would need iOS,MacOS and wasm... Seems like cross-compile support for all those targets are already in 8.0.0 alpha?
Awesome project! C# is my preferred language, but I love the less-heavyweight tooling in other langs, so bflat fits nicely. What inspired you to start this / what was your use case? Where did existing tooling fall short for you? I had a brief look at the repo but didnt see a 'Why' or similar section.
I also like the less heavyweight approach where one can compose tools to do things. The existing .NET tooling is so entwined with MSBuild that one can't even (easily) invoke the C# compiler without an MSBuild project file.
Don't get me wrong, I like MSBuild and even bflat is built with MSBuild (wrote a custom target to invoke bflat from MSBuild), but sometimes I really just want to write a Makefile and be done with it instead of fiddling with MSBuild targets.
bflat integrates nicely with Make or other build systems - pass `-c` to generate object files (link it later with the commands bflat would use - use `-x` to see the commands), or use `bflat build-il` to create .NET assemblies and pass those as a reference (`-r`) to `bflat build` later.
I'm a pedant, I know, but I can't deal with this being called B♭ (the enharmonic equivalent for A♯), instead of D♭ (the enharmonic equivalent of C♯). Is there another layer of the joke that I'm missing?
ultra-pedantically: that's true with the "well tempered" tuning that is used almost universally. But this is really just an approximation, and one that our ears have grown used to.
Mathematically speaking, B# <> C. On a piano or other instrument that quantizes tones, we have little choice but to go with the approximation. But for violinists and others that can choose from a spectrum of tones, it's possible to hit those tones exactly. Of course, when playing with others, the violin still needs to quantize just to be in tune with the others.
We changed the URL from https://github.com/bflattened/bflat to the project page. Both are nice, but the project page is more unique. Yes I know you can't be "more unique".
Some executable file formats have time stamps in them.
bflat follows the platform convention that respects this and writes current date/time in the file header by default.
It is also a platform convention to offer options to set these to something stable (e.g. `/Brepro` on lld-link, or `--deterministic` for bflat) if one wants stable output.
More than 20 years ago, there was another C-like compiler for Flat Real Mode[0] called BFlat[1][2]. I want to say there are other tools called BFlat or some near variant.
Well, this looks like fun! `Bflat build` doesn't seem to support .csproj files, so testing this on existing code wasn't a huge success for me right away (and of course the trivial stuff always works), but definitely worth looking into later.
Quite interesting that this is literally the first time I've come across this project, despite it being in a space I'm interested in and even having been discussed here previously...
That's wonderful! I think C# is pretty decent, but totally hated working with the tooling. Thanks for making this and thanks to MS for producing some open source tooling enabling it.
This puts .NET back into the "stuff you can use" territory, which is nice.
Though, to be honest, the rest of the ecosystem (language, patterns, libraries) could still use some polish too. But it's a great step!
It is common for some Microsoft groups to sabotage the work of others.
See Longhorn, with its design ideas being redone in COM from Vista onwards.
XNA being dropped on the floor, replaced by DirectXTK.
C++/CX being replaced by C++/WinRT with complete disregard for the developer experience, and when they got tired of working on it, they left it on its C++17 state without VS tooling, and started hacking away on Rust/WinRT, which still can't even support COM authoring.
XAML Islands being replaced by WinUI 3/WinAppSDK without the same level of tooling and feature set as UWP and WinUI 2.x.
Which might help them achieve whatever KPIs they need, but burn bridges in the developer community, and then they are surprised that even the most hardcore supporters no longer jump into adopting their tech solutions as in the "Developers, Developers, Developers" days.
Looks interesting, will review, but just a note: the grey text is unreadable in bright conditions. Sitting here having breakfast in direct sunlight and I can't really see the text.
It says in "Zero" that there is no garbage collector, does that mean you can't use any reference-type objects at all? Or does that mean that reference-type objects stay allocated forever?
I think it means that reference type objects stay allocated forever, but I'm not sure. From [0]: If an API allocates as part of its operation and doesn't return that as result, it's not good. We don't have a GC. This is all memory leaks.
The license is confusing. Some source files say "GNU Affero General Public License (AGPL)", but the homepage says "MIT license".
Does this mean that if you build a project with this compiler, your code is distributed under the terms of the GNU Affero General Public License (AGPL)?
92 comments
[ 0.25 ms ] story [ 175 ms ] threadWhy do you say so?
Although you don't need to remove GC for everything. IDisposable already covers explicit destruction and the memory mapping can be handled with native functions where necessary. (I don't know if that's possible in bflat, I'm talking in general - but pinvoke should be enough)
See the whole linage of Mesa/Cedar, Modula-2+, Modula-3, Oberon, Midori,... OSes.
C#, like those languages, allows for manual memory management, unsafe code, stack allocation. MSIL was designed to also support languages like C++, and since C# 7, many of those capabilities are no longer only available to C++/CLI, rather being incremently exposed to C# as well.
Besides being a GC algorithm anyway, RC has performance issues, which can only be solved by using multiple optimization techniques from tracing GC, like background threads for cascading deletion of complex graphs, deferred counter manipulation, lock free data structures,...
Project Oberon source code,
http://www.projectoberon.com/
Active Oberon source code (the latest language OS from the Oberon tree still being actively developed),
https://gitlab.inf.ethz.ch/felixf/oberon
It would be great if Microsoft some day releases Midori source code.
Waiting for this since years. But it won't happen, I guess, as it didn't happen until now.
Also: doing this means as soon as you start using predictable finalisation to close DB connections, or anything, your code stops being C# code. It can no longer be used except in a highly nontrivial environment. It depends, in an undeclared manner, on a nonstandard runtime.
So for open source code, or anything you're hoping to reuse: please don't do this.
If you want this, switch to C++, Swift or Rust, or some other language that has it built in.
Memory deallocation can be done as deterministic as malloc/free via the native heap APIs.
https://en.wikipedia.org/wiki/Singularity_(operating_system)
https://www.gocosmos.org/
https://www.wildernesslabs.co/
http://joeduffyblog.com/2015/11/03/blogging-about-midori/
Like others said here, it could certainly be done in C#, but you have to fight the language on some level. Events/delegates, strings and almost all collection classes are heap based and therefore can trigger the GC. It's difficult to do useful stuff without them.
But you can do what you'd do in typical hard real-time applications anyway anyway, that is, preallocate everything you think you'll need. The APIs are there.
The most problematic data type is the string, it's almost impossible to do anything in C# without it, and it's in the heap, therefore can trigger the GC. Also it's immutable, so almost all useful operations on it produce another string(s). See Joe Duffy's old blog post here [0].
However Using the new Span<char> or ReadOnlySpan<char> which are stack based can alleviate that. Almost all string actions can be done on them. There's also a library called ZString which can be useful [1].
Another problem is the jit, which in "regular" dotnet works lazily, only on the first call to a function or even to string literals, so again you don't get deterministic execution. It is very quick but calling a new function chain in succession can add up. However using an AOT compilation like bflat does eliminates that problem.
Even without using AOT, you can approximate it by using reflection and calling RuntimeHelpers.PrepareMethod() on all methods in all referenced assemblies (except the system ones which should already be precompiled).
[0] https://github.com/joeduffy/joeduffy.github.io/blob/master/_...
[1] https://github.com/Cysharp/ZString
https://www.ptc.com/en/products/developer-tools/perc
https://www.aicas.com/wp/products-services/jamaicavm/
It just happens that .NET never went down this path.
Regular .NET can’t do AOT for asp.net core. Can bflat? If so, that would be really helpful for example for lambdas. The cold start time is quite poor without AOT.
As for the cold start you can use tiered JIT and Ready2Run, until full AOT becomes available.
https://github.com/MichalStrehovsky/zerosharp/tree/master/ef...
My remark was regarding the support you get with dotnet publish for NativeAOT, which by the way still doesn't support the GUI/EF scenarios from .NET Native.
I think at this point https://flattened.net/ is a better introduction to bflat than the repo. The repo also links to it, but wanted to highlight it.
https://learn.microsoft.com/en-us/dotnet/standard/native-int...
Apple's platforms are difficult to crosscompile for - most instructions I've seen that need to reach dependencies beyond what libc offers require one to copy files from an Xcode installation and pray one didn't break EULAs. I don't even know where to start if I want to build from Linux or Windows to target macOS without a mac.
Then there's a philosophical issue that I have where Apple essentially would like everyone to pay up if they want to distribute software that targets their platforms (required code signing, etc.), and breaking backwards compat all the time (essentially treating developers like a resource to be exploited). This practice needs to go away. I know bflat not being available for macOS won't change that, but we need to start somewhere...
However, given that a significant percentage of developers are on Macs, this makes bflat a non-option for a large number of use cases. Even if I'm writing a library (and not an app or a service), I can't pick bflat because sooner or later someone will want to target Macs and iOS.
Also, will .Net V8 NativeAOT being able to target Apple Silicon help with the effort?
Or you literally mean that you don't want to do it because of ideological reasons regardless of the aforementioned technical argument?
It's a bunch of work to even get a hobbled version of crosscompilation to macOS running.
Luckily this should be solved in 1.20
https://go-review.googlesource.com/c/go/+/446178/
Is there a simpler way you could achieve Try/Catch semantics and throw-like flow control, without having to check return values after each function call? eg. Even though VB6 lacks exceptions, I built such a framework for it out of primitives like "On Error Goto" (and helper tooling that would automatically wrap and instrument functions to propogate the Err details). It sounds a bit heavy but was in fact really slick in practice.
The way exceptions are done in .NET is by keeping track of extra metadata to be able to unwind methods (remove them from the execution stack) and execute handlers. That way the overhead of exception handling is close to zero if no exception is ever thrown. This requires quite a bit of infrastructure.
Propagating exceptions as error codes would come with execution time costs even if no exception is thrown and would have to be written from scratch (since .NET doesn't do it this way).
Either approach requires more investment than I'd want to put into it right now.
Don't get me wrong, I like MSBuild and even bflat is built with MSBuild (wrote a custom target to invoke bflat from MSBuild), but sometimes I really just want to write a Makefile and be done with it instead of fiddling with MSBuild targets.
bflat integrates nicely with Make or other build systems - pass `-c` to generate object files (link it later with the commands bflat would use - use `-x` to see the commands), or use `bflat build-il` to create .NET assemblies and pass those as a reference (`-r`) to `bflat build` later.
Edit: Db language https://www.codeproject.com/Articles/13639/Db-The-Future-Is-...
bflat triggers a lot of engagement on Twitter and elsewhere because everyone wants to correct it. Couldn't have chosen a better name.
C D E F G A B [C]
is
do re mi fa sol la si [do]
mi (E) and si (B) end with "i" and are the two ones that are half a step away from the next ones.
Mathematically speaking, B# <> C. On a piano or other instrument that quantizes tones, we have little choice but to go with the approximation. But for violinists and others that can choose from a spectrum of tones, it's possible to hit those tones exactly. Of course, when playing with others, the violin still needs to quantize just to be in tune with the others.
Well-temperament still has different intervals depending on key. Even temperament just uses x^n/12 for note n and basis frequency x.
("CIL is object-oriented, stack-based bytecode" implies there's quite a bit of flattening to be done before reaching machine code.)
https://github.com/bflattened/bflat/issues/6
Higher entropy? We talk about number of bits of identifying information in privacy issues; same idea seems applicable here.
Bflat: C# as you know it but with Go-like tooling - https://news.ycombinator.com/item?id=27634456 - June 2021 (185 comments)
Show HN: C# compiler with Go-like tooling (build small/standalone/native apps) - https://news.ycombinator.com/item?id=27628817 - June 2021 (1 comment)
bflat follows the platform convention that respects this and writes current date/time in the file header by default.
It is also a platform convention to offer options to set these to something stable (e.g. `/Brepro` on lld-link, or `--deterministic` for bflat) if one wants stable output.
[0] https://en.wikipedia.org/wiki/Unreal_mode
[1] http://www.lanet.lv/simtel.net/msdos/misclang-pre.html
[2] http://www.lanet.lv/ftp/simtelnet/msdos/misclang/bflat050.zi...
How would one go about accessing something like basic network sockets using the zero lib? I assume there's a lot missing there.
Quite interesting that this is literally the first time I've come across this project, despite it being in a space I'm interested in and even having been discussed here previously...
(Technically, there used to be a language called B, too, but I don't think it has been in use since before I was born.)
That's wonderful! I think C# is pretty decent, but totally hated working with the tooling. Thanks for making this and thanks to MS for producing some open source tooling enabling it.
This puts .NET back into the "stuff you can use" territory, which is nice.
Though, to be honest, the rest of the ecosystem (language, patterns, libraries) could still use some polish too. But it's a great step!
i will test
Long live to the people who worked on CoreRT/NativeAOT and the people who pushed for it
Great tech that will empower great projects
An example: https://icculus.org/finger/flibitijibibo
See Longhorn, with its design ideas being redone in COM from Vista onwards.
XNA being dropped on the floor, replaced by DirectXTK.
C++/CX being replaced by C++/WinRT with complete disregard for the developer experience, and when they got tired of working on it, they left it on its C++17 state without VS tooling, and started hacking away on Rust/WinRT, which still can't even support COM authoring.
XAML Islands being replaced by WinUI 3/WinAppSDK without the same level of tooling and feature set as UWP and WinUI 2.x.
Which might help them achieve whatever KPIs they need, but burn bridges in the developer community, and then they are surprised that even the most hardcore supporters no longer jump into adopting their tech solutions as in the "Developers, Developers, Developers" days.
[0] https://github.com/bflattened/bflat/tree/master/src/zerolib
https://github.com/bflattened/bflat/blob/master/src/zerolib/...
Can you build Minecraft with this thing? Can you still build Minecraft at all?
https://github.com/bflattened/bflat/blob/master/src/zerolib/...
They use CIL as opposed to C#.
Does this mean that if you build a project with this compiler, your code is distributed under the terms of the GNU Affero General Public License (AGPL)?