Show HN: I'm making a dynamic language in Rust (github.com)
This started out as a learning project to teach myself Rust. It has grown into a decently substantial piece of software and I've learned quite a bit in the process!
Some neat things:
+ A garbage collector that can store dynamically sized types without any double-indirection (i.e. I have my own Box implementation with manual alloc/dealloc)
+ The smart pointer used to reference GCed data is a thin pointer. The ptr metadata needed for DSTs is stored in the GC allocation itself, so that the GC smart pointer is just a single usize wide. This allows me to keep the core value enum Variant down to 16 bytes (8 bytes for data, the enum discriminant, and some padding).
+ The GC also supports weak references!
+ Statically dispatched type object model using a newtype wrapper and Rust's declarative macros. Ok, what that means is that I have a MetaObject trait that I can use to easily add new data types and define the behavior for specific types. Similar idea to Python's PyTypeObject though very different in implementation. However, I don't resort to dynamic dispatch or trait objects despite working with dynamically type data. Instead, I have a newtype wrapper over the core value enum Variant that statically dispatches to each of the enum branches! And then a few macros that minimize the boilerplate required if I want to add a new branch to Variant or a new method to MetaObject (just a single line in each case).
+ Different string representations! This was inspired by the flexstr crate. Strings that are short enough to fit inside a Variant are "inlined" directly in the value. Longer strings are either GCed or interned in a thread-local string table. All identifiers are interned.
+ An efficient implementation of closures inspired by Lua's upvalues.
The language is still pretty WIP. I'm planning to add an import system, a small standard library, and a few other things
(Yes, the name might not be the best, being also used by a well-known ReST docs generator, I'll take suggestions. I do like the name though, both as a reference to the mythological creature and the cat :D)
48 comments
[ 2.5 ms ] story [ 148 ms ] threadPhixns (fixins), "the dyslexic sphinx".
I'm honored my string crate was able to inspire. Kinda surprised to see it mentioned as it isn't all that popular (yet? I hope...)
Of course, unsafe _can_ also be used to implement code that takes shortcuts beyond what the borrow checker may be able to handle. But if you're building an interpreter, speed is probably not a primary concern anyway.
This is true for a toy interpreter. You don’t “need” unsafe. But anything other then a toy interpreter has many reasons to reach for unsafe: garbage collection, flexstr, object dispatch, tagged pointers, etc. Not to mention the performance gains from using unchecked functions in hot sections of the interpreter loop.
I agree that writing an interpreter for an existing language (e.g. Python) one would want to match the performance of existing interpreters and thus would need to use the techniques you mention.
[1]: https://github.com/gleam-lang/gleam
A similar language to gleam built on WASM is grain: https://github.com/grain-lang/grain
Gluon is statically typed but it's also functional and a bit weird.
P.S. I know both Lua and Python and you are making syntax intentionally confusing. Do either "fun name() {}" or "def name():". Make it familiar.
You have a GCPtr to some cell on the heap, and you want to dereference the pointer to modify the cell. But the GC also needs to dereference the cell, e.g. to update object references after moving. So while a GCPtr is dereferenced, you must never trigger a collection, which means no allocation. If you do, you violate Rust's "only one &mut" rule, and also risk a dangling pointer. How do you enforce this?
One way you could enforce this is to require a shared reference to the GC to dereference any GCPtr. Instead of `cell.count += 1` you would write `cell.deref(gc).count += 1`. This works but is verbose.
Another way you could enforce it is dynamically, by setting a value in a cell whenever it is dereferenced; but this incurs a runtime cost.
How does Sphinx solve this?
[1] https://coredumped.dev/2022/04/11/implementing-a-safe-garbag...
If we were writing a GC for general Rust code then this and other issues around ensuring values are rooted would become much bigger problems. For example, the rust-gc crate has to deal with these. Just to be safe I took a page out of rust-gc's book and implemented a guard to ensure that the runtime will at least panic if I make a mistake and that happens.
- Bastet (Bast) - The beautiful goddess of cats, women's secrets, childbirth, fertility, and protector of the hearth and home from evil or misfortune.
- Mau - The divine cat who, in some stories, is present at the dawn of creation as an aspect of Ra.
https://www.worldhistory.org/article/885/egyptian-gods---the...
https://en.wikipedia.org/wiki/Cultural_depictions_of_cats
https://moderncat.com/articles/cats-mythology/#:~:text=What%....
Mau is cool too but for a lot of people I'd suspect it would be a meaningless word.
Maybe another cat and mythology related name? If you want to stick with the Egyptian theme, maybe something do to with Bastet: https://en.wikipedia.org/wiki/Bastet
Static vs dynamic: when a variable is declare does the compiler know and enforce the type? e.g. can I declare a variable as a string and then assign it an int to it? If you can then you have a dynamic language. If you can’t then you have a static language. Examples of static are Java, Rust, and C++. Examples of dynamic are Python, JavaScript, and Ruby.
The other attribute is weak vs strong typing. Weakly typed languages will coerce types where appropriate. for example if I try to compare “1” with 1 a runtime error occur for a strongly typed language, a weakly typed language will coerce “1” -> 1 and return “true”. Examples of strongly typed languages are Ruby and Java. Examples of weakly typed languages are JavaScript.
Mostly statically typed languages are strongly typed.
My definitions probably aren’t rigorous, but I think they’re good enough.
You can probably ask a large language model like GPT-N to generate a summary.
Crafting Interpreters is a great guide on this: http://www.craftinginterpreters.com
True, but if everyone who asked any question about something had to implement those things to even understand the basics of them, we'd never get anything done :)
You don't need a degree to be a developer (I don't have one) but I think it's a good indication that it is actually reasonable to think that every developer could implement a programming language. (And I mean developer as in someone who writes code professionally as their main job, not necessarily sysadmins or designers or whatnot).
To reiterate, I think many more people than think they can can develop their own language.
In this case, it comes down to whether the compiler tracks the "type" of every value when compiling (static typing) or whether types are stored somewhere in memory that is read from when your program is actually running.
Oh please, no. This is probably the worst feature of Python.
Maybe the choice of keyword is bad because of the association but it's also not quite the same as in Python. You don't have to "nonlocal x" up front in your function to access a nonlocal variable, which is a pretty huge difference IMO.
My purpose for "nonlocal" was to have a visual highlight of whenever a nonlocal variable is getting modified, because that's the exact point where assignment becomes a side-effect. So you only ever use it when assigning.
https://slideplayer.com/slide/4470263/
https://gamesfromwithin.com/managing-data-relationships
If I understand correctly, this approach is try and address the expression problem[1]. It makes it easier to define new types for `Variant` (everything is defined in one impl block) at the cost of making it harder to add new `MetaObject` functions (you need to update every applicable impl block).
Also static dispatch seems like the wrong word here, because you are still doing runtime dispatch on `Variant`, even if not using `dyn` Objects. Static dispatch typically refers to monomorphization.
[1]https://craftinginterpreters.com/representing-code.html#the-...
I am developing this music live coding language and audio library with Rust and it runs in browsers:
https://glicol.org
I am now using Rhai.rs as the embedding language to write `meta' node in the audio graph. But for audio programming, the running time for each block should not exceed 3ms. In some cases, I found Rhai quite struggling with that. Perhaps dynamic languages have an inherit limitation on that? Wondering how do you see this issue and will performance be part of the future consideration of sphinx-lang?
Tips to notice is WASM runs inside a sandbox, which means threads, random numbers generator(not sure for WASI), FFIs, etc. have to be moved out of the core to prevent them being compiled to WASM, which would probably fail. For major part of the code, they can be compiled to WASM and WASI(WASM System Interface) very easily.