From Ocaml I still use MLDonkey daily. I hope the multicore Ocaml brings a speed up for it. I am not saying MLDonkey is slow or unstable, but it could gain a performance boost on current machines.
A developer at our place did our log search tool in OCaml, the reason being "because I've never worked with it before". It had to be scrapped shortly after he left.
Naur's classic essay (and maybe my favorite essay in all of software engineering) "Programming as theory building" [0 ] comes to mind. If the theory of the program is lost then it's quite hard to continue work no matter what the language used.
Based on your description it might as well have been scrapped regardless of the programming language used?
No, the problem is that hardly noone had even heard of OCaml, much less had any experience with any functional language, much less knew how to maintain this tool.
My point is that it was very tone deaf to introduce a new technology onto the stack for this reason (so called resume-based development I think?).
People who make statements like this always seem to be programming language operators rather than computer scientists. If you are good you can make almost anything work.
Or they could be engineers. Cost benefit of learning a new language to support something designed and built in random language by some one who didn't actually know how to do high quality design work in that language is rather low.
It is like reading the first Turkish works of an English poet, who decided to learn Turkish by writing poems.
I heard about similar story from some old coworkers. I left the company because it always had a problem with good communication, devs being siloed, etc...
A "python dev" left the company. A different python dev was assigned to maintain his projects. They were surprised to find out all of ex-employee's "python" projects over the past 2 years were actually written in Rust. There was absolutely no code review in his department so no one was aware of this! No one else at the company knew Rust and they were debating rewriting it all, or hiring a Rust dev.
My old coworkers were basically putting all the blame on that one guy. And sure, that's shitty and unprofessional of him. But how could someone write Rust for 2 years and no one else notices?? I didn't understand how they could blame the guy and also not feel supremely embarrassed for even getting into that situation.
Hi, author here. Yeah, I thought the same and tried that too, but it failed due to the same reason mentioned in the "Problem with the definition using variants" section. Namely, the return value type of `read_arg` would not be uniquely determinable. We CAN define separate `read_arg8` for `read_arg16` for the `arg8` and `arg16` respectively, but I thought the GADT solution was more clean.
Great job! Really well written, easily one of the better technical posts I've seen on HN in a while. I loved messing around with the (homebrew?) ROMs on the online emulator, too!
It seems like the RosettaBoy project/repo is not well known at all. I've search information about gameboy emulators before and never saw it mentioned, the project has only one contributor that's active, a few language, not much stars or watchers. I'm surprised that you expected the author to know that project.
That's true, though the author's HN account wasn't created at this time (but he may have been reading it still), and since the post wasn't that popular it's fair to expect that the author didn't read it. You could contact him about it I think.
Really nice write up! It would have been nice to see more on how you handled the display, on my emulator journey the two big "i have no idea how to solve this" problems I had were 1) synchronising cpu/gpu/sound (and I settled on the exact method you use, 'catching up'), and 2) writing the framebuffer to the screen. The resources I've used always seem to skip over it, so I assume it's obvious, but it isn't to me.
Thanks! I ended up omitting the GPU since it didn't use much interesting OCaml features. But yes, the GPU was the most time consuming part for me too (both to understand and to implement), so I can relate to the "i have no idea how to solve this" feeling.
An emulator is still on my "bucket list as an engineer." I feel like it's a hurdle I still haven't crossed and I'm reasonably certain I could make one.
I have some free time this weekend...I might take this repo and port it over to clojure to get the hang of things.
Try JimTCL, it uses SDL2 for "GUI"'s instead of TK, but it's as compatible (at code level, not TCL extensions, Jimsh uses its own ones) as TCL 8.5 so most stuff will work as is.
If you look at the examples, you'll find that writing a Chip8 emulator with JimTCL it's a breeze.
I wrote (or tried writing) a gameboy emulator once.
My only qarning is that there are a lot of edge cases in the specification--and the proper specs are wrong.
I ended up spending most of the time searching through undocumented/missing CPU specifications. It became rather tedious.
For a learning project, this made it poorly suited. There are so many good emulators out there, the only reason to write one is for the learning experience. I just felt that I stopped oearning after a while and was just debugging obscure combinations of cpu states.
That seems to be the consensus that I've read online.
Still, the reason I want to do it online is specifically because I want an excuse to really understand low-level hardware. I (and probably a majority of people on HN who do software) really only have "real" experience with "Java or Higher" with maybe hitting C if I have the rare occasion of having to worry about saving a few milliseconds. An emulator would force me understand low level hardware, or at least I think it would.
Another fun option for learning low-level stuff is analyzing existing games. I did this to create a level map dumper for one of my favorite NES games[1]. You can see other ideas on the excellent Displaced Gamers YouTube channel, for example[2].
I think Nand 2 Tetris [1] is a really good way to get a good intro to this stuff at a basic level without too much friction with tools. You essentially start with a nand gate, and eventually build up a whole computer including a Java-like programming language.
That was more or less my experience. I got the original Game Boy working well, and quite a few Game Boy Color games, but it eventually devolved into debugging CPU timing issues and edge cases with sprite bugs. At some point I realized that I had gained all of the knowledge I had wished to learn and since my own emulator was never really going to do anything that hundreds of others weren't already doing better I moved on. I will say it was a good learning experience however. Chip-8 is a good start, but there are lots of concepts that emulating a Game Boy will require that Chip-8 does not.
I'd recommend maybe trying something simpler, like a Motorola 6811 or a MIPS processor. Their instruction sets are pretty simple and straightforward. But, I can see how that might not be as fulfilling.
My unpopular opinion is to emulate an extremely reduced subset of x86. For example, jump to main and emulate the result of `int main() {return 2;}` compiled with gcc. You already have the compiler (no need to cross compile), you already have the decompiler and you can run the binary on the reference machine without an emulator. You save so much time having this entire toolchain already and you can start emulating simple C programs for x86 with <10 instructions and 1 or 2 addressing modes. I've written about this process before in Node and Go [0].
Yeah everyone will comment after me that x86 gets way more complex over time but if you just want to get a quick intro and you aren't a perfectionist x86 is kind of compelling.
And recently I wanted to try emulating MSP341 at someone's recommendation because it has a very simple architecture. But just getting a C compiler for it isn't easy! When I've tried to cross-compile C to other archs like RISC-V it also wasn't straightforward and I just got stuck in this toolchain spot.
If it were easier to cross-compile simple languages like C then writing emulators for other environments would be easier.
If you're on a new Mac, my whole argument applies for aarch64 instead of x86.
> My unpopular opinion is to emulate an extremely reduced subset of x86
I don't think 808x is such an unpopular machine for approaching emulation.
Some people use Space invaders (or contemporary titles), which is based on an 8080, as stepping stone between CHIP-8 and more modern (complex) platforms.
I wish there was an easy way to compile to a subset of x86 (e.g. mem access only with simple mov, only using one flag, no extensions, no floats, pure 32bit instructions).
But emulating the CPU is probably the least interesting part about writing an emulator for a video game system. Writing the other hardware components and getting them to work together is the interesting part imo. I'd say skip writing your own CPU emulator and just start with someone else's.
As a backend dev, being able to emulate my own program compiled with GCC was pretty rewarding personally.
But I'm not arguing everyone must do it. Just trying to share why one unintuitive approach (the "complex" x86 architecture) might actually be easier and cooler than you think.
For a fun and fairly basic task, consider giving the 2019 advent of code a try. You will end up with a basic emulator for an imaginary machine, but you will get to solve interesting puzzles with it.
I've written somewhere around 1/3rd of a NES emulator. It does the CPU, and the backround half of the PPU, but not foreground sprites, sound, controllers, or running at decent speed ;)
Anyhow, I found the CPU part a lot of fun, and there's public domain test roms with expected registers after each instruction that you can use to test and confirm. Interaction with other hardware and emulation of that hardware gets a lot more challenging. The gameboy is certainly different than the NES, but I suspect there's a lot of similarity at the high level. It's certainly reasonable to stop (or pause, perhaps), when it stops being so fun, too.
I'm also not sure porting an emulator is going to give you the same understanding as writing one, but maybe.
I wrote a decently complete NES emulator[1], and the 6502 is definitely a fun CPU to emulate compared to other CPUs. The Z80-ish CPU in the Game Boy is not as fun, and as soon as you get into 68000 territory, it definitely stops being fun for a hobbyist.
Getting the graphics and sound in sync with the CPU is a lot more challenging if you're looking to support many games, but if you just want to play some of the super simple early NES games, like Donkey Kong, then you can get away with a lot of inaccuracies.
Start with a CHIP-8 emulator, it's a great first emulator to write that will expose you to a lot of core concepts without all the complexity that comes with more advanced systems.
It is definitely a fun learning exercise, but also quite addictive and grueling (yes, a contradiction)!
I wrote a Gameboy Color emulator[1] earlier this year, but stopped as soon as I realized that I wasn't learning anything new. As a result, my emulator has a few issues and does not support sound, but it is able to run quite a few games. I also got it running in a browser via WASM, which was fun.
If you think you can do it, you probably can. It's not really all that conceptually difficult until you want to do dynamic recompilation.
I wrote my first emulator a while ago, for the Gigatron. People recommend CHIP-8 a lot but I just thought it looked too trivial and uninteresting. The Gigatron is still incredibly simple, but it's also a really elegant little beast worth studying for its own sake.
Great article, thanks for writing that. A small question for the author: in the Addressable_intf.S signature, read_byte has regular arguments and write_byte named arguments. Is there a reason for that?
I added labels based on the number of arguments. `write_byte` has two arguments (other than `t`), the `addr` and `data`, so I used labeled arguments to distinguish the two easily. On the other hand, `read_byte` only has one argument, so I didn't label the argument. I saw this approach used in some Jane Street libraries: for example, `Hashtbl.set` takes two arguments and is labeled, while `Hashtbl.find` only has one argument and is not labeled.
But I was sometimes confused and wrongly provided labels to `read_byte`, so looking back at it now, maybe I should have just labeled both functions for consistency.
OCaml has recently interested me for being one of the few languages that:
- Compiles to a binary
- Achieves good speed
- Is functional, while accommodating imperative styles.
- Has a static type
In particular, it interests me because of how Forth or a threaded code-based VM could be implemented in it. You could implement the operations of such a machine with a bunch of individual function calls, but that's slow and rather unnecessary. A jump to the next VM instruction's code is what you want instead.
Switch statements or pointers to labels are what are often used, but that means you end up with hundreds or even thousands of lines of code which cannot be split into modules. The solution OCaml and other functional languages provide involves tail-call optimization:
let rec first x = print_endline "foo" ; second(x)
and second x = print_endline "bar" ; first(x);;
first 1;;
In a usual compiled language, these two functions would only be able to print 'foobar' a few hundred or thousand times before causing a stack overflow. In a language like OCaml this can run forever, and quickly too. This allows for what this web page calls continuation-passing threaded code:
I have mocked up the concept in lua, an imperative language with tail call optimization, and it was quite easy to implement a basic Forth while keeping the code legible. To take the idea further OCaml feels like a good choice. If anyone has experience with OCaml, or any warnings, it would be helpful.
73 comments
[ 4.6 ms ] story [ 139 ms ] threadInb4 piracy, thanks to CBR's I've got more physical mangas and comic books than ever.
I'd also like to add that it seems like a missed opportunity here. It could have been named OBoy.
Based on your description it might as well have been scrapped regardless of the programming language used?
[0] https://pages.cs.wisc.edu/~remzi/Naur.pdf
My point is that it was very tone deaf to introduce a new technology onto the stack for this reason (so called resume-based development I think?).
Sounds like the real issue is you wanted to write your own log tool.
It is like reading the first Turkish works of an English poet, who decided to learn Turkish by writing poems.
Yeah, that is the wrong reason to pick a new language to build an internal tool.
A "python dev" left the company. A different python dev was assigned to maintain his projects. They were surprised to find out all of ex-employee's "python" projects over the past 2 years were actually written in Rust. There was absolutely no code review in his department so no one was aware of this! No one else at the company knew Rust and they were debating rewriting it all, or hiring a Rust dev.
My old coworkers were basically putting all the blame on that one guy. And sure, that's shitty and unprofessional of him. But how could someone write Rust for 2 years and no one else notices?? I didn't understand how they could blame the guy and also not feel supremely embarrassed for even getting into that situation.
I have some free time this weekend...I might take this repo and port it over to clojure to get the hang of things.
http://devernay.free.fr/hacks/chip8/C8TECH10.HTM#2.1
If you look at the examples, you'll find that writing a Chip8 emulator with JimTCL it's a breeze.
https://github.com/msteveb/jimtcl
My only qarning is that there are a lot of edge cases in the specification--and the proper specs are wrong.
I ended up spending most of the time searching through undocumented/missing CPU specifications. It became rather tedious.
For a learning project, this made it poorly suited. There are so many good emulators out there, the only reason to write one is for the learning experience. I just felt that I stopped oearning after a while and was just debugging obscure combinations of cpu states.
Still, the reason I want to do it online is specifically because I want an excuse to really understand low-level hardware. I (and probably a majority of people on HN who do software) really only have "real" experience with "Java or Higher" with maybe hitting C if I have the rare occasion of having to worry about saving a few milliseconds. An emulator would force me understand low level hardware, or at least I think it would.
Also, read the book Code by Charles Petzold.
[1] https://tcrf.net/Proto:M.C._Kids#Level_Changes
[2] https://www.youtube.com/watch?v=zGdpst_6k9k
[1]: https://www.nand2tetris.org/
Yeah everyone will comment after me that x86 gets way more complex over time but if you just want to get a quick intro and you aren't a perfectionist x86 is kind of compelling.
And recently I wanted to try emulating MSP341 at someone's recommendation because it has a very simple architecture. But just getting a C compiler for it isn't easy! When I've tried to cross-compile C to other archs like RISC-V it also wasn't straightforward and I just got stuck in this toolchain spot.
If it were easier to cross-compile simple languages like C then writing emulators for other environments would be easier.
If you're on a new Mac, my whole argument applies for aarch64 instead of x86.
[0] https://notes.eatonphil.com/tags/emulators.html
https://notes.eatonphil.com/emulating-amd64-starting-with-el...
I don't think 808x is such an unpopular machine for approaching emulation.
Some people use Space invaders (or contemporary titles), which is based on an 8080, as stepping stone between CHIP-8 and more modern (complex) platforms.
But I'm not arguing everyone must do it. Just trying to share why one unintuitive approach (the "complex" x86 architecture) might actually be easier and cooler than you think.
https://multigesture.net/articles/how-to-write-an-emulator-c...
https://git.sr.ht/~rabbits/chip8uxn
Anyhow, I found the CPU part a lot of fun, and there's public domain test roms with expected registers after each instruction that you can use to test and confirm. Interaction with other hardware and emulation of that hardware gets a lot more challenging. The gameboy is certainly different than the NES, but I suspect there's a lot of similarity at the high level. It's certainly reasonable to stop (or pause, perhaps), when it stops being so fun, too.
I'm also not sure porting an emulator is going to give you the same understanding as writing one, but maybe.
Getting the graphics and sound in sync with the CPU is a lot more challenging if you're looking to support many games, but if you just want to play some of the super simple early NES games, like Donkey Kong, then you can get away with a lot of inaccuracies.
[1]: https://github.com/ltriant/nes
I wrote a Gameboy Color emulator[1] earlier this year, but stopped as soon as I realized that I wasn't learning anything new. As a result, my emulator has a few issues and does not support sound, but it is able to run quite a few games. I also got it running in a browser via WASM, which was fun.
[1]: https://github.com/aksiksi/gbc
I wrote my first emulator a while ago, for the Gigatron. People recommend CHIP-8 a lot but I just thought it looked too trivial and uninteresting. The Gigatron is still incredibly simple, but it's also a really elegant little beast worth studying for its own sake.
I added labels based on the number of arguments. `write_byte` has two arguments (other than `t`), the `addr` and `data`, so I used labeled arguments to distinguish the two easily. On the other hand, `read_byte` only has one argument, so I didn't label the argument. I saw this approach used in some Jane Street libraries: for example, `Hashtbl.set` takes two arguments and is labeled, while `Hashtbl.find` only has one argument and is not labeled.
But I was sometimes confused and wrongly provided labels to `read_byte`, so looking back at it now, maybe I should have just labeled both functions for consistency.
I am but a fool
- Compiles to a binary - Achieves good speed - Is functional, while accommodating imperative styles. - Has a static type
In particular, it interests me because of how Forth or a threaded code-based VM could be implemented in it. You could implement the operations of such a machine with a bunch of individual function calls, but that's slow and rather unnecessary. A jump to the next VM instruction's code is what you want instead.
Switch statements or pointers to labels are what are often used, but that means you end up with hundreds or even thousands of lines of code which cannot be split into modules. The solution OCaml and other functional languages provide involves tail-call optimization:
In a usual compiled language, these two functions would only be able to print 'foobar' a few hundred or thousand times before causing a stack overflow. In a language like OCaml this can run forever, and quickly too. This allows for what this web page calls continuation-passing threaded code:https://www.complang.tuwien.ac.at/forth/threaded-code.html
I have mocked up the concept in lua, an imperative language with tail call optimization, and it was quite easy to implement a basic Forth while keeping the code legible. To take the idea further OCaml feels like a good choice. If anyone has experience with OCaml, or any warnings, it would be helpful.