This is cool but seems to be "just" the output from IDA with the functions renamed in the output? If that's the case I'm wondering why we don't see this sort of thing more often (other than the effort required of course)
>It's not an ordinary decompilation generated by IDA. They actually rewrote all the functions from reading MIPS assembly and compiled it with the original compiler, adjusting the code until it produced identical output to a vanilla ROM.
Reverse engineering a reproducible build is quite a bit more than "just" output.
A perfect decompiler would just do it, but writing the exact inverse of a compilation tool chain would be... Difficult. You would have to have a different decompiler for each version each tiny little difference.
These guys were capable of figuring out and naming what every function did and then rewriting each of them over and over and over to get the original tool chain to output bit for bit the exact same binary as they started with.
Something like taking x-rays of an unknown machine and being able to recreate perfect pixel replicas of the engineering drawings or listening to a piece of music and being able to exactly write out the score.
I'd recommend watching this CppCon talk [1] about compiler optioning. It isn't just that recompilation is hard, or that it is difficult for humans to do. At a fundamental level, the information is not there at all, because the compiler can make very impressive optimizations. Machine learning can deal with weak signals, but it can't deal with no signal.
They mention in the thread that the binary seemed to have been compiled without optimisation, which means it's a lot easier to translate back to source than most games would be.
There has been similar things done in the past, for example for Dune Legacy (Dune II) or REminiscence (Flashback)
Mario 64 is more recent so might be a magnitude more complex.
There are also other approaches, like a total re-implementation, not a decompile/re-source. ScummVM engine for Blade Runner, the adventure game from 90's is a recent very impressive example of that
I mean, that's just Nintendo being Nintendo. You know they will freak out over the silliest stuff when it comes to their Mario franchise, it's just a given. Maybe these folks should have decompiled some "orphaned" code/game where we literally have no idea who the copyright owner even is, then at least it would bring the stuff back from obscurity. But Mario 64 is just 'meh'.
What are you talking about? Super Mario 64 is an extremely loved game and is still played. People will find much more interest in this than some obscure game that nobody cares about.
People are passionate about this game and there is a good chance of a PC port of the game now (as well as more substantial mods) because of that.
The game modding community never ceases to amaze me. I know of a few communities that are Reverse Engineering shockwave just to be able to retrieve original game code, and in some cases create private servers for defunct games.
Also theres a cool project archiving all Flash and Shockwave games called BlueMaximas Flashpoint. The people working on preserving these old games are amazing.
I do fear the death of some of those awesome NewGrounds games.
Anyone know how much manual work was needed to do this? Did they just use a tool to generate the C code? The functions unhelpfully have names like func_802507AC.
It's in the third post or so. Apparently Nintendo didn't turn on any compiler optimizations so porting the mips assembly back to c and fiddling around a bit until the original compiler produced identical instructions again was a feasible thing to do.
That makes me wonder how much better we can make it perform by turning on optimizations (and likely fixing undefined behavior in places). There's quite a few locations in the game where the framerate and gameplay suffer.
It was one of the first games and easily outclassed by later releases so I'm sure there are many optimizations that could be made.
I hope we get the same with Waverace one day, nothing since has matched the feel of the wave physics in that one, it would be interesting to see what they did.
Some people need everything explained don't they? "What do you mean first game? I distinctly remember playing tag at school and that preceded Wave Race 64 by at least 2 years. Please stop spreading FUD".
My unfounded suspicion is that optimizations might have been turned off to avoid bugs in the toolchain. The developers would have had to deal with very early versions of these tools before the comsole was launched and in development itself.
its cool and all, but i've seen this before with other games and what happens is the people that want to do game development and people that want muck around with old decompiled source code are two different camps of people so I wouldn't expect much
The result is often a classic game running on a modern system without needing to jump through hoops, which is a huge win to people who enjoy playing the game in question.
OpenTTD, OpenRCT2, and OpenMW all come to mind, not to mention the many ports of Id games.
That should be feasible even with emulation. My memory seems to be a bit faulty, but IIRC UltraHLE could run SM64 at full speed on systems with less than 1GB and 1Ghz.
I can't remember if it was full speed but it was perfectly playable to emulate SM64 with UltraHLE on my Pentium 200MMX OC to 225MHz with a Voodoo 2 8MB and 96MB of RAM.
I had to run it in Windows NT5 beta though, it stuttered in Windows 95.
They are reimplementations but sadly far from clean room. The ASM was often referenced and even used side by side as the projects migrated to their own code base.
Yeah, the end result of this isn't even opensource. Nintendo is still the owner of the code that was decompiled. This is why the people working on the project didn't want it be leaked in the first place, because nintendo will try to shutdown the project before its done.
It should be also noted that the inner working of Super Mario 64 is already extremely well known due to numerous TAS attempts, so I found this manual decompilation hardly surprising. A recent video [1] detailing the "one-key" TAS [2] is mind-blowing.
It's also worth pointing to Kaze's videos for what dedication can bring even without the original source code. You'll find among other things recreations of later Mario games, or even games like Portal (and mechanics!) implemented in SM64. https://youtube.com/user/KazeBG0/videos
I went to college with him, and I genuinely had no idea that it was him until several years afterwards. I definitely wish I had realized back when I had the chance to pick his brain about this sort of thing!
His video on the Rolling Rocks 0.5 A press clear is one of the most informative and entertaining technical videos I've ever watched.
I had never heard of the guy before watching that video, but I gleaned from the comments that there was something about it turning into a meme, and turning the guy off from making more commentated vidoes? If so, that's a real shame, because I could watch content from him all day long.
In this leak, there's still a fair bit of decomp work to be done on the audio systems.
Otherwise, you are right: the majority of the work left is to document the functions, to organize the code so that it's easy to mod and play around with, and to provide high level documentation on the systems/how the decomp was made. You wouldn't want anyone to think that all you have to do to decomp something is to run a binary through IDA, right?
Plus, there are more released versions of SM64, and who knows what's in there?
The build is reproducible (provides the same binary output as the original), so the remaining work is to rename the function to make it easier to understand.
Interestingly, there exists other projects that aim to rebuild exact original ROM's. For example, Pokémon Ruby and Pokémon Emerald have been disassembled[1][2] and then slowly converted to C making sure that the hash of the ROM always stays the same.
Well given a set of known, used optimization techniques the backward operation would be possible to try. If it matches the target binary, would be one step closer to something considered source reconstruction.
95 comments
[ 4.6 ms ] story [ 264 ms ] threadReverse engineering a reproducible build is quite a bit more than "just" output.
These guys were capable of figuring out and naming what every function did and then rewriting each of them over and over and over to get the original tool chain to output bit for bit the exact same binary as they started with.
Something like taking x-rays of an unknown machine and being able to recreate perfect pixel replicas of the engineering drawings or listening to a piece of music and being able to exactly write out the score.
[1] https://www.youtube.com/watch?v=nLv_INgaLq8
Mario 64 is more recent so might be a magnitude more complex.
There are also other approaches, like a total re-implementation, not a decompile/re-source. ScummVM engine for Blade Runner, the adventure game from 90's is a recent very impressive example of that
[1] https://github.com/scummvm/scummvm/tree/master/engines
[2] https://wiki.scummvm.org/index.php?title=HOWTO-Reverse_Engin...
People are passionate about this game and there is a good chance of a PC port of the game now (as well as more substantial mods) because of that.
There's no way we're letting this slip. If anybody can't find a copy PM me and I'll help you track one down.
I imagine it'll get taken down eventually, but I don't see a problem with letting as many people get their eyes on it in the meantime.
In situations like this I highly recommend anyone with a spare 15.2MB of disk space to download this just to help preserve it.
https://github.com/matt-kempster/mips_to_c
Also theres a cool project archiving all Flash and Shockwave games called BlueMaximas Flashpoint. The people working on preserving these old games are amazing.
I do fear the death of some of those awesome NewGrounds games.
I hope we get the same with Waverace one day, nothing since has matched the feel of the wave physics in that one, it would be interesting to see what they did.
What on earth do you mean?
https://en.m.wikipedia.org/wiki/Chronology_of_Nintendo_64_ga...
I vaguely remember it was a launch title.
I try to assume that such things are genuine questions unless facetiousness is obvious.
Makes for a more pleasant place that way.
OpenTTD, OpenRCT2, and OpenMW all come to mind, not to mention the many ports of Id games.
I also wonder if you could add stuff like raytracing given the original source. (Obv not on the Pi)
You guys remember mario.com on MS-DOS? A full clone of Super Mario running fullspeed on DOS?
Can you guys imagine mario64.exe? ^^
I had to run it in Windows NT5 beta though, it stuttered in Windows 95.
Technically, yes, you could use a decompiled game code to port the game to another platform, but the legalities are not in your favor.
[1] https://www.youtube.com/watch?v=wjge1bVobN0
[2] http://tasvideos.org/3264M.html
His explanations are mind-blowing as well and fun to watch :)
His wiki: https://mario64hacks.fandom.com/wiki/Kaze_Emanuar
This is a good way to turn kids into hackers!
I had never heard of the guy before watching that video, but I gleaned from the comments that there was something about it turning into a meme, and turning the guy off from making more commentated vidoes? If so, that's a real shame, because I could watch content from him all day long.
I think by then it would have already been pretty mature.
Otherwise, you are right: the majority of the work left is to document the functions, to organize the code so that it's easy to mod and play around with, and to provide high level documentation on the systems/how the decomp was made. You wouldn't want anyone to think that all you have to do to decomp something is to run a binary through IDA, right?
Plus, there are more released versions of SM64, and who knows what's in there?
[1] https://github.com/pret/pokeruby
[2] https://github.com/pret/pokeemerald
You just need one :)
Can't forget that trailing slash :)