I believe this is more commonly known as "overlapping instructions/opcodes"; and it was a relatively common technique for size optimisation or obfuscation in the 80s and early 90s, when Asm was the norm for PC software.
To my knowledge, there are no tools which do a good job of presenting the parallel execution streams. Instead, the typical flow requires the human to iterate through attempting disassembly at different offsets.
I remember seeing one (tracing) disassembler which does this, presenting the different streams side-by-side until they realign again. No doubt cracking groups probably have their own private versions of such tools.
Yes, indeed, this technique was frequently used in ancient programs.
For example, many decades ago I have disassembled many Microsoft programs, like the MS-DOS Basic interpreter or Fortran compiler.
In all those Microsoft programs, there were many loops where the jump at the end of the loop jumped in the middle of the initial instruction. Thus the first execution of the loop executed different instructions than the next executions, which was used, e.g., for the loop initialization.
There were also many if-then-else structures where one of the branches ended with a jump towards the middle of some instruction, so that depending on which was the value of the condition different instructions were executed.
The instructions whose middle position was the target of a jump were usually instructions with immediate operands, so that when executed from the beginning the immediate value was treated as data, while when reached through a jump in the middle of the instruction, the immediate data was treated as an instruction opcode.
The term unintended instructions is common in the infosec community and was (as far as I know) popularized by the seminal paper on ROP https://hovav.net/ucsd/dist/geometry.pdf
I'm sure different terms are used in other communities.
I wonder which unintended instruction streams decode to the highest number of decodable partial instructions/instruction sequences, and then I wonder how useful those sequences might be in, say, a 4k or 1k demo....
Meanwhile, people are clinging to decade-old Thinkpads like they were made of gold.
Most laptops and pretty much all desktops are x86-64, as are both major home consoles.
Sure, ARM is dominant on mobile, and it's definitely going to grab a share of the laptop market now that Apple is going that route.
Eventually we might also see ARM based desktop computers again, but having them grab the majority of sales will take time. I'm sure it'll happen - just not within a decade.
if for some reason x86-64 will become incompatible with novel highly advantageous architectural decisions of competitors, AMD and Intel will ditch it, and make cpus with different/new architectures
Certainly for Intel I think that it's unlikely that they will switch and maintain anything like their market position in CPUs.
- There is too much attachment to x86 and too much pain associated with historical attempts to move to other architectures - obviously Itanium being the most recent.
- Moving from x86 means losing the architecture duopoly.
- By the time x86 is clearly failing in the market competitors will have gained a major foothold in the market.
- The lead in time to switch is too long to reverse the pattern of decline.
I strongly suspect that it's already too late given the pattern we're seeing with M1 and AWS and Graviton.
Most RISC and RISC-derived instruction sets are this way. Power ISA also faults if the PC isn't aligned to 32 bits. ARM Thumb encoding makes this a hair more complex, but certainly less than x86.
You know, I've spent the past two weeks tinkering with a toy x86_64 bootloader/kernel, and I think that x86 as an ISA is okay. Not great, but okay. Yes, 16-bit addressing modes are restrictive, the encoding hack that enables addressing of r8-r15 in the long mode is probably against one of Geneva conventions, the constant proliferation of control registers is vexing, and CPUID will one day probably be able to dump a full 10-volume PDF of Software Developer's Manual into the provided memory... but still, it's kinda okay.
But what's absolutely terrible is the interface between the CPU and everything else attached to the mainboard (and with the mainboard itself, for that matter). The interrupt handling somehow gets worse when moving from PIC to APIC, ACPI is nauseating, I/O ports are insanely opaque, and the default memory-mapped devices are organized in such a way that there are irreparable wholes in the physical address space that you can't use anyway. Having physical RAM start, say, at address 0x800000000000 and everything below it be MMIO would be pretty convenient, but it can't be arranged for legacy reasons. The legacy boot sequence is appalling. UEFI is marginally better, but is itself an impressive bag of worms.
Holy fatcats. I hope RISC-V will have saner peripherals, but kinda afraid that it won't either.
Yep. It's le bon ton to frown at the x86's baroqueness, but other ISAs had their share of bizarre ideas as well: e.g. MIPS had no interlocks until MIPS-II, so every instruction had a delay slot.
I personally liked the SIB byte from x86-32, it's a neat way to repurpose an ACU into basically another ALU.
>"the encoding hack that enables addressing of r8-r15 in the long mode is probably against one of Geneva conventions ..."
Might you or anyone else have any links or literature on what this hack is or how it works?
>"the constant proliferation of control registers is vexing ..."
I'm guessing this isn't referring to control registers proliferating on different of generations of CPUs at the discussion is about the ISA. Could you elaborate on how exactly they've proliferated?
> Might you or anyone else have any links or literature on what this hack is or how it works?
Intel 64 and IA-32 Architectures Software Developer's Manual, volume 2, chapter 2.2.1 "REX Prefixes". I understand that there was literally no place in either opcode or ModR/M byte to squeeze in another bit, but still.
As for control registers: we now have 9 "control registers", CR0 through CR8, and 4 of them are still reserved which kinda makes this whole "reserved" business seem pointless. And then, of course, there are MSRs, loads and loads of MSRs. In fact, there are so many of them that we now have a whole separate volume 4 of Intel SDM that is dedicated exclusively to them and is about 300 pages long. No doubt it will be 400 pages in the near future. Honestly, writing about this stuff feels like quoting from the "See Figure 1" doc.
Spoiler: It probably won't. ARM-based systems are even more of a mess than x86 in this regard, as they haven't been standardized up until the last few years when UEFI/SBSA started to be adopted. x86, while shit, at least followed approximate standards and systems were broadly compatible.
I wish the world had standardized on OpenFirmware, it might have at least let us avoid the mess that it ACPI.
27 comments
[ 1.9 ms ] story [ 67.9 ms ] threadTo my knowledge, there are no tools which do a good job of presenting the parallel execution streams. Instead, the typical flow requires the human to iterate through attempting disassembly at different offsets.
I remember seeing one (tracing) disassembler which does this, presenting the different streams side-by-side until they realign again. No doubt cracking groups probably have their own private versions of such tools.
For example, many decades ago I have disassembled many Microsoft programs, like the MS-DOS Basic interpreter or Fortran compiler.
In all those Microsoft programs, there were many loops where the jump at the end of the loop jumped in the middle of the initial instruction. Thus the first execution of the loop executed different instructions than the next executions, which was used, e.g., for the loop initialization.
There were also many if-then-else structures where one of the branches ended with a jump towards the middle of some instruction, so that depending on which was the value of the condition different instructions were executed.
The instructions whose middle position was the target of a jump were usually instructions with immediate operands, so that when executed from the beginning the immediate value was treated as data, while when reached through a jump in the middle of the instruction, the immediate data was treated as an instruction opcode.
I'm sure different terms are used in other communities.
https://paws.kettering.edu/~jhuggins/humor/opcodes.html
https://vyznev.net/corewar/guide.html#start_instr
ARM chips last longer on battery, and every high volume laptop manufacturer had an ARM model. Every smartphone is already arm.
ARM already has mainline support from every major OS vendor.
Every videogame engine builds for ARM targets.
In AWS, Graviton2 instances are 40% cheaper for cpu bound workloads vs x86. Next time our reserved instances expire we will switch.
Most laptops and pretty much all desktops are x86-64, as are both major home consoles.
Sure, ARM is dominant on mobile, and it's definitely going to grab a share of the laptop market now that Apple is going that route.
Eventually we might also see ARM based desktop computers again, but having them grab the majority of sales will take time. I'm sure it'll happen - just not within a decade.
- There is too much attachment to x86 and too much pain associated with historical attempts to move to other architectures - obviously Itanium being the most recent.
- Moving from x86 means losing the architecture duopoly.
- By the time x86 is clearly failing in the market competitors will have gained a major foothold in the market.
- The lead in time to switch is too long to reverse the pattern of decline.
I strongly suspect that it's already too late given the pattern we're seeing with M1 and AWS and Graviton.
But what's absolutely terrible is the interface between the CPU and everything else attached to the mainboard (and with the mainboard itself, for that matter). The interrupt handling somehow gets worse when moving from PIC to APIC, ACPI is nauseating, I/O ports are insanely opaque, and the default memory-mapped devices are organized in such a way that there are irreparable wholes in the physical address space that you can't use anyway. Having physical RAM start, say, at address 0x800000000000 and everything below it be MMIO would be pretty convenient, but it can't be arranged for legacy reasons. The legacy boot sequence is appalling. UEFI is marginally better, but is itself an impressive bag of worms.
Holy fatcats. I hope RISC-V will have saner peripherals, but kinda afraid that it won't either.
You may want to look at the AArch64 "system registers" space... or the instruction encoding, for that matter.
I think 32-bit x86 made the most sense and was well thought-out, but the 64-bit version was definitely not extended very well.
I personally liked the SIB byte from x86-32, it's a neat way to repurpose an ACU into basically another ALU.
Might you or anyone else have any links or literature on what this hack is or how it works?
>"the constant proliferation of control registers is vexing ..."
I'm guessing this isn't referring to control registers proliferating on different of generations of CPUs at the discussion is about the ISA. Could you elaborate on how exactly they've proliferated?
Intel 64 and IA-32 Architectures Software Developer's Manual, volume 2, chapter 2.2.1 "REX Prefixes". I understand that there was literally no place in either opcode or ModR/M byte to squeeze in another bit, but still.
As for control registers: we now have 9 "control registers", CR0 through CR8, and 4 of them are still reserved which kinda makes this whole "reserved" business seem pointless. And then, of course, there are MSRs, loads and loads of MSRs. In fact, there are so many of them that we now have a whole separate volume 4 of Intel SDM that is dedicated exclusively to them and is about 300 pages long. No doubt it will be 400 pages in the near future. Honestly, writing about this stuff feels like quoting from the "See Figure 1" doc.
Spoiler: It probably won't. ARM-based systems are even more of a mess than x86 in this regard, as they haven't been standardized up until the last few years when UEFI/SBSA started to be adopted. x86, while shit, at least followed approximate standards and systems were broadly compatible.
I wish the world had standardized on OpenFirmware, it might have at least let us avoid the mess that it ACPI.