What Five Architectures Revealed

This chapter reads the InSPECtor evaluation as evidence about the correctness of SLEIGH processor specifications, not as a leaderboard of instruction sets. The focus is how to move from five unlike test targets to a defensible interpretation of coverage, discrepancies, and root causes. The useful judgment is to keep the denominators separate: a generated case, an observed difference, a constructor group, and a confirmed specification bug are different objects.

Testing one architecture could show that the method works somewhere. Testing x86-64, AArch64, ARM/Thumb, RISC-V, and MSP430 asks a harder question: can the same structural traversal and state-aware comparison handle different encoding designs, specification styles, modes, operand relationships, and semantic conventions? The results show broad reach, but they also show that the most consequential failures are often in the semantic model rather than in the visible opcode pattern.

Five specifications, several denominators

The five targets were chosen for diversity. x86-64 has variable-length encodings and complex prefix behavior. AArch64 and ARM/Thumb use different forms of mode and operand structure. RISC-V includes compressed and extension-related behavior. MSP430 makes extensive use of addressing modes and a different scale of processor. The point of this set is not that these architectures represent every possible ISA. It is that a generator that depends on one author’s style or one encoding convention would not be persuasive as a general SLEIGH validation method.

The first measurement is generation reach. In the paper’s Table 2, GenSYS produced 71,852 instruction cases for x86-64, 25,031 for AArch64, 30,709 for ARM/Thumb, 9,725 for RISC-V, and 7,436 for MSP430. It also generated state variants—712,668, 123,031, 207,999, 38,481, and 124,712 respectively. These figures combine constructor-path choices with representative encodings and edge-case states; they are not counts of all possible byte strings or all possible machine states.

The corresponding constructor-coverage figures were 97% for x86-64, 97% for AArch64, 96% for ARM/Thumb, 99% for RISC-V, and 90% for MSP430. A reader could be tempted to treat the last number as evidence that MSP430’s specification is “worse” than RISC-V’s. That conclusion does not follow. Coverage is a proxy for how much of each specification’s constructor structure the configured generator reached. Different specifications contain different numbers of constructors, modes, extensions, recursive forms, and unavailable features, and the generation costs also vary. The metric is useful for locating unexplored structure, not for ranking architectures or authors.

The authors investigated 460 constructors that received no generated test case. The reasons mattered. Some were not applicable because the tested configuration did not enable the relevant processor mode or extension. Some were redundant because more specific constructors completely covered the same encodings. Some belonged to incomplete parts of a specification, such as a subtable with no reachable parent. A small group represented specification issues. A single percentage would hide these distinctions; the missing-path investigation restores them.

The second measurement is differential output. Across the five architectures, the paper’s detailed evaluation reports 589,713 raw discrepancies between the SLEIGH-based emulator and hardware.