Skip to content

Add differential decode test harness - #636

Draft
flobernd wants to merge 3 commits into
masterfrom
differential-test-harness
Draft

Add differential decode test harness#636
flobernd wants to merge 3 commits into
masterfrom
differential-test-harness

Conversation

@flobernd

Copy link
Copy Markdown
Member

Summary

Adds a self-contained differential decode harness for catching behavioral
regressions in the decoder. ZydisDiffDump decodes an input file at every
byte offset across a fixed set of decoder configurations and emits per-block
FNV-1a hashes of the full decode result (status, instruction struct, operands,
and the pointed-to CPU/FPU flag records). Two builds of the tool over the same
input are guaranteed to produce byte-identical output, so any semantic drift
between a baseline and a candidate build shows up as a hash mismatch.

Intention

Decoder refactors (e.g. the performance work) need a cheap, high-coverage way
to prove that a change is behavior-preserving. Exhaustive per-offset decoding
over a large corpus exercises far more of the decoder than the unit tests, and
reducing each decode to a hash makes a full corpus comparison a single diff.

Changes

  • tools/ZydisDiffDump.c - new tool with three subcommands:
    • gen <file> <MiB> - write a deterministic xorshift64 corpus so both builds
      hash the exact same bytes.
    • dump <file> [start end] - emit per-block hashes; the optional range prints
      per-offset detail lines to bisect a mismatch down to the offending offset.
    • bench <file> - simple decode throughput benchmark (instr-only and full).
  • CMakeLists.txt - builds ZydisDiffDump when ZYDIS_BUILD_TOOLS and
    ZYDIS_FEATURE_DECODER are enabled.
  • tests/differential/diffcheck.sh - gate script that diffs the current
    build's dump against reference dumps captured from a baseline build (kept
    untracked in .diffref/) and reports the first differing lines on mismatch.

Testing

  • cmake -B build -DZYDIS_BUILD_TOOLS=ON -DCMAKE_BUILD_TYPE=Release +
    cmake --build build --target ZydisDiffDump - builds clean.
  • ZydisDiffDump gen corpus.bin 1 then ZydisDiffDump dump corpus.bin - runs
    across all five configs; two successive dumps of the same corpus are
    byte-identical (determinism confirmed via diff).

Notes

  • The pointer members cpu_flags / fpu_flags reference static tables whose
    addresses differ between binaries, so their contents are hashed and the
    pointers nulled before hashing the instruction struct - otherwise identical
    decodes would hash differently across builds.
  • Reference dumps live untracked in .diffref/; the harness compares builds,
    it does not ship a golden baseline.
  • Opened as a draft for review of the approach.

flobernd added 3 commits July 18, 2026 22:20
clock_gettime/CLOCK_MONOTONIC are POSIX and unavailable under MSVC and
ClangCL, which failed the Windows CI jobs. Standard C clock() is portable
and, for this single-threaded decode microbenchmark, tracks wall time
closely enough.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant