[551] Evidence gap: fixed and RDW record pipeline

[EffortlessSteven]

Parent

#551

Verified evidence status (current main)

• Fixed and RDW evidence is split across planes:
  • crates/copybook-codec/tests/rdw_comprehensive.rs: test_rdw_length_recomputation_on_encode, test_rdw_multiple_records, test_rdw_raw_record_only_mode.
  • crates/copybook-codec/tests/comprehensive_rdw_tests.rs: test_rdw_basic_parsing, test_rdw_length_recomputation, test_rdw_with_odo_variable_length, test_rdw_encoding_round_trip.
  • crates/copybook-codec/tests/streaming_file_tests.rs: decode_single_thread_deterministic, decode_thread_count_in_summary, decode_file_partial_record_at_eof.
  • crates/copybook-codec/tests/rdw_microcrate_integration.rs: rdw_decode_rejects_ascii_corrupted_header, rdw_reserved_nonzero_increments_run_summary_warnings.
• Missing evidence-row coverage exists for fixed-vs-RDW framing/rejection, raw-mode byte-fidelity, and worker-path behavior together in one plane-indexed artifact.

Gap type

• Missing evidence plane mapping, with one pending candidate to create by format.fixed.framing.rejection matrix. Not yet confirmed as a defect.

Required split candidates (post-inventory)

• format.fixed.framing.rejection (parse/decode/encode/round-trip)
• format.rdw.header.rejection (raw + strict/lenient)
• format.fixed_rdw.raw_mode (CLI + API evidence)

[EffortlessSteven]

Evidence-gap lane for #551 is now covered in PR scope on branch chore/test-raw-threaded-fidelity (commits a15ed16, be03ad2).

Added streaming-file plane artifacts:

• raw_fidelity_threaded_fixed_and_rdw (fixed+RDW byte fidelity + thread invariance + raw modes)
• decode_rdw_reserved_warning_threaded_raw_mode
• decode_fixed_threaded_rejection_missing_lrecl (fixed framing rejection + missing fixed LRECL)
• decode_rdw_threaded_header_ascii_corruption_is_fatal (RDW header rejection + exact error code)
• decode_rdw_threaded_reserved_warning_is_deterministic (thread-invariant warning semantics)

These tests now provide the format.fixed.framing.rejection and format.rdw.header.rejection planes with thread matrix + deterministic behavior. Closing as evidence now complete for this issue.

[EffortlessSteven]

Recovery lane is now split into explicit PRs again. #584 opened for fixed/RDW raw-fidelity evidence from �15ed16+�e03ad2, scoped to crates/copybook-codec/tests/streaming_file_tests.rs with pass/fail proof in CI command set. main issue state is intentionally kept open until #584 merges and current-main evidence is green.

[EffortlessSteven]

Progress update after #584 updates: completed evidence rows now covered by #584

• ormat.fixed.raw_fidelity.payload

  • decode_file_to_jsonl_raw_mode_records_emit_fixed_raw_payload
  • asserts
    ecords_processed=2, �ytes_processed=10, canonical
    aw_b64 payload bytes.

• ormat.rdw.raw_fidelity.header_plus_payload

  • decode_file_to_jsonl_raw_mode_rdw_emits_header_and_payload
  • asserts
    ecords_processed=2, �ytes_processed=18, canonical raw payload includes 4-byte header + payload.

• ormat.rdw.raw_mode.reserved.strict

  • decode_file_to_jsonl_raw_mode_rdw_strict_reserved_rejects_nonzero
  • rejects with CBKR211.

• ormat.rdw.raw_mode.reserved.lenient

  • decode_rdw_reserved_warning_is_a_lenient_warning
  • same bytes preserved and warning recorded.

• ormat.rdw.header.suspect_ascii

  • decode_rdw_suspect_ascii_header_is_fatal
  • rejects with CBKF104.

• ormat.fixed.missing_lrecl

  • decode_file_to_jsonl_raw_mode_missing_lrecl_fixed_invalid_state
  • rejects with CBKI001.

Open in #572 remains: true worker-path matrix and any ODO/RDW/redefinition framing claims that are not yet demonstrated on current main.

[EffortlessSteven]

Progress update after #584 rebase on current main:

#584 now closes these evidence rows (on branch chore/recover-raw-rdw-evidence-main):

• format.fixed.raw_fidelity.payload
• format.rdw.raw_fidelity.header_plus_payload
• format.rdw.raw_mode.reserved.strict (CBKR211)
• format.rdw.raw_mode.reserved.lenient (warning path)
• format.rdw.header.suspect_ascii
• format.fixed.missing_lrecl

Keep #572 open for remaining gaps:

• true worker-path determinism/evidence
• any RDW+ODO framing behaviors not yet proven on current main
• any future dialect-level or pipeline coverage explicitly missing after [551] Evidence gap: fixed and RDW record pipeline #572 scope is re-audited on current main

Validation run used:

• cargo fmt -p copybook-codec -- --check
• cargo clippy -p copybook-codec --test streaming_file_tests -- -D warnings
• cargo test -p copybook-codec --test streaming_file_tests
• cargo test -p copybook-codec --features comprehensive-tests --test comprehensive_rdw_tests
• git diff --check

[EffortlessSteven]

Update after merge: PR #584 merged as 1cf4bea and rebased on current main.

Closed rows now proven on merged main:

• fixed raw fidelity payload: decode_file_to_jsonl_raw_mode_records_emit_fixed_raw_payload
• RDW header+payload raw fidelity: decode_file_to_jsonl_raw_mode_rdw_emits_header_and_payload
• strict RDW reserved-byte rejection: decode_file_to_jsonl_raw_mode_rdw_strict_reserved_rejects_nonzero
• lenient RDW reserved warning: decode_rdw_reserved_warning_is_a_lenient_warning
• suspect ASCII RDW header rejection: decode_rdw_suspect_ascii_header_is_fatal
• missing fixed lrecl invalid state: decode_fixed_missing_lrecl_fixed_invalid_state
• raw output key policy (
  aw_b64 canonical + __raw_b64 compatibility row)

Commands (on merged main):

• cargo fmt -p copybook-codec -- --check
• cargo clippy -p copybook-codec --test streaming_file_tests -- -D warnings
• cargo test -p copybook-codec --test streaming_file_tests
• cargo test -p copybook-codec --features comprehensive-tests --test comprehensive_rdw_tests

Known exclusions still open:

• worker-path determinism evidence (see [551] Evidence gap: determinism across worker configurations #578)
• RDW+ODO/ODO framing gaps from comprehensive_rdw_tests ( est_rdw_length_recomputation, est_rdw_with_odo_variable_length)