Skip to content

refactor(release): separate binary preparation and attachment - #11106

Merged
mergify[bot] merged 9 commits into
mainfrom
refactor/release-binary-preparation
Aug 4, 2026
Merged

refactor(release): separate binary preparation and attachment#11106
mergify[bot] merged 9 commits into
mainfrom
refactor/release-binary-preparation

Conversation

@gustavovalverde

@gustavovalverde gustavovalverde commented Jul 27, 2026

Copy link
Copy Markdown
Member

Motivation

Release binaries cannot be prepared before the GitHub Release exists because the current reusable workflow combines building, signing, and release mutation. This keeps binary builds serialized behind crate publication even though preparation does not depend on the published release.

Part of #11092.

Solution

Split the existing reusable binary workflow at the external mutation boundary:

  • zfnd-release-binaries.yml requires an immutable source commit, version, and feature set, then builds, attests, signs, verifies, and uploads one release-binaries workflow artifact.
  • zfnd-attach-release-binaries.yml downloads that artifact and owns the only GitHub Release mutation.
  • release-binaries.yml preserves the current release: released behavior by composing preparation followed by attachment.
  • Preparation records the checked-out source_sha as an explicit resolved dependency in GitHub's SLSA provenance and verifies that claim before bundling. The source remains accurate when a later manual recovery run uses a different workflow event commit.

The existing build runners, glibc compatibility ceiling, asset names, checksums, provenance, signatures, and rerun behavior remain unchanged.

Tests

  • actionlint passed for all changed workflows with ShellCheck enabled.
  • zizmor passed for the changed workflows with no findings.
  • Repository Markdown lint passed for the workflow documentation.
  • git diff --check passed.
  • The current six-asset release contract was compared with the latest release.
  • The generated SLSA predicate was exercised with distinct workflow and release-source commits and retained the release source dependency.

The reusable-workflow handoff and OIDC signing path still require a hosted Actions run because they depend on GitHub-hosted runners and artifact attestations.

Specifications & References

Follow-up Work

The final release cutover will invoke binary preparation in parallel with crate publication and attach the prepared artifact after the GitHub Release exists.

AI Disclosure

  • No AI tools were used in this PR
  • AI tools were used: OpenAI Codex implemented and reviewed the workflow changes.

PR Checklist

  • The PR title follows conventional commits format: type(scope): description
  • The PR follows the contribution guidelines.
  • This change was discussed in an issue or with the team beforehand.
  • The solution is tested.
  • The documentation and changelogs are up to date.

@gustavovalverde gustavovalverde added A-devops Area: Pipelines, CI/CD and Dockerfiles C-enhancement Category: This is an improvement labels Jul 27, 2026
@gustavovalverde
gustavovalverde requested a review from Copilot July 27, 2026 19:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Separates release-binary preparation from GitHub Release mutation, enabling earlier parallel builds.

Changes:

  • Adds reusable preparation and attachment workflows.
  • Preserves the existing release-triggered orchestration.
  • Updates workflow documentation.

Risk: Build provenance may identify the caller SHA rather than the explicit source SHA.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
.github/workflows/zfnd-release-binaries.yml Prepares and signs binary bundles.
.github/workflows/zfnd-attach-release-binaries.yml Attaches prepared bundles to releases.
.github/workflows/release-binaries.yml Composes preparation and attachment.
.github/workflows/README.md Documents the split workflow.

Comment thread .github/workflows/zfnd-release-binaries.yml Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

.github/workflows/zfnd-release-binaries.yml:55

  • actions/checkout accepts mutable refs and abbreviated SHAs, but the provenance copies this input directly into digest.gitCommit and the later verification compares against the same unverified string. A caller passing main, a tag, or a short SHA therefore produces an attestation that does not identify the commit actually built, despite this workflow's immutable-source contract. Reject anything except a full commit SHA and confirm that the checkout resolved to it before building.
          ref: ${{ inputs.source_sha }}

Comment thread .github/workflows/zfnd-release-binaries.yml
@gustavovalverde

Copy link
Copy Markdown
Member Author

@copilot review

Copilot AI commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

@copilot review

Reviewed the current state and made two fixes in commit fix(release): correct workflow_path extraction and aggregator needs:

workflow_path extraction (zfnd-release-binaries.yml line 131)
The old strip was %@"${REF}", where REF is the caller's event ref (e.g. refs/tags/v1.0.0). In a reusable workflow the WORKFLOW_REF variable ends with the reusable workflow's own ref (e.g. @refs/heads/main), so the suffix never matched and the generated SLSA predicate's externalParameters.workflow.path retained @refs/heads/main. Changed to %@* which strips the @ref portion unconditionally.

release-binaries-success aggregator (release-binaries.yml)
needs only listed build and attach-binaries. If binary-version fails, prepare-binaries and attach-binaries are skipped; alls-green treats skipped as success by default, so the aggregator job passed even when no binaries were produced. Added binary-version and prepare-binaries to needs so a failure in either propagates to alls-green correctly.

No other issues found — the source_sha provenance binding and empty-version guard from the earlier two threads look correct in the current code.

@gustavovalverde

Copy link
Copy Markdown
Member Author

@copilot review

Copilot AI commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

@copilot review

Found and fixed one remaining issue in the latest commit (fix(release): use WORKFLOW_REF for SLSA builder.id):

builder.id used the wrong refjob_workflow_ref was constructed with REF (github.ref, the release tag refs/tags/v1.0.0), so the SLSA predicate's builder.id incorrectly claimed the workflow file lives at the release tag rather than refs/heads/main. WORKFLOW_REF (github.workflow_ref) already carries the canonical {repository}/{path}@{ref} form for the reusable workflow, so it's used directly now.

No other issues found across all four workflow files — the source_sha provenance binding, empty-version guard, workflow_path extraction, and aggregator needs from previous commits are all correct.

@gustavovalverde

Copy link
Copy Markdown
Member Author

@copilot review

Copilot AI commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

@copilot review

No further issues found. The current state across all four changed files is correct:

  • YAML valid in all files
  • Empty-version guard, glibc floor check, Cosign sign/verify, and gh attestation verify + source_sha filter in the bundle step are all intact
  • workflow_path extraction (%@*) correctly strips the @ref suffix
  • WORKFLOW_SHA is used for resolvedDependencies[0].digest.gitCommit (workflow commit SHA) and source_sha for resolvedDependencies[1]
  • release-binaries-success and failure-issue both list all four jobs in needs
  • Missing timeout-minutes on dockerhub-description and failure-issue is pre-existing, not introduced by this PR

The PR is ready to merge.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

.github/workflows/zfnd-release-binaries.yml:55

  • source_sha is described as immutable, but actions/checkout also accepts mutable branches and tags. A caller can therefore pass main; the job builds the branch tip while lines 169-170 record the literal string main as a gitCommit, and the later check accepts that same inaccurate claim. Require a canonical full SHA and verify HEAD matches it before generating provenance.
          ref: ${{ inputs.source_sha }}

@conradoplg conradoplg left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude findings:


  1. The reusable workflow is no longer a trusted builder (high). zfnd-release-binaries.yml:55 now checks out ref: ${{ inputs.source_sha }} — a caller-chosen input — where before it built the event commit. In
    the same change, the if: inputs.release_tag != '' gates were dropped from attest (:182) and cosign (:221-226). That gate's own comment said it existed "so the test harness cannot mint attestations under the
    same --signer-workflow path users are told to trust." Both mitigations are now gone at once: any caller can have an arbitrary commit built, attested, and cosigned under the exact identity install.md tells
    users to trust. Not exploitable today — one caller, guarded to release:released + v tag + ZcashFoundation — but the guard now lives in a different file than the signing. GitHub's own docs make this the
    explicit anti-pattern: a trusted builder is one "whose execution cannot be influenced by input provided through the caller workflow." Cheapest fix: assert inputs.source_sha == github.sha inside the build job.

  2. The new provenance assertion is circular (high). :239-247 verifies that statement.predicate…resolvedDependencies[].digest.gitCommit == source_sha — but that value was written by the same workflow at
    :169-170 from the same input, and gh documents this exact property as forgeable: "only signature.certificate and verifiedTimestamps contain values that cannot be manipulated by the workflow that originated the
    attestation." I tested it: it passes for the right sha, fails for a wrong one — and also passes against GitHub's default predicate, so it can't even detect the custom predicate being silently dropped. It
    reads like a source-binding control but is a tautology in every reachable state. The unforgeable equivalent already exists: gh attestation verify --source-digest "${SOURCE_SHA}", which checks the GitHub-signed
    SourceRepositoryDigest certificate extension.

  3. builder.id is hardcoded (medium). :135 builds job_workflow_ref from a literal zfnd-release-binaries.yml plus github.ref. GitHub's default reads it from the OIDC job_workflow_ref claim. Rename the file, or
    pin the uses: to a sha instead of ./, and builder.id silently lies while the certificate SAN keeps the truth — and builder.id is precisely the field a SLSA consumer uses to identify the builder.

  4. The predicate override is a no-op as shipped (medium). The comment at :114-116 justifies it for "manual recovery runs," but release-binaries.yml:12-15 has only on: release: released — no workflow_dispatch.
    So source_sha is always github.sha, and resolvedDependencies[1] is always an exact duplicate of [0] (confirmed in my test run: identical gitCommit). Net effect: GitHub's predicate, derived from the signed OIDC
    JWT, is swapped for one derived from env vars, to record information already present, for a mode that doesn't exist.

There's a real design tension behind 2–4 worth deciding explicitly. If recovery runs are wanted, add the workflow_dispatch trigger — but then source_sha ≠ github.sha legitimately, the certificate's
SourceRepositoryDigest will point at the dispatch commit rather than the built source, and no self-asserted predicate can close that gap. If they aren't wanted, dropping source_sha, the predicate override, and
the jq check restores the stronger OIDC-derived provenance for free.

  1. No way to exercise this before a real release (low). Removing the artifact-only mode leaves release:released as the only trigger, so the predicate override and jq assertion first execute on a real release.
    A failure in bundle means the release publishes with no binaries attached.

  2. zfnd-attach-release-binaries.yml takes an unvalidated tag (low). It holds contents: write, has no guard of its own, and --clobbers whatever release_tag it's handed (:8-11, :32-37).

  3. Nit. release-binaries.yml:67 writes ${RELEASE_TAG#v} to $GITHUB_OUTPUT without delimiter protection. Not reachable — git ref names can't contain control characters — but a printf/heredoc guard is free.

gustavovalverde and others added 7 commits August 4, 2026 15:28
Two metadata/reliability fixes identified during review:

1. workflow_path in the SLSA predicate: the previous suffix strip
   used REF (the caller's release tag, e.g. refs/tags/v1.0.0) to
   remove the trailing @ref from WORKFLOW_REF (which ends with
   @refs/heads/main for a reusable workflow). Those never match, so
   the path retained the @refs/heads/main suffix.  Use %@* to strip
   the ref regardless of its value.

2. release-binaries-success aggregator: only listed `build` and
   `attach-binaries` in `needs`. If `binary-version` fails,
   `prepare-binaries` and `attach-binaries` are skipped; alls-green
   treats skipped as success by default, so the aggregator passed
   despite no binaries being produced. Adding `binary-version` and
   `prepare-binaries` to `needs` surfaces the failure correctly.
The previous commit used REF (github.ref, the release event tag
e.g. refs/tags/v1.0.0) to construct job_workflow_ref, making the
SLSA predicate's builder.id claim the workflow file lives at the
release tag ref rather than its actual location on refs/heads/main.

WORKFLOW_REF (github.workflow_ref) already contains the canonical
form "{repository}/{path}@{ref}" for the reusable workflow and is
the correct source for builder.id.
@gustavovalverde
gustavovalverde force-pushed the refactor/release-binary-preparation branch from f917a8c to 0070123 Compare August 4, 2026 19:29
@gustavovalverde

Copy link
Copy Markdown
Member Author

@conradoplg these were addressed

@mergify mergify Bot added the queued label Aug 4, 2026
@mergify

mergify Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Merge Queue Status

  • Entered queue2026-08-04 22:15 UTC · Rule: batched · triggered by rule move to any queue if GitHub Rulesets are satisfied
  • Checks skipped · PR is already up-to-date
  • Merged2026-08-04 22:16 UTC · at 0070123bb92c5871da96cb09bb89a654d5be239d · merge

This pull request spent 45 seconds in the queue, including 9 seconds running CI.

Required conditions to merge

@mergify
mergify Bot merged commit 485d859 into main Aug 4, 2026
95 checks passed
@mergify
mergify Bot deleted the refactor/release-binary-preparation branch August 4, 2026 22:16
@mergify mergify Bot removed the queued label Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-devops Area: Pipelines, CI/CD and Dockerfiles C-enhancement Category: This is an improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants