test(updates): macOS auto-update E2E with real install + relaunch by charlesvien · Pull Request #2828 · PostHog/code

charlesvien · 2026-06-22T02:59:06Z

Problem

The electron-builder + electron-updater migration gave us a real auto-update path (download, install, relaunch), but nothing tests it end to end.

Changes

Two tiny test-gated hooks in the main process, both inert unless POSTHOG_E2E_UPDATE_FEED is set (no production behavior change):

electron-updater.ts points the updater at a local feed via setFeedURL. disableDifferentialDownload, autoDownload, autoInstallOnAppQuit and isSupported are untouched.
main/index.ts exposes globalThis.__e2eUpdates (check/download/install/status) wired to the real UpdatesService.

Harness:

scripts/dev-update/serve.mjs: dependency-free, range-capable static feed server.
scripts/dev-update/build-pair.sh: builds a signed 2.0.0 feed and a runnable signed 1.0.0 app (SKIP_NOTARIZE=1).
tests/e2e/fixtures/update.ts: ditto run-copy so a retry restarts from 1.0.0, plist version read, pkill helpers.
tests/e2e/tests/update.spec.ts: phase 1 drives download then install via the main-process hook; phase 2 waits for the on-disk bundle to swap, kills the auto-relaunched instance, then fresh-launches and asserts app.getVersion() is 2.0.0.

CI: a dedicated code-update-e2e.yml, macOS only, reusing the existing build-macos steps and Apple signing secrets. It is not on the PR gate (nightly + workflow_dispatch); a temporary push trigger on this branch lets it run on the PR and is meant to be removed before merge.

The swap is signature-gated (Squirrel.Mac matches the designated requirement), which is why this runs in CI where the signing cert is present rather than as a local unit test. Notarization is skipped on purpose: it is a Gatekeeper concern, not what the in-place update verifies.

How did you test this?

Ran locally:

pnpm --filter code typecheck passes with the hooks.
Biome clean on all new and changed files.
node --check serve.mjs and bash -n build-pair.sh pass.
The e2e tsconfig typechecks the two new e2e files (the only errors are pre-existing in smoke.spec.ts).

Not run locally: the full signed build plus end-to-end swap (two signed mac builds, needs the Developer ID cert). That runs in the new workflow on this branch via the temporary push trigger, which is the real validation.

Automatic notifications

Publish to changelog?
Alert Sales and Marketing teams?

github-actions · 2026-06-22T02:59:36Z

React Doctor found no issues in the changed files. 🎉

_{Reviewed by React Doctor for commit 8257a56.}

greptile-apps · 2026-06-22T03:02:51Z

Prompt To Fix All With AI

Fix the following 3 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 3
.github/workflows/code-update-e2e.yml:10-14
**Temporary push trigger not removed before merge**

The PR description explicitly states this push trigger "is meant to be removed before merge." It's still present. If the source branch (`test/macos-auto-update-e2e`) continues to exist in the repo after the merge, any future push to it will fire this workflow — which runs two full signed macOS builds and exercises the real install. That's expensive (and confusing) for what should be a nightly-only job.

### Issue 2 of 3
apps/code/tests/e2e/tests/update.spec.ts:63-130
**`app` and `updated` not closed in finally**

If `pollStatus` for check/download times out (phase 1 fails), the `app` Electron process is left running — only `feed.kill()` is in the `finally` block. Similarly, if the `expect(version).toBe(NEW_VERSION)` assertion at line 126 fails, `updated` is leaked. Playwright does not automatically close `electron.launch()` instances on test failure. Adding `app.close().catch(() => {})` / `updated.close().catch(() => {})` to the `finally` block (tracked via `let app` / `let updated` in outer scope) would prevent stale processes from interfering with a retry.

### Issue 3 of 3
apps/code/scripts/dev-update/serve.mjs:50-52
**Suffix-range requests (`bytes=-N`) compute wrong offsets**

When the range header is `bytes=-500` (last 500 bytes), the regex captures `match[1] = ""` and `match[2] = "500"`. Because `""` is falsy, `start` is set to `0` instead of `stat.size - 500`, so the response sends the wrong slice of the file. The electron-updater differential/blockmap code can emit suffix ranges during verification. In practice `electron-updater` on this path likely uses open-ended ranges (`bytes=0-`), but a defensive fix would be: `const start = match?.[1] ? Number(match[1]) : match?.[2] ? stat.size - Number(match[2]) : 0`.

_{Reviews (1): Last reviewed commit: "add macos auto-update e2e harness and ci" | Re-trigger Greptile}

greptile-apps · 2026-06-22T03:02:54Z

+  # Temporary: also run on this branch so the harness can be exercised on the PR.
+  # Remove this push trigger once merged; nightly + dispatch is the steady state.
+  push:
+    branches:
+      - test/macos-auto-update-e2e


Temporary push trigger not removed before merge

The PR description explicitly states this push trigger "is meant to be removed before merge." It's still present. If the source branch (test/macos-auto-update-e2e) continues to exist in the repo after the merge, any future push to it will fire this workflow — which runs two full signed macOS builds and exercises the real install. That's expensive (and confusing) for what should be a nightly-only job.

Prompt To Fix With AI

This is a comment left during a code review. Path: .github/workflows/code-update-e2e.yml Line: 10-14 Comment: **Temporary push trigger not removed before merge** The PR description explicitly states this push trigger "is meant to be removed before merge." It's still present. If the source branch (`test/macos-auto-update-e2e`) continues to exist in the repo after the merge, any future push to it will fire this workflow — which runs two full signed macOS builds and exercises the real install. That's expensive (and confusing) for what should be a nightly-only job. How can I resolve this? If you propose a fix, please make it concise.

greptile-apps · 2026-06-22T03:02:55Z

+    try {
+      // Phase 1: drive the real download + install on the old build.
+      const app = await electron.launch({
+        executablePath: RUN_APP_BIN,
+        args: [],
+        env: {
+          ...process.env,
+          ELECTRON_DISABLE_GPU: "1",
+          POSTHOG_E2E_UPDATE_FEED: FEED_URL,
+        },
+      });
+
+      await expect
+        .poll(
+          () => app.evaluate(() => typeof (globalThis as Hooked).__e2eUpdates),
+          {
+            timeout: 30_000,
+            message: "update hook was never installed",
+          },
+        )
+        .toBe("object");
+
+      await app.evaluate(() => (globalThis as Hooked).__e2eUpdates.check());
+      await pollStatus(
+        app,
+        (s) => s.available === true && s.availableVersion === NEW_VERSION,
+        "update never became available",
+      );
+
+      await app.evaluate(() => (globalThis as Hooked).__e2eUpdates.download());
+      await pollStatus(
+        app,
+        (s) => s.updateReady === true,
+        "update never finished downloading",
+      );
+
+      const closed = app.waitForEvent("close");
+      void app
+        .evaluate(() => {
+          void (globalThis as Hooked).__e2eUpdates.install();
+        })
+        .catch(() => undefined);
+      await closed;
+
+      // Phase 2: prove the bundle swapped and a fresh launch is the new version.
+      await waitUntil(
+        () => readBundleVersion(RUN_APP) === NEW_VERSION,
+        120_000,
+        "bundle was not swapped to the new version",
+      );
+      killApp();
+      await waitUntil(
+        () => !isAppRunning(),
+        30_000,
+        "relaunched instance did not exit",
+      );
+
+      const updated = await electron.launch({
+        executablePath: RUN_APP_BIN,
+        args: [],
+        env: { ...process.env, ELECTRON_DISABLE_GPU: "1" },
+      });
+      const version = await updated.evaluate(({ app: a }) => a.getVersion());
+      expect(version).toBe(NEW_VERSION);
+      await updated.close();
+    } finally {
+      feed.kill();
+    }


app and updated not closed in finally

If pollStatus for check/download times out (phase 1 fails), the app Electron process is left running — only feed.kill() is in the finally block. Similarly, if the expect(version).toBe(NEW_VERSION) assertion at line 126 fails, updated is leaked. Playwright does not automatically close electron.launch() instances on test failure. Adding app.close().catch(() => {}) / updated.close().catch(() => {}) to the finally block (tracked via let app / let updated in outer scope) would prevent stale processes from interfering with a retry.

Prompt To Fix With AI

This is a comment left during a code review. Path: apps/code/tests/e2e/tests/update.spec.ts Line: 63-130 Comment: **`app` and `updated` not closed in finally** If `pollStatus` for check/download times out (phase 1 fails), the `app` Electron process is left running — only `feed.kill()` is in the `finally` block. Similarly, if the `expect(version).toBe(NEW_VERSION)` assertion at line 126 fails, `updated` is leaked. Playwright does not automatically close `electron.launch()` instances on test failure. Adding `app.close().catch(() => {})` / `updated.close().catch(() => {})` to the `finally` block (tracked via `let app` / `let updated` in outer scope) would prevent stale processes from interfering with a retry. How can I resolve this? If you propose a fix, please make it concise.

greptile-apps · 2026-06-22T03:02:56Z

+    const match = /bytes=(\d*)-(\d*)/.exec(range);
+    const start = match?.[1] ? Number(match[1]) : 0;
+    const end = match?.[2] ? Number(match[2]) : stat.size - 1;


Suffix-range requests (bytes=-N) compute wrong offsets

When the range header is bytes=-500 (last 500 bytes), the regex captures match[1] = "" and match[2] = "500". Because "" is falsy, start is set to 0 instead of stat.size - 500, so the response sends the wrong slice of the file. The electron-updater differential/blockmap code can emit suffix ranges during verification. In practice electron-updater on this path likely uses open-ended ranges (bytes=0-), but a defensive fix would be: const start = match?.[1] ? Number(match[1]) : match?.[2] ? stat.size - Number(match[2]) : 0.

Prompt To Fix With AI

This is a comment left during a code review. Path: apps/code/scripts/dev-update/serve.mjs Line: 50-52 Comment: **Suffix-range requests (`bytes=-N`) compute wrong offsets** When the range header is `bytes=-500` (last 500 bytes), the regex captures `match[1] = ""` and `match[2] = "500"`. Because `""` is falsy, `start` is set to `0` instead of `stat.size - 500`, so the response sends the wrong slice of the file. The electron-updater differential/blockmap code can emit suffix ranges during verification. In practice `electron-updater` on this path likely uses open-ended ranges (`bytes=0-`), but a defensive fix would be: `const start = match?.[1] ? Number(match[1]) : match?.[2] ? stat.size - Number(match[2]) : 0`. How can I resolve this? If you propose a fix, please make it concise.

charlesvien · 2026-06-22T06:01:35Z

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

stamphog

Gates denied because this PR adds a new CI/CD workflow file (.github/workflows/code-update-e2e.yml), which is on the infra/CI deny-list and requires human review. Additionally, there are unresolved bot comments flagging a push trigger that was explicitly noted for removal before merge but wasn't removed, and a functional bug in the range-request handling in serve.mjs.

greptile-apps Bot reviewed Jun 22, 2026

View reviewed changes

charlesvien force-pushed the feat/auto-update-ux branch from 97fb702 to d07a09a Compare June 22, 2026 06:01

charlesvien force-pushed the test/macos-auto-update-e2e branch from b6f48b2 to afd5861 Compare June 22, 2026 06:01

charlesvien added the Stamphog This will request an autostamp by stamphog on small changes label Jun 22, 2026

stamphog Bot reviewed Jun 22, 2026

View reviewed changes

stamphog Bot removed the Stamphog This will request an autostamp by stamphog on small changes label Jun 22, 2026

charlesvien force-pushed the test/macos-auto-update-e2e branch 4 times, most recently from 71f5705 to 817d137 Compare June 22, 2026 07:21

charlesvien added 11 commits June 22, 2026 08:32

add macos auto-update e2e harness and ci

c9c05ce

pass --publish never in e2e build-pair

d53bbe5

gate update e2e spec behind RUN_UPDATE_E2E

98debc3

add local auto-update testing doc

c1e9122

harden update e2e checks and evidence

6fc7e22

fix update e2e stats report path

ec644f5

assert Squirrel auto-relaunch in update e2e

77dc853

add update proof artifact and per-build uploads

b0b389f

document testing with CI-signed builds

3d4b24d

add run-from-ci local update test script

3e2d915

scope update e2e push trigger to code changes

8257a56

charlesvien force-pushed the test/macos-auto-update-e2e branch from 817d137 to 8257a56 Compare June 22, 2026 15:32

charlesvien force-pushed the feat/auto-update-ux branch from f2656c7 to 482f3bf Compare June 22, 2026 15:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test(updates): macOS auto-update E2E with real install + relaunch#2828

test(updates): macOS auto-update E2E with real install + relaunch#2828
charlesvien wants to merge 11 commits into
feat/auto-update-uxfrom
test/macos-auto-update-e2e

charlesvien commented Jun 22, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 22, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Jun 22, 2026

Uh oh!

greptile-apps Bot Jun 22, 2026

Uh oh!

greptile-apps Bot Jun 22, 2026

Uh oh!

greptile-apps Bot Jun 22, 2026

Uh oh!

charlesvien commented Jun 22, 2026 •

edited

Loading

Uh oh!

stamphog Bot left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

charlesvien commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Changes

How did you test this?

Automatic notifications

Uh oh!

github-actions Bot commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps Bot commented Jun 22, 2026

Uh oh!

greptile-apps Bot Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

charlesvien commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stamphog Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

charlesvien commented Jun 22, 2026 •

edited

Loading

github-actions Bot commented Jun 22, 2026 •

edited

Loading

charlesvien commented Jun 22, 2026 •

edited

Loading