Skip to content

Ensure Copilot proxy-auth template resolves for PR Sous Chef #40623

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pelikhan merged 13 commits into from
Jun 21, 2026
+127 −23
Merged

Ensure Copilot proxy-auth template resolves for PR Sous Chef #40623

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
4dc7ac3
Initial plan
Copilot Jun 21, 2026
ec60fc9
Handle missing Copilot proxy auth template gracefully
Copilot Jun 21, 2026
0eb4a79
Polish Copilot auth fallback formatting
Copilot Jun 21, 2026
c737b1c
Resolve Copilot proxy auth template path
Copilot Jun 21, 2026
9b93555
Deduplicate prompt dir test setup
Copilot Jun 21, 2026
d6e7634
Scope prompt path fixture to tests
Copilot Jun 21, 2026
6996c40
Make prompt path tests portable
Copilot Jun 21, 2026
ccb067a
Cover runtime prompt path resolution
Copilot Jun 21, 2026
26c99dd
Clean up prompt path temp dirs
Copilot Jun 21, 2026
f7264e3
Exercise real prompt template resolution
Copilot Jun 21, 2026
225b514
Deduplicate prompt env cleanup
Copilot Jun 21, 2026
3473c08
Share prompt template test setup
Copilot Jun 21, 2026
9817f04
Tighten prompt path integration tests
Copilot Jun 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions actions/setup/js/copilot_harness.cjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,8 @@
require("./shim.cjs");

const fs = require("fs");
const path = require("path");
const crypto = require("crypto");
const { renderTemplateFromFile } = require("./messages_core.cjs");
const { getPromptPath, renderTemplateFromFile } = require("./messages_core.cjs");
const { runProcess, formatDuration, sleep, isCopilotSDKEnabled, buildCopilotSDKEnv } = require("./process_runner.cjs");
const { buildCopilotSDKServerArgs, getCopilotSDKServerPort, startCopilotSDKServer, stopCopilotSDKServer, waitForCopilotSDKServer } = require("./copilot_sdk_sidecar.cjs");
const {
Expand Down Expand Up @@ -81,7 +80,7 @@ const PROMPT_FILE_INLINE_THRESHOLD_LABEL = "100KB";
const MAX_ENV_VAR_PREVIEW_LENGTH = 120;
const OUTPUT_TAIL_MAX_CHARS = 600;
const OUTPUT_TAIL_MAX_LINES = 12;
const COPILOT_REQUESTS_PROXY_AUTH_403_TEMPLATE_PATH = path.join(__dirname, "../md/copilot_requests_proxy_auth_403.md");
const COPILOT_REQUESTS_PROXY_AUTH_403_TEMPLATE_NAME = "copilot_requests_proxy_auth_403.md";
// Pattern to detect transient CAPIError 400 in copilot output
const CAPI_ERROR_400_PATTERN = /CAPIError:\s*400/;

Expand Down Expand Up @@ -420,9 +419,10 @@ function envFlagEnabled(value) {
* Build a more actionable Copilot auth diagnostic when a 401/403 came from the gh-aw API proxy.
* @param {string} output
* @param {NodeJS.ProcessEnv} [env]
* @param {{ renderTemplateFromFile?: typeof renderTemplateFromFile }} [options]
* @returns {string}
*/
function buildCopilotProxyAuthFailureDiagnostic(output, env = process.env) {
function buildCopilotProxyAuthFailureDiagnostic(output, env = process.env, options = {}) {
const authFailure = parseProviderAuthFailure(output);
if (!authFailure || !isLikelyAWFAPIProxyURL(authFailure.providerUrl)) {
return "";
Expand All @@ -431,7 +431,8 @@ function buildCopilotProxyAuthFailureDiagnostic(output, env = process.env) {
const selectedModel = typeof env.COPILOT_MODEL === "string" && env.COPILOT_MODEL.trim() ? env.COPILOT_MODEL.trim() : "(unset)";
const stage = detectCopilotAuthFailureStage(output);
if (authFailure.statusCode === "403" && envFlagEnabled(env.S2STOKENS)) {
return renderTemplateFromFile(COPILOT_REQUESTS_PROXY_AUTH_403_TEMPLATE_PATH, {
const render = options.renderTemplateFromFile || renderTemplateFromFile;
return render(getPromptPath(COPILOT_REQUESTS_PROXY_AUTH_403_TEMPLATE_NAME), {

@github-actions github-actions Bot Jun 21, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnose] getPromptPath() throws if neither GH_AW_PROMPTS_DIR nor RUNNER_TEMP is set — the old hardcoded __dirname constant never threw. In normal GitHub Actions runs RUNNER_TEMP is always present, but if either variable is missing for any reason the harness will crash here instead of returning a graceful empty diagnostic.

💡 Suggested guard 
let templatePath;
try {
  templatePath = getPromptPath(COPILOT_REQUESTS_PROXY_AUTH_403_TEMPLATE_NAME);
} catch {
  // RUNNER_TEMP not set — skip the 403 template diagnostic
  return "";
}
return render(templatePath, { selected_model: selectedModel, stage });

Alternatively, define a safeGetPromptPath helper that returns null on failure so the guard is reusable across branches.

selected_model: selectedModel,
stage,
Comment on lines 433 to 437
});
Expand Down
139 changes: 121 additions & 18 deletions actions/setup/js/copilot_harness.test.cjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,50 @@ function makeHarnessTempDir(name) {
return fs.mkdtempSync(path.join(agentTempDir, name));
}

function withTestPromptsDir(promptsDir, callback) {
const originalPromptsDir = process.env.GH_AW_PROMPTS_DIR;
if (typeof promptsDir === "string") {
process.env.GH_AW_PROMPTS_DIR = promptsDir;
} else {
delete process.env.GH_AW_PROMPTS_DIR;
}
try {
return callback();
} finally {
if (typeof originalPromptsDir === "string") {
process.env.GH_AW_PROMPTS_DIR = originalPromptsDir;
} else {
delete process.env.GH_AW_PROMPTS_DIR;
}
}
}

function withRunnerTemp(runnerTempDir, callback) {
const originalRunnerTemp = process.env.RUNNER_TEMP;
process.env.RUNNER_TEMP = runnerTempDir;
try {
return callback();
} finally {
if (typeof originalRunnerTemp === "string") {
process.env.RUNNER_TEMP = originalRunnerTemp;
} else {
delete process.env.RUNNER_TEMP;
}
}
}
Comment on lines +77 to +89

function withTemporaryPromptTemplate(prefix, sourceTemplateDir, promptDirResolver, callback) {
const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), prefix));
try {
const promptsDir = promptDirResolver(tempDir);
fs.mkdirSync(promptsDir, { recursive: true });
fs.copyFileSync(path.join(sourceTemplateDir, "copilot_requests_proxy_auth_403.md"), path.join(promptsDir, "copilot_requests_proxy_auth_403.md"));

@github-actions github-actions Bot Jun 21, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] withTemporaryPromptTemplate hardcodes copilot_requests_proxy_auth_403.md, making this utility unusable for future template tests. A templateName parameter would cost nothing and allow reuse.

💡 Suggested signature 
function withTemporaryPromptTemplate(prefix, sourceTemplateDir, templateName, promptDirResolver, callback) {
  const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), prefix));
  try {
    const promptsDir = promptDirResolver(tempDir);
    fs.mkdirSync(promptsDir, { recursive: true });
    fs.copyFileSync(
      path.join(sourceTemplateDir, templateName),
      path.join(promptsDir, templateName)
    );
    return callback(tempDir, promptsDir);
  } finally {
    fs.rmSync(tempDir, { recursive: true, force: true });
  }
}

Call sites would pass "copilot_requests_proxy_auth_403.md" as the third argument.

return callback(tempDir, promptsDir);
} finally {
fs.rmSync(tempDir, { recursive: true, force: true });
}
}

describe("copilot_harness.cjs", () => {
// Test the core logic patterns used by the driver without importing the module
// (importing the module would invoke main() which calls process.exit).
Expand Down Expand Up @@ -959,6 +1003,8 @@ describe("copilot_harness.cjs", () => {
});

describe("gh-aw API proxy auth diagnostics", () => {
const promptsSourceDir = path.resolve("../md");

@github-actions github-actions Bot Jun 21, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] path.resolve("../md") is CWD-relative — it resolves correctly when vitest runs from actions/setup/js/ (the package.json root), but breaks if tests are ever invoked from a different working directory. Anchoring to __dirname is more robust.

💡 Suggested fix 
const promptsSourceDir = path.resolve(__dirname, "../md");

__dirname is always the directory of this .cjs file (actions/setup/js/), so the path resolves correctly regardless of where vitest is invoked.


it("rewrites local proxy 401 errors to COPILOT_GITHUB_TOKEN guidance", () => {
const diagnostic = buildCopilotProxyAuthFailureDiagnostic("Authentication failed with provider at http://172.30.0.30:10002 (HTTP 401).\nCheck your COPILOT_PROVIDER_API_KEY or COPILOT_PROVIDER_BEARER_TOKEN.", {
COPILOT_MODEL: "claude-sonnet-4.5",
Expand All @@ -974,30 +1020,87 @@ describe("copilot_harness.cjs", () => {
});

it("rewrites local proxy 403 errors in copilot-requests mode to org-billing guidance", () => {
const diagnostic = buildCopilotProxyAuthFailureDiagnostic("Authentication failed with provider at http://172.30.0.30:10002 (HTTP 403).\nCheck your COPILOT_PROVIDER_API_KEY or COPILOT_PROVIDER_BEARER_TOKEN.", {
COPILOT_MODEL: "claude-sonnet-4.5",
S2STOKENS: "true",
});
withTestPromptsDir(promptsSourceDir, () => {
const diagnostic = buildCopilotProxyAuthFailureDiagnostic("Authentication failed with provider at http://172.30.0.30:10002 (HTTP 403).\nCheck your COPILOT_PROVIDER_API_KEY or COPILOT_PROVIDER_BEARER_TOKEN.", {
COPILOT_MODEL: "claude-sonnet-4.5",
S2STOKENS: "true",
});

expect(diagnostic).toContain("Copilot requests authentication failed");
expect(diagnostic).toContain("HTTP 403");
expect(diagnostic).toContain("model=claude-sonnet-4.5");
expect(diagnostic).toContain("stage=starting the Copilot CLI request");
expect(diagnostic).toContain("permissions.copilot-requests: write");
expect(diagnostic).toContain("centralized Copilot billing");
expect(diagnostic).toContain("https://github.github.com/gh-aw/reference/billing/");
expect(diagnostic).not.toContain("COPILOT_PROVIDER_API_KEY");
expect(diagnostic).toContain("Copilot requests authentication failed");
expect(diagnostic).toContain("HTTP 403");
expect(diagnostic).toContain("model=claude-sonnet-4.5");
expect(diagnostic).toContain("stage=starting the Copilot CLI request");
expect(diagnostic).toContain("permissions.copilot-requests: write");
expect(diagnostic).toContain("centralized Copilot billing");
expect(diagnostic).toContain("https://github.github.com/gh-aw/reference/billing/");
expect(diagnostic).not.toContain("COPILOT_PROVIDER_API_KEY");
});
});

it("treats truthy S2STOKENS values as copilot-requests mode for 403 guidance", () => {
const diagnostic = buildCopilotProxyAuthFailureDiagnostic("Authentication failed with provider at http://172.30.0.30:10002 (HTTP 403).", {
COPILOT_MODEL: "claude-sonnet-4.5",
S2STOKENS: " YES ",
withTestPromptsDir(promptsSourceDir, () => {
const diagnostic = buildCopilotProxyAuthFailureDiagnostic("Authentication failed with provider at http://172.30.0.30:10002 (HTTP 403).", {
COPILOT_MODEL: "claude-sonnet-4.5",
S2STOKENS: " YES ",
});

expect(diagnostic).toContain("Copilot requests authentication failed");
expect(diagnostic).toContain("https://github.github.com/gh-aw/reference/billing/");
expect(diagnostic).not.toContain("COPILOT_PROVIDER_API_KEY");
});
});

expect(diagnostic).toContain("Copilot requests authentication failed");
expect(diagnostic).toContain("https://github.github.com/gh-aw/reference/billing/");
expect(diagnostic).not.toContain("COPILOT_PROVIDER_API_KEY");
it("resolves the 403 guidance template from the runtime prompts directory", () => {
withTemporaryPromptTemplate(
"runtime-prompts-",
promptsSourceDir,
tempDir => tempDir,
(_tempDir, runtimePromptsDir) => {
withTestPromptsDir(runtimePromptsDir, () => {
const renderTemplateFromFile = vi.fn((templatePath, context) => {

@github-actions github-actions Bot Jun 21, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] The vi.fn mock reimplements a simplified template engine using .replace("{selected_model}", ...). Since renderTemplate does use {key} syntax this happens to be correct, but it creates an implicit contract between the mock and the real implementation. The key assertion here is toHaveBeenCalledWith (verifying path resolution) — the content checks are already covered by the non-mock 403 tests above.

💡 Simpler alternative

Return a predictable sentinel string so the mock remains honest about its scope:

const renderTemplateFromFile = vi.fn(() => "mock-render-result");
// ...
expect(renderTemplateFromFile).toHaveBeenCalledWith(
  path.join(runtimePromptsDir, "copilot_requests_proxy_auth_403.md"),
  { selected_model: "claude-sonnet-4.5", stage: "starting the Copilot CLI request" }
);
// content correctness is already asserted in the earlier non-mock 403 tests

return fs.readFileSync(templatePath, "utf8").replace("{selected_model}", context.selected_model).replace("{stage}", context.stage);
});
const diagnostic = buildCopilotProxyAuthFailureDiagnostic(
"Authentication failed with provider at http://172.30.0.30:10002 (HTTP 403).",
{
COPILOT_MODEL: "claude-sonnet-4.5",
S2STOKENS: "true",
},
{ renderTemplateFromFile }
);

expect(diagnostic).toContain("Copilot requests authentication failed");
expect(diagnostic).toContain("model=claude-sonnet-4.5");
expect(diagnostic).toContain("stage=starting the Copilot CLI request");
expect(renderTemplateFromFile).toHaveBeenCalledWith(path.join(runtimePromptsDir, "copilot_requests_proxy_auth_403.md"), {
selected_model: "claude-sonnet-4.5",
stage: "starting the Copilot CLI request",
});
});
}
);
});

it("resolves the 403 guidance template from RUNNER_TEMP when GH_AW_PROMPTS_DIR is unset", () => {
withTemporaryPromptTemplate(
"runner-temp-",
promptsSourceDir,
tempDir => path.join(tempDir, "gh-aw", "prompts"),
runnerTempDir => {
withTestPromptsDir(undefined, () => {
withRunnerTemp(runnerTempDir, () => {
const diagnostic = buildCopilotProxyAuthFailureDiagnostic("Authentication failed with provider at http://172.30.0.30:10002 (HTTP 403).", {
COPILOT_MODEL: "claude-sonnet-4.5",
S2STOKENS: "true",
});

expect(diagnostic).toContain("Copilot requests authentication failed");
expect(diagnostic).toContain("model=claude-sonnet-4.5");
expect(diagnostic).toContain("stage=starting the Copilot CLI request");
});
});
}
);
});

it("returns empty string for proxy 403 when S2STOKENS is not set (BYOK mode)", () => {
Expand Down
Loading