Skip to content

Commit 75cefec

Browse files
hkiratclaude
andcommitted
fix: enable parallel tool calls and prompt LLM to batch them
- Add parallel_tool_calls: true to both agent and setup LLM requests - Add system prompt instruction to batch multiple tool calls in one response (e.g. read 3 files at once instead of one per round-trip) - Reduces total LLM API calls and the triangular token cost growth Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent d0e9355 commit 75cefec

3 files changed

Lines changed: 4 additions & 2 deletions

File tree

apps/server/src/services/agent-script.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -128,7 +128,7 @@ async function callLLM(msgs) {
128128
Authorization: \`Bearer \${llmApiKey}\`,
129129
"X-Title": "Vendi",
130130
},
131-
body: JSON.stringify({ model: llmModel, messages: msgs, tools, max_tokens: 4096 }),
131+
body: JSON.stringify({ model: llmModel, messages: msgs, tools, max_tokens: 4096, parallel_tool_calls: true }),
132132
signal: controller.signal,
133133
});
134134
if (!res.ok) throw new Error(\`LLM error \${res.status}: \${await res.text()}\`);

apps/server/src/services/session.service.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -144,6 +144,7 @@ function buildSystemPrompt(project: {
144144
`- Startup commands: ${project.startupCommands.length > 0 ? project.startupCommands.join(" && ") : "check package.json"}`,
145145
"",
146146
"RULES:",
147+
"- IMPORTANT: Batch multiple tool calls in a single response whenever possible. For example, if you need to read 3 files, call read_file 3 times in one response instead of reading them one at a time. This saves time and cost.",
147148
"- Only modify files matching these patterns: " +
148149
(project.allowedFilePatterns.length > 0
149150
? project.allowedFilePatterns.join(", ")

apps/server/src/services/setup.service.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,7 @@ OUTPUT FORMAT — when you have everything:
103103
[/SETUP_COMPLETE]
104104
105105
RULES:
106+
- IMPORTANT: Batch multiple tool calls in a single response whenever possible. For example, read package.json, .env.example, and docker-compose.yml all in one response instead of one at a time. This saves time and cost.
106107
- ALWAYS read the codebase FIRST before saying anything to the user
107108
- Do NOT ask the user about services, ports, startup commands, or file patterns — detect them yourself
108109
- Do NOT ask about or suggest code changes — this is setup, not development
@@ -160,7 +161,7 @@ async function callLLM(messages: any[]): Promise<any> {
160161
Authorization: `Bearer ${key}`,
161162
"X-Title": "Vendi",
162163
},
163-
body: JSON.stringify({ model, messages, tools, max_tokens: 4096 }),
164+
body: JSON.stringify({ model, messages, tools, max_tokens: 4096, parallel_tool_calls: true }),
164165
});
165166
if (!res.ok) throw new Error(`LLM error ${res.status}: ${await res.text()}`);
166167
const json = await res.json();

0 commit comments

Comments
 (0)