Carrying the torch: 48 community PRs merged + fork fixes (v0.101.0-v0.109.0)#1028
Open
wishborn wants to merge 170 commits into
Open
Carrying the torch: 48 community PRs merged + fork fixes (v0.101.0-v0.109.0)#1028wishborn wants to merge 170 commits into
wishborn wants to merge 170 commits into
Conversation
Add comprehensive Replicate provider implementation supporting all core features: text generation, streaming (SSE), structured output, embeddings, image generation, and audio (TTS/STT). Features: - Text generation with system prompts and conversation history - Real-time SSE streaming with automatic fallback to simulated streaming - Structured output with JSON schema validation - Image generation (FLUX, Stable Diffusion XL, etc.) - Text-to-Speech with multiple voices (Kokoro-82m) - Speech-to-Text with Whisper (WAV, MP3, FLAC, OGG, M4A) - Embeddings (single and batch, 768-dimensional vectors) Implementation: - Async prediction management with configurable polling - Sync mode (Prefer: wait header) for lower latency - Comprehensive error handling with typed exceptions - Full PHPStan level 8 compliance - 21 tests with 60 assertions (100% feature coverage) - 455 lines of comprehensive documentation Files changed: 58 files, 4,444+ lines added
…chronously This adds the ability to be able to send a request to a provider to create a transcript where the provider will give you an id and then send a webhook to you in the future when the job is done with that id. This is just supplying the interface that a provider can utilize in the future.
Add comprehensive support for Alibaba Cloud's Qwen models via the DashScope native API (/api/v1), covering text generation, streaming, structured output, embeddings, image generation, and image editing. Key features: - Text generation with multi-step tool calling - Multi-modal (VL) support with automatic endpoint routing - Streaming with DashScope SSE protocol and reasoning/thinking tokens - Structured output with both JSON Object and JSON Schema modes - Embeddings with configurable dimensions - Image generation (qwen-image-max/plus) and editing (qwen-image-edit) - Region-aware configuration (International, China, US deployments) - 52 tests with real API fixtures (176 assertions) Co-authored-by: Cursor <cursoragent@cursor.com>
StreamEndEvent.usage can be null when providers don't include usage data in their final stream chunk, causing a TypeError downstream. Add `?? new Usage(0, 0)` fallback to emitStreamEndEvent() in all providers missing it, matching the existing pattern in the OpenAI stream handler.
Add an `api_format` config option to the OpenAI driver that allows switching from the default `/responses` endpoint to `/chat/completions`. This enables using Prism with OpenAI-compatible backends like vLLM, LiteLLM, and LocalAI that only implement the chat/completions API. Set `OPENAI_API_FORMAT=chat_completions` in your env to use it. Only text, structured, and stream methods dispatch conditionally — other modalities (embeddings, images, moderation, TTS, STT) already use standard endpoints that work with compatible backends as-is.
…nfigured Providers that reject unknown parameters (e.g. Perplexity via LiteLLM) return HTTP 400 when `"tools": []` is sent. Return null instead so Arr::whereNotNull() filters it out entirely.
Providers with integrated search capabilities (e.g. Perplexity, You.com) return top-level `citations` and `search_results` fields in chat/completions responses. These were previously ignored. Add ChatCompletionsCitationsMapper to map these into Prism's existing Citation infrastructure, and extract them once per stream in the ChatCompletions stream handler. Citations are passed through on the StreamEndEvent, matching the existing pattern used by the Anthropic handler.
…ob management and result handling
…h job management, result handling, and error mapping
…-anthropic-and-openai
…lue for data retrieval
… provider methods for batch management
… tool loop ## Context When Anthropic's server-side tools (like `web_search`) are used alongside regular user-defined tools, the model can do both in a single response: perform a web search, write text with citations referencing the search results, and call a regular tool. Because a regular tool was called, Prism enters its multi-step tool loop. It executes the tool, then replays the entire conversation back to the API for the next turn. The problem is that when Prism builds the replayed assistant message, it includes the text with citations but drops the `server_tool_use` and `web_search_tool_result` content blocks that the citations reference. The API validates that every citation points to an existing search result, finds none, and rejects the request with: `invalid_request_error - Could not find search result for citation index.` This only triggers when the model performs a server-side tool call AND a regular tool call in the same response. If either happens alone, everything works fine. ## Changes Both the Text and Stream handlers had the same gap in their tool loop replay logic. **Text handler (`Text.php`):** Added `extractProviderToolContent()` that pulls `server_tool_use` and `*_tool_result` content blocks from the API response and stores them in `additionalContent` as `provider_tool_calls` and `provider_tool_results`, the same keys that `MessageMap::mapAssistantMessage()` already reads and serializes back to the API. This follows the existing pattern of `extractText()`, `extractCitations()`, and `extractThinking()`. **Stream handler (`Stream.php`):** The stream state already tracked provider tool calls, provider tool results, and citations during streaming, but `handleToolCalls()` only included `thinking` and `thinking_signature` in the replayed `AssistantMessage`'s `additionalContent`. Now it also includes `citations`, `provider_tool_calls`, and `provider_tool_results`.
…delete, and metadata retrieval functionalities
…e batch job handling with inputFileId support
…xtRequest
- Use ?? [] on items to avoid passing null to buildAndUploadFile()
- Cast json_encode() result to string to satisfy non-empty-string return type
- Change clientRetry default from [] to [0] to satisfy array{0: int} type constraint
Made-with: Cursor
…s from OpenAI responses
…update tests for empty array responses
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts: # src/Audio/PendingRequest.php # src/Providers/Provider.php
…turn it # Conflicts: # src/Providers/OpenRouter/Handlers/Stream.php # src/Providers/OpenRouter/Handlers/Structured.php # src/Providers/OpenRouter/Handlers/Text.php
…streaming citations (includes prism-php#902) # Conflicts: # src/Providers/OpenAI/OpenAI.php # src/Providers/OpenRouter/Handlers/Stream.php
…ge generation # Conflicts: # config/prism.php # src/Enums/Provider.php
…llow-up) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…provider # Conflicts: # config/prism.php # docs/.vitepress/config.mts # docs/core-concepts/structured-output.md # src/Enums/Provider.php # src/PrismManager.php
# Conflicts: # .gitignore # config/prism.php # docs/.vitepress/config.mts # docs/getting-started/introduction.md # src/Enums/Provider.php # src/PrismManager.php
The PR predates the providerToolCalls constructor parameter on Text\Step. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Azure Stream: guard nullable toException() before handleRequestException. - Qwen ImageRequestMap: handle null url() in resolveImageSource. - Replicate Images: annotate Http::get() response type. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… types Models routinely serialize every tool argument as a JSON string (notably Llama models on Groq) even when the schema declares boolean or number, and under strict types the handler crashed with a raw TypeError. Tool::handle now reflects the handler signature and coerces string values into declared int/float/bool types and converts string/int values into BackedEnum instances. Arguments that do not match a declared parameter pass through untouched so the existing validation-error reporting still fires. Closes upstream prism-php#1016; addresses the coercion half of prism-php#1007. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… provider option withProviderOptions(["anthropic_beta" => "skills-2025-10-02"]) — string or array — merges with the config-level ANTHROPIC_BETA flags and lands on the anthropic-beta header for text, structured and stream requests. Unlocks per-request Skills / code-execution / files betas (upstream prism-php#953). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…backfill) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reimplementation of upstream prism-php#932 (answers upstream prism-php#921) against current main — fork issue #4, phase 1. - Tool::clientExecuted() marks a tool the application executes itself: the request loop stops and pending tool calls come back on the response. - Tool::requiresApproval(bool|Closure) gates execution on an approval decision. Pending calls surface as ToolApprovalRequests (server-generated approval ids) on the assistant message and step; the app resumes by appending a ToolResultMessage carrying ToolApprovalResponses. Approved calls execute on resume; denied or unanswered ones produce denial results (deny-by-default). - New value objects (ToolApprovalRequest/Response), stream event type + ToolApprovalRequestEvent, broadcast (BroadcastAdapter mapping), and setMessages() on Text/Structured requests. - CallsTools gains callToolsWithPending / callToolsAndYieldEventsWithPending; legacy signatures and behavior are untouched for providers not yet wired. - Wired end-to-end for Anthropic and OpenAI Text + Structured handlers. Streaming emission and further providers tracked in fork issue #4. - Ports the prism-php#932 unit suite (39 tests) adapted to the split method design. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Provider-supplied output URLs are now fetched only over https from allowlisted hosts (replicate.delivery by default, configurable via prism.providers.replicate.download_hosts), with an explicit 30s timeout and a 25MB size cap. Rejected or failed downloads fall back to the URL as before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extends the v0.106.0 HITL mechanics (fork issue #4, phase 2) beyond Anthropic/OpenAI Responses to every Text and Structured handler with a tool loop: Azure, DeepSeek, Gemini, Groq, Mistral, Ollama, OpenRouter, Qwen, Requesty, Vertex, xAI, Z, and OpenAI ChatCompletions. Each handler resolves pending approvals before sending, stops its loop when marked tool calls are pending, and carries the approval requests on the assistant message for resume correlation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Streams now resolve pending approvals at start (yielding the resolved tool result events) and, when client-executed or approval-required tool calls are encountered, yield ToolApprovalRequestEvents and end the stream with FinishReason::ToolCalls instead of looping. Ports the upstream prism-php#932 stream fixtures and tests for both paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every Stream handler now resolves pending approvals at stream start and, when client-executed or approval-required tool calls are encountered, yields ToolApprovalRequestEvents and ends the stream with FinishReason::ToolCalls instead of looping — DeepSeek, Gemini, Groq, Mistral, Ollama, OpenRouter, Qwen, Requesty, Vertex, xAI, Z, and OpenAI ChatCompletions join Anthropic + OpenAI Responses. Completes the streaming item on fork issue #4. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rism-php#965) Covers both paths of the tools + response_format split: a tool call followed by the tool-less json_schema re-send, and the discard + re-send path when the model stops without calling the offered tools. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ls (prism-php#1004) Replaces the unrunnable test dropped in 5c08d92: this one ships a real two-response SSE fixture and consumes the stream before asserting the request payload omits the parameters key for a parameterless tool. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… and OpenRouter parameterless streaming Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…d Gemini These handlers reported cacheReadInputTokens while leaving the cached tokens inside promptTokens, so promptTokens + cacheReadInputTokens overstated input (double counting; upstream prism-php#1017). They now follow the convention already used by Anthropic, OpenAI (Responses) and DeepSeek: promptTokens holds the non-cached portion, guarded at zero. Gemini previously subtracted only when an explicit cachedContentName was set, missing implicit caching. Regression tests added for all three. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ders Extends the v0.108.1 usage convention (promptTokens = non-cached input) to every provider whose API reports prompt_tokens_details.cached_tokens: - OpenAI chat_completions path (Text/Structured/Stream) now has parity with the Responses path: cacheReadInputTokens populated, cached tokens excluded from promptTokens. Also applies to vLLM/LiteLLM-style backends used via api_format. - Azure, Groq, Qwen (native DashScope fields), xAI: cached-token visibility wired up (previously ignored entirely). - Z.AI and Requesty streams: fixed genuine double counting - they reported cacheReadInputTokens while leaving those tokens inside promptTokens. Their Text/Structured paths gain cache visibility. Regression tests for all seven providers (Text path), stream-level SSE fixture tests for Z.AI and Requesty, and the first Azure provider tests. Z.AI fixture-based expectations updated: the real captures contain large cached_tokens values, confirming the double counting in production responses. Removes a leftover unused providerOptions() read in the Gemini Text handler flagged by Rector. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…checker composer-require-checker surfaced undeclared direct dependencies, now declared: ext-mbstring, ext-openssl, psr/http-message, and symfony/http-foundation (all already installed transitively via laravel/framework, so this changes nothing for consumers). laravel/mcp moves to suggest for the optional LaravelMcpTool integration; its symbols plus the PHPUnit assertion used by the testing fakes are whitelisted in composer-require-checker.json. The custom ReorderMethodsRector was dev tooling shipping inside src/ (the source of phantom PhpParser/Rector symbols) - moved to dev/ under a Prism\Dev namespace, autoloaded only in autoload-dev and export-ignored from the dist package. New Require Checker CI workflow runs the tool on every push/PR. This is the proper guard for the goal upstream prism-php#1025 aimed at (catching use of undeclared symbols) without splitting laravel/framework into illuminate/* packages. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…penAI verbosity Three bug fixes from the upstream issue tracker plus one found in review: - DeepSeek and Qwen streaming handlers sent `stream: true` in the request body but never set the Guzzle `stream` transport option, so the full response body was buffered before the first event yielded — matching the other 14 providers now (upstream prism-php#990; Qwen found in review). - OpenRouter ToolCallMap eagerly ran json_decode() and passed the result to the ToolCall constructor, so malformed model JSON produced a raw TypeError (json_decode returns null) instead of a handled error. The map now stores the raw argument string; ToolCall::arguments() decodes lazily and wraps a decode failure in PrismException::malformedToolCallArguments(), which the tool-execution loop already converts into a tool result the model sees. The loop's error path is hardened to not re-throw while building that result (upstream prism-php#1006). Also adds OpenAI text_verbosity support to the structured (Responses) path and both chat/completions paths — it was previously only wired on the Responses text path (upstream prism-php#1015). Tests: OpenRouter ToolCallMap mapping + lazy-decode behavior, verbosity pass-through for structured + chat/completions, updated the ToolCall value object test to assert the handled PrismException. 1,891 passing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…stency Per upstream prism-php#1017: promptTokens is normalized to exclude cached tokens everywhere, but completionTokens is inclusive of reasoning tokens on OpenAI/Anthropic/OpenAI-compatible providers and exclusive on Gemini/Vertex. Documents the difference rather than changing billing-relevant numbers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Upstream has been quiet since March 2026 (v0.100.1), so particle-academy/prism — a drop-in fork,
Prism\Prismnamespace unchanged — has been absorbing the open backlog and shipping releases (context: discussion #1027). This PR offers all of that work back upstream in one piece: 166 commits, nine releases (v0.101.0–v0.109.0), gated throughout by Pest + PHPStan + Pint/Rector.If maintainership resumes, merge wholesale or tell us how you'd like it split — we're happy to break it into reviewable chunks. Either way the fork remains active.
Community PRs from this repo merged into the fork (48)
v0.101.0 — correctness fixes (17): #952, #958, #964, #971, #977, #985, #986, #987, #989, #991, #996, #1001, #1004, #1009, #1012, #1013, #1024
v0.103.0 — provider correctness / API drift (16): #949, #954, #961, #965, #975, #976, #980, #992, #993, #995, #997, #1000, #1002, #1008, #1020, #1021
v0.104.0 — features (9): #951 (batches + files APIs), #960 (xAI images), #978 (Vertex AI provider, answers #795), #988 (fine-grained tool streaming), #998 (Anthropic adaptive thinking), #1003 (pause_turn/refusal), #1014 (Mistral FIM), #1018 (provider-agnostic withReasoning()), #1026 (Requesty provider)
v0.105.0 — features + providers (6): #757 (Replicate provider), #810 (async STT interface), #835 (Azure OpenAI provider), #898 (Qwen provider), #907 (OpenAI chat/completions api_format + streaming citations, answers #900), #920 (cost tracking in Usage)
Reimplemented rather than rebased: #932 (client-executed tools + human-in-the-loop approval, answers #921) — clean-room implementation across all providers including streaming; docs at https://ai.particle.academy/docs/core-concepts/human-in-the-loop
Adjudicated, not merged (rationale posted): #950 (duplicate of #977), #937 and #1005 (superseded by an escape-based control-character fix), #999 (superseded), #1025 (rejected — a composer-require-checker CI gate solves the underlying goal properly; analysis in Particle-Academy/prism#3)
Fork-original changes
laravel/framework ^12.61.1|^13.12.0).Tool::requiresApproval(bool|Closure)/Tool::clientExecuted(), deny-by-default resume from message history, streaming approval events — text/structured/stream on all 18 providers.promptTokens= non-cached input everywhere;cacheReadInputTokenspopulated wherever the provider exposes it. Fixed silent double counting in Gemini, Vertex, OpenRouter (v0.108.1) and Z.AI + Requesty streams (v0.109.0); added cache visibility for OpenAI chat/completions, Azure, Groq, Qwen, xAI.anthropic_betaprovider option.src/.Full release notes: https://github.com/Particle-Academy/prism/releases
🤖 Generated with Claude Code