Stream example rendering to avoid OOM for large specs#24160
Conversation
There was a problem hiding this comment.
20 issues found and verified against the latest diff
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="modules/openapi-generator/src/main/resources/Java/libraries/jersey3/pojo.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/Java/libraries/jersey3/pojo.mustache:209">
P2: Example annotations now depend on a new `hasExample` flag; if it is not populated everywhere `example` used to be, generated Jersey3 POJOs will lose `example = ...` metadata.</violation>
</file>
<file name="modules/openapi-generator/src/main/resources/htmlDocs/index.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/htmlDocs/index.mustache:141">
P1: Response example rendering missing HTML escaping after switching to lambdaExample; add {{#lambdaEscapeHtml}} wrapper to prevent XSS in generated HTML docs</violation>
</file>
<file name="modules/openapi-generator/src/main/java/org/openapitools/codegen/templating/mustache/EscapeHtmlLambda.java">
<violation number="1" location="modules/openapi-generator/src/main/java/org/openapitools/codegen/templating/mustache/EscapeHtmlLambda.java:60">
P2: Buffers the full fragment before escaping, undermining the streaming/OOM mitigation.</violation>
</file>
<file name="modules/openapi-generator/src/main/resources/kotlin-spring/methodBody.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/kotlin-spring/methodBody.mustache:8">
P1: The kotlin-spring generator's `lambdaEscapeInNormalString` and `lambdaRemoveLineBreak` inline lambdas call `fragment.execute()`, which fully buffers the rendered example into a `String`. This defeats the streaming `lambdaExample` introduced to avoid OOM for large specs.</violation>
</file>
<file name="modules/openapi-generator/src/main/resources/csharp-functions/api_doc.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/csharp-functions/api_doc.mustache:75">
P1: C# Functions doc template relies on `lambdaExample` for primitive parameter examples, but if the generator does not provide this lambda the Mustache section renders empty, producing invalid C# (`var x = ;`). The generator contract should be verified to ensure `lambdaExample` is always present for primitive `allParams` contexts in the C# Functions generator.</violation>
</file>
<file name="modules/openapi-generator/src/main/java/org/openapitools/codegen/TemplateManager.java">
<violation number="1" location="modules/openapi-generator/src/main/java/org/openapitools/codegen/TemplateManager.java:360">
P2: Streaming file comparison incorrectly assumes matching chunk sizes from InputStream.read(byte[]), which can falsely report equal files as different and cause minimal-update mode to rewrite unchanged files.</violation>
</file>
<file name="modules/openapi-generator/src/main/java/org/openapitools/codegen/CodegenParameter.java">
<violation number="1" location="modules/openapi-generator/src/main/java/org/openapitools/codegen/CodegenParameter.java:47">
P1: equals()/hashCode() do not account for new exampleJsonNode field</violation>
</file>
<file name="modules/openapi-generator/src/main/resources/graphql-nodejs-express-server/resolvers.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/graphql-nodejs-express-server/resolvers.mustache:13">
P1: lambdaExample output is inserted into a quoted JavaScript string without JS escaping, which can produce invalid resolver code for examples containing quotes, backslashes, newlines, or JSON literals.</violation>
</file>
<file name="modules/openapi-generator/src/main/resources/csharp-functions/controller.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/csharp-functions/controller.mustache:76">
P1: Streamed example JSON is missing backslash escaping for the C# string literal, which causes the C# compiler to reinterpret JSON escape sequences (e.g., `\\d`, `\\n`, `\\uXXXX`) as C# escapes. This leads to compilation errors or semantically different JSON in generated code.</violation>
</file>
<file name="modules/openapi-generator/src/main/resources/aspnetcore/2.1/controller.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/aspnetcore/2.1/controller.mustache:76">
P1: Missing backslash escaping in C# string literal when streaming example JSON via lambdas.</violation>
</file>
<file name="modules/openapi-generator/src/main/resources/aspnetcore/2.0/controller.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/aspnetcore/2.0/controller.mustache:52">
P1: Streamed examples may not escape backslashes before embedding raw JSON in a C# string literal. The template only wraps `lambdaExample` with `lambdaEscapeNewLine` and `lambdaEscapeDoubleQuote`, but C# string literals require backslashes to be escaped too (e.g., `\\`). JSON backslash escapes like `\\d+`, `\\/`, or `\\uXXXX` can produce invalid C# escape sequences or corrupted runtime strings. Either add a backslash-escaping lambda or switch to a verbatim string literal (`@"..."`) with appropriate quote doubling.</violation>
</file>
<file name="modules/openapi-generator/src/main/resources/fsharp-giraffe-server/HandlerTestsHelper.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/fsharp-giraffe-server/HandlerTestsHelper.mustache:34">
P1: Staled `WebUtility.HtmlDecode` call retained after switching from Mustache `{{example}}` interpolation (which HTML-escapes by default) to writer-based lambdas that bypass Mustache escaping. Any literal HTML/XML entities in the example, such as `&` or `<`, will be incorrectly decoded at runtime, corrupting the example content and potentially breaking XML test payloads.</violation>
</file>
<file name="modules/openapi-generator/src/main/java/org/openapitools/codegen/templating/mustache/UniqueLambda.java">
<violation number="1" location="modules/openapi-generator/src/main/java/org/openapitools/codegen/templating/mustache/UniqueLambda.java:97">
P1: Delimiter processing in `writeCompleteValues()` degrades toward O(N²) for large buffers because `StringBuilder.delete(0, n)` shifts all remaining characters on every loop iteration. In a PR focused on streaming large specs to avoid memory pressure, quadratic buffer copying negates the streaming benefit for content with many repeated delimiter-separated values.</violation>
</file>
Note: This PR contains a large number of files. cubic only reviews up to 200 files per PR, so some files may not have been reviewed. cubic prioritizes the most important files to review.
Re-trigger cubic
There was a problem hiding this comment.
1 issue found across 38 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="modules/openapi-generator/src/main/resources/Java/libraries/jersey3/pojo.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/Java/libraries/jersey3/pojo.mustache:209">
P2: Example annotations now depend on a new `hasExample` flag; if it is not populated everywhere `example` used to be, generated Jersey3 POJOs will lose `example = ...` metadata.</violation>
</file>
<file name="modules/openapi-generator/src/main/resources/htmlDocs/index.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/htmlDocs/index.mustache:141">
P1: Response example rendering missing HTML escaping after switching to lambdaExample; add {{#lambdaEscapeHtml}} wrapper to prevent XSS in generated HTML docs</violation>
</file>
<file name="modules/openapi-generator/src/main/java/org/openapitools/codegen/templating/mustache/EscapeHtmlLambda.java">
<violation number="1" location="modules/openapi-generator/src/main/java/org/openapitools/codegen/templating/mustache/EscapeHtmlLambda.java:60">
P2: Buffers the full fragment before escaping, undermining the streaming/OOM mitigation.</violation>
</file>
<file name="modules/openapi-generator/src/main/resources/kotlin-spring/methodBody.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/kotlin-spring/methodBody.mustache:8">
P1: The kotlin-spring generator's `lambdaEscapeInNormalString` and `lambdaRemoveLineBreak` inline lambdas call `fragment.execute()`, which fully buffers the rendered example into a `String`. This defeats the streaming `lambdaExample` introduced to avoid OOM for large specs.</violation>
</file>
<file name="modules/openapi-generator/src/main/java/org/openapitools/codegen/TemplateManager.java">
<violation number="1" location="modules/openapi-generator/src/main/java/org/openapitools/codegen/TemplateManager.java:270">
P2: Non-minimal template writes now use a predictable `<target>.tmp` path that can overwrite, move, or delete an existing sibling file. Concurrent generator runs targeting the same output directory could race on this deterministic temp file, and any legitimate `.tmp` sidecar files will be silently clobbered. Consider using a unique temp file name (e.g., via `Files.createTempFile` in the output directory or a nonce suffix) instead of a deterministic `filename + ".tmp"`.</violation>
<violation number="2" location="modules/openapi-generator/src/main/java/org/openapitools/codegen/TemplateManager.java:360">
P2: Streaming file comparison incorrectly assumes matching chunk sizes from InputStream.read(byte[]), which can falsely report equal files as different and cause minimal-update mode to rewrite unchanged files.</violation>
</file>
<file name="modules/openapi-generator/src/main/resources/csharp-functions/controller.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/csharp-functions/controller.mustache:76">
P1: Streamed example JSON is missing backslash escaping for the C# string literal, which causes the C# compiler to reinterpret JSON escape sequences (e.g., `\\d`, `\\n`, `\\uXXXX`) as C# escapes. This leads to compilation errors or semantically different JSON in generated code.</violation>
</file>
<file name="modules/openapi-generator/src/main/resources/aspnetcore/2.1/controller.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/aspnetcore/2.1/controller.mustache:76">
P1: Missing backslash escaping in C# string literal when streaming example JSON via lambdas.</violation>
</file>
<file name="modules/openapi-generator/src/main/resources/aspnetcore/2.0/controller.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/aspnetcore/2.0/controller.mustache:52">
P1: Streamed examples may not escape backslashes before embedding raw JSON in a C# string literal. The template only wraps `lambdaExample` with `lambdaEscapeNewLine` and `lambdaEscapeDoubleQuote`, but C# string literals require backslashes to be escaped too (e.g., `\\`). JSON backslash escapes like `\\d+`, `\\/`, or `\\uXXXX` can produce invalid C# escape sequences or corrupted runtime strings. Either add a backslash-escaping lambda or switch to a verbatim string literal (`@"..."`) with appropriate quote doubling.</violation>
</file>
<file name="modules/openapi-generator/src/main/resources/fsharp-giraffe-server/HandlerTestsHelper.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/fsharp-giraffe-server/HandlerTestsHelper.mustache:34">
P1: Staled `WebUtility.HtmlDecode` call retained after switching from Mustache `{{example}}` interpolation (which HTML-escapes by default) to writer-based lambdas that bypass Mustache escaping. Any literal HTML/XML entities in the example, such as `&` or `<`, will be incorrectly decoded at runtime, corrupting the example content and potentially breaking XML test payloads.</violation>
</file>
<file name="modules/openapi-generator/src/main/java/org/openapitools/codegen/templating/mustache/UniqueLambda.java">
<violation number="1" location="modules/openapi-generator/src/main/java/org/openapitools/codegen/templating/mustache/UniqueLambda.java:97">
P1: Delimiter processing in `writeCompleteValues()` degrades toward O(N²) for large buffers because `StringBuilder.delete(0, n)` shifts all remaining characters on every loop iteration. In a PR focused on streaming large specs to avoid memory pressure, quadratic buffer copying negates the streaming benefit for content with many repeated delimiter-separated values.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
There was a problem hiding this comment.
1 issue found across 10 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="modules/openapi-generator/src/main/resources/Java/libraries/jersey3/pojo.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/Java/libraries/jersey3/pojo.mustache:209">
P2: Example annotations now depend on a new `hasExample` flag; if it is not populated everywhere `example` used to be, generated Jersey3 POJOs will lose `example = ...` metadata.</violation>
</file>
<file name="modules/openapi-generator/src/main/resources/htmlDocs/index.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/htmlDocs/index.mustache:141">
P1: Response example rendering missing HTML escaping after switching to lambdaExample; add {{#lambdaEscapeHtml}} wrapper to prevent XSS in generated HTML docs</violation>
</file>
<file name="modules/openapi-generator/src/main/resources/kotlin-spring/methodBody.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/kotlin-spring/methodBody.mustache:8">
P1: The kotlin-spring generator's `lambdaEscapeInNormalString` and `lambdaRemoveLineBreak` inline lambdas call `fragment.execute()`, which fully buffers the rendered example into a `String`. This defeats the streaming `lambdaExample` introduced to avoid OOM for large specs.</violation>
</file>
<file name="modules/openapi-generator/src/main/java/org/openapitools/codegen/CodegenParameter.java">
<violation number="1" location="modules/openapi-generator/src/main/java/org/openapitools/codegen/CodegenParameter.java:298">
P2: Including `exampleJsonNode` in `hashCode()` and `equals()` can cause deep recursive traversal of large JSON trees, undermining the PR's goal of avoiding OOM/performance issues with large examples. The analogous `CodegenProperty` class intentionally omits `exampleJsonNode` from its `hashCode`/`equals`.</violation>
</file>
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
There was a problem hiding this comment.
1 issue found across 13 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="modules/openapi-generator/src/main/resources/Java/libraries/jersey3/pojo.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/Java/libraries/jersey3/pojo.mustache:209">
P2: Example annotations now depend on a new `hasExample` flag; if it is not populated everywhere `example` used to be, generated Jersey3 POJOs will lose `example = ...` metadata.</violation>
</file>
<file name="modules/openapi-generator/src/main/resources/kotlin-spring/methodBody.mustache">
<violation number="1" location="modules/openapi-generator/src/main/resources/kotlin-spring/methodBody.mustache:8">
P1: The kotlin-spring generator's `lambdaEscapeInNormalString` and `lambdaRemoveLineBreak` inline lambdas call `fragment.execute()`, which fully buffers the rendered example into a `String`. This defeats the streaming `lambdaExample` introduced to avoid OOM for large specs.</violation>
</file>
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
There was a problem hiding this comment.
7 issues found across 327 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="modules/openapi-generator/src/main/java/org/openapitools/codegen/CodegenParameter.java">
<violation number="1" location="modules/openapi-generator/src/main/java/org/openapitools/codegen/CodegenParameter.java:47">
P2: New private `exampleJsonNode` can become inconsistent with the existing public `example` field because direct field assignments bypass the synchronization in `setExample`.</violation>
</file>
<file name="modules/openapi-generator/src/main/java/org/openapitools/codegen/templating/mustache/EscapeHtmlLambda.java">
<violation number="1" location="modules/openapi-generator/src/main/java/org/openapitools/codegen/templating/mustache/EscapeHtmlLambda.java:61">
P2: The HTML escaping lambda uses ESCAPE_HTML4 which does not escape apostrophes/single quotes, making it unsafe for single-quoted HTML attribute contexts.</violation>
</file>
Note: This PR contains a large number of files. cubic only reviews up to 200 files per PR, so some files may not have been reviewed. cubic prioritizes the most important files to review.
Re-trigger cubic
| public String nameInPascalCase; // property name in pascal case (e.g. ModifiedDate) | ||
| public String nameInSnakeCase; // property name in upper snake case | ||
| public String example; // example value (x-example) | ||
| private JsonNode exampleJsonNode; |
There was a problem hiding this comment.
P2: New private exampleJsonNode can become inconsistent with the existing public example field because direct field assignments bypass the synchronization in setExample.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At modules/openapi-generator/src/main/java/org/openapitools/codegen/CodegenParameter.java, line 47:
<comment>New private `exampleJsonNode` can become inconsistent with the existing public `example` field because direct field assignments bypass the synchronization in `setExample`.</comment>
<file context>
@@ -41,6 +44,7 @@ public class CodegenParameter implements IJsonSchemaValidationProperties {
public String nameInPascalCase; // property name in pascal case (e.g. ModifiedDate)
public String nameInSnakeCase; // property name in upper snake case
public String example; // example value (x-example)
+ private JsonNode exampleJsonNode;
public Map<String, Example> examples;
public String jsonSchema;
</file context>
| int end = off + len; | ||
|
|
||
| if (pendingHighSurrogate != 0 && start < end) { | ||
| StringEscapeUtils.ESCAPE_HTML4.translate(new String(new char[]{pendingHighSurrogate, cbuf[start]}), writer); |
There was a problem hiding this comment.
P2: The HTML escaping lambda uses ESCAPE_HTML4 which does not escape apostrophes/single quotes, making it unsafe for single-quoted HTML attribute contexts.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At modules/openapi-generator/src/main/java/org/openapitools/codegen/templating/mustache/EscapeHtmlLambda.java, line 61:
<comment>The HTML escaping lambda uses ESCAPE_HTML4 which does not escape apostrophes/single quotes, making it unsafe for single-quoted HTML attribute contexts.</comment>
<file context>
@@ -0,0 +1,83 @@
+ int end = off + len;
+
+ if (pendingHighSurrogate != 0 && start < end) {
+ StringEscapeUtils.ESCAPE_HTML4.translate(new String(new char[]{pendingHighSurrogate, cbuf[start]}), writer);
+ pendingHighSurrogate = 0;
+ start++;
</file context>
There was a problem hiding this comment.
This is used only in contexts where there is just tags content, not attributes of tags.
Avoid materializing huge generated examples, model JSON, rendered templates, and OpenAPI spec supporting files as single in-memory strings. Large specs such as the issue OpenAPITools#23849 reproduction can exceed Jackson/SnakeYAML String/TextBuffer limits or exhaust heap when examples and generated source are buffered before writing. Introduce writer-backed Mustache lambdas for JSON/OpenAPI output, add a streaming template render path, and update templates to invoke lambdas instead of direct example/spec string values.
Add streaming-safe escaping lambdas for newline and HTML-sensitive output, and snapshot model JSON before later example generation mutates schemas. Update JetBrains HTTP client request examples to render per media type, omit duplicate headings and value-less examples, and refresh generated samples. Update Rust server example generation and generated sample output for streamed JSON examples.
Keep exampleJsonNode in CodegenParameter.equals so object examples remain part of parameter equality, but omit it from hashCode to avoid deep traversal of large JSON example trees during hashing.
…en/TemplateManager.java Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
Compare exampleJsonNode in CodegenProperty.equals so object examples remain part of property equality. Keep it out of hashCode to avoid deep traversal of large JSON example trees during hashing.
Use a JSON-literal example lambda in GraphQL resolvers so string examples remain escaped strings while numbers, booleans, arrays, and objects are emitted as their native JavaScript literal types.
Escape streamed request body examples with the JSON string escaping lambda after removing line breaks, so generated F# regular string literals preserve quotes, backslashes, and control characters without reintroducing HTML decoding.
Replace direct CodegenParameter.example assignments with setExample calls so generator-specific example rewrites clear or synchronize the backing exampleJsonNode state instead of leaving stale streamed examples behind.
Show raw Mustache interpolation in the EscapeHtmlLambda Javadoc example so the documented usage does not double-escape content before the lambda runs.
Clarify that EscapeHtmlLambda is intended for HTML element text content such as code blocks, not HTML attribute values where escaping depends on the attribute quoting style.
Render JSON value-node examples through lambdaExample as legacy text while keeping object and array examples streamed as JSON. This avoids adding JSON string quotes inside annotation string attributes.
Compare streamed JSON output structurally in tests instead of asserting exact pretty-printer line endings, which differ between Unix and Windows builds.
Fixes #23849.
This PR fixes the OOM/memory pressure reported in #23849 by changing example and OpenAPI spec rendering to stream through the templating engine instead of materializing large rendered strings up front.
This is likely a major-version change. The template-facing example rendering contract changes from using pre-rendered string values such as
example,modelJson,openapi-json, andopenapi-yamlin many places to using writer-backed lambdas such aslambdaExample,lambdaModelJson,lambdaOpenapiJson, andlambdaOpenapiYaml. Because custom templates may reference the old values directly, this can break existing custom templates and should be treated as a backwards-incompatible template API change.The main change is in the core templating/example rendering path:
This also includes related fixes found while applying the streaming path across generators, including JetBrains
.httprequest example rendering/content-type handling and Java generator example rendering updates.Samples, docs, and site output have been regenerated.
All tests were run.
PR checklist
Summary by cubic
Streams example and OpenAPI spec rendering end-to-end to prevent OOM on large specs by writing through Mustache writer APIs directly to files. Keeps scalar examples as plain text in annotations while streaming object/array examples as JSON.
Bug Fixes
writeTemplateto the templating engine, atomic writes via unique temp files, and minimal-update file comparisons.lambdaExample,lambdaModelJson,lambdaOpenapiJson,lambdaOpenapiYaml; updated templates across generators (incl. annotations) and rendered GraphQL examples as typed literals.escapeJavaStringandescapeJsonString.hashCode..httprequests: per-media-type examples, content-type filtering, deduplication, and omission of value-less examples.Migration
{{{example}}},{{{modelJson}}},{{{openapi-json}}},{{{openapi-yaml}}}with{{#lambdaExample}}{{/lambdaExample}},{{#lambdaModelJson}}{{/lambdaModelJson}},{{#lambdaOpenapiJson}}{{/lambdaOpenapiJson}},{{#lambdaOpenapiYaml}}{{/lambdaOpenapiYaml}}.Written for commit 39145da. Summary will update on new commits.