Skip to content

fix(genai): propagate the stop instance attribute to the model#1818

Open
obchain (obchain) wants to merge 1 commit into
langchain-ai:mainfrom
obchain:fix/genai-stop-attribute-propagation
Open

fix(genai): propagate the stop instance attribute to the model#1818
obchain (obchain) wants to merge 1 commit into
langchain-ai:mainfrom
obchain:fix/genai-stop-attribute-propagation

Conversation

@obchain

Copy link
Copy Markdown

Why

Fixes #1586. stop is a documented model parameter on ChatGoogleGenerativeAI, but setting it at construction time is silently ignored:

llm = ChatGoogleGenerativeAI(model="gemini-2.5-flash-lite", stop=["model"])

# Wrong: stop sequence not applied -> full output
llm.invoke("Repeat after me: 'I am a model from Google.'")

# Works: stop passed per-call
llm.invoke("Repeat after me: 'I am a model from Google.'", stop=["model"])

Root cause

_build_base_generation_config sets stop_sequences directly from the per-call stop argument:

"stop_sequences": stop,

Every other parameter in that config falls back to its instance attribute (kwargs.get("temperature", self.temperature), self.max_output_tokens, self.top_k, ...). stop is the only one that does not, so self.stop is never consulted.

What

  • Fall back to self.stop when no per-call stop is supplied:
    "stop_sequences": stop if stop is not None else self.stop,
  • Add test_instance_stop_is_propagated confirming a model-level stop reaches the generation config. The existing test_kwargs_override_stop already covers the per-call-override path, which remains unchanged.

Areas requiring careful review

  • Precedence is preserved: a per-call stop still overrides the instance attribute (the existing override test passes). The instance attribute now only applies when no per-call stop is given.

Disclaimer

AI agents were involved in drafting this contribution; the diagnosis, fix, and test were reviewed manually before submission.

`ChatGoogleGenerativeAI(stop=[...])` was ignored: `_build_base_generation_config`
set `stop_sequences` solely from the per-call `stop` argument, with no
fallback to `self.stop` (unlike `temperature`, `max_output_tokens`,
`top_k`, etc., which all fall back to their instance attributes). As a
result, stop sequences configured on the model only took effect when
also passed to `invoke(..., stop=[...])`.

Fall back to `self.stop` when no per-call `stop` is supplied.

Closes langchain-ai#1586

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a solid and clean fix for the stop sequences propagation bug (#1586).

Using stop if stop is not None else self.stop correctly handles the case where stop is explicitly passed as an empty list [] at call time to override an instance-level self.stop configuration (which evaluates to [] and bypasses the fallback, as intended).

The unit test is comprehensive and covers the exact regression path. Verified the diff and this looks great!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ChatGoogleGenerativeAI model attribute stop is not propagated to the model

2 participants