Skip to content

test(genai): make grounding_metadata assertions resilient to schema additions#1806

Open
obchain (obchain) wants to merge 1 commit into
langchain-ai:mainfrom
obchain:fix/genai-grounding-test-version-resilient
Open

test(genai): make grounding_metadata assertions resilient to schema additions#1806
obchain (obchain) wants to merge 1 commit into
langchain-ai:mainfrom
obchain:fix/genai-grounding-test-version-resilient

test(genai): make grounding_metadata assertions resilient to schema a…

cb8019e
Select commit
Loading
Failed to load commit list.
Google Cloud Build / langchain-google-genai-us (llm-integration-tests) failed May 26, 2026 in 19m 22s

Summary

Build Information

Trigger langchain-google-genai-us
Build fd0e45a2-12ba-4aa0-8e64-a400837b5fe9
Start 2026-05-26T01:20:07-07:00
Duration 19m21.018s
Status FAILURE

Steps

Step Status Duration
integration tests FAILURE 19m3.077s

Details


starting build "fd0e45a2-12ba-4aa0-8e64-a400837b5fe9"

FETCHSOURCE
From https://github.com/langchain-ai/langchain-google
 * branch            cb8019ea5aad85d6b2502419400b56fcbf9ee0c7 -> FETCH_HEAD
HEAD is now at cb8019e test(genai): make grounding_metadata assertions resilient to schema additions
GitCommit:
cb8019ea5aad85d6b2502419400b56fcbf9ee0c7
BUILD
Pulling image: python:3.11
3.11: Pulling from library/python
f32f49ce655a: Pulling fs layer
8a7504cd2818: Pulling fs layer
b53089dca505: Pulling fs layer
8d6d44b254da: Pulling fs layer
b3085a3e73e9: Pulling fs layer
3dfcdc764bf9: Pulling fs layer
bc5f9aa62a15: Pulling fs layer
bc5f9aa62a15: Waiting
b3085a3e73e9: Verifying Checksum
b3085a3e73e9: Download complete
bc5f9aa62a15: Verifying Checksum
bc5f9aa62a15: Download complete
3dfcdc764bf9: Download complete
8a7504cd2818: Verifying Checksum
8a7504cd2818: Download complete
f32f49ce655a: Verifying Checksum
f32f49ce655a: Download complete
b53089dca505: Verifying Checksum
b53089dca505: Download complete
8d6d44b254da: Verifying Checksum
8d6d44b254da: Download complete
f32f49ce655a: Pull complete
8a7504cd2818: Pull complete
b53089dca505: Pull complete
8d6d44b254da: Pull complete
b3085a3e73e9: Pull complete
3dfcdc764bf9: Pull complete
bc5f9aa62a15: Pull complete
Digest: sha256:58a54fbf1d5320af4f6761f9c6721329320a46ea086b5504640f19d1b7467d6a
Status: Downloaded newer image for python:3.11
docker.io/library/python:3.11
Using pip 24.0 from /usr/local/lib/python3.11/site-packages/pip (python 3.11)
Collecting uv==0.8.13
  Obtaining dependency information for uv==0.8.13 from https://files.pythonhosted.org/packages/82/6b/81387a715dd045f7edea452fb76a5896dcfc11b8ecf0db5106f4b0f633ec/uv-0.8.13-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading uv-0.8.13-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (11 kB)
Downloading uv-0.8.13-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (19.5 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.5/19.5 MB 82.5 MB/s eta 0:00:00
Installing collected packages: uv
Successfully installed uv-0.8.13
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 24.0 -> 26.1.1
[notice] To update, run: pip install --upgrade pip
Using CPython 3.11.15 interpreter at: /usr/local/bin/python3
Creating virtual environment at: .venv
Resolved 81 packages in 1ms
   Building langchain-google-genai @ file:///workspace/libs/genai
Downloading pygments (1.2MiB)
Downloading cryptography (4.3MiB)
Downloading zstandard (5.3MiB)
Downloading pydantic-core (2.0MiB)
Downloading numpy (16.1MiB)
 Downloading pydantic-core
 Downloading zstandard
 Downloading cryptography
 Downloading pygments
 Downloading numpy
      Built langchain-google-genai @ file:///workspace/libs/genai
Prepared 68 packages in 1.80s
warning: Failed to hardlink files; falling back to full copy. This may lead to degraded performance.
         If the cache and target directories are on different filesystems, hardlinking may not be supported.
         If this is intentional, set `export UV_LINK_MODE=copy` or use `--link-mode=copy` to suppress this warning.
Installed 68 packages in 301ms
 + annotated-types==0.7.0
 + anyio==4.12.1
 + certifi==2026.2.25
 + cffi==2.0.0
 + charset-normalizer==3.4.4
 + cryptography==46.0.7
 + distro==1.9.0
 + execnet==2.1.2
 + filetype==1.2.0
 + freezegun==1.5.5
 + google-auth==2.48.0
 + google-genai==1.65.0
 + h11==0.16.0
 + httpcore==1.0.9
 + httpx==0.28.1
 + idna==3.15
 + iniconfig==2.3.0
 + jsonpatch==1.33
 + jsonpointer==3.0.0
 + langchain-core==1.4.0
 + langchain-google-genai==4.2.3 (from file:///workspace/libs/genai)
 + langchain-protocol==0.0.15
 + langchain-tests==1.1.5
 + langsmith==0.8.0
 + markdown-it-py==4.0.0
 + mdurl==0.1.2
 + numpy==2.4.2
 + orjson==3.11.7
 + packaging==26.0
 + pluggy==1.6.0
 + protobuf==6.33.5
 + py-cpuinfo==9.0.0
 + pyasn1==0.6.3
 + pyasn1-modules==0.4.2
 + pycparser==3.0
 + pydantic==2.13.4
 + pydantic-core==2.46.4
 + pygments==2.20.0
 + pytest==9.0.3
 + pytest-asyncio==1.3.0
 + pytest-benchmark==5.2.3
 + pytest-codspeed==4.3.0
 + pytest-mock==3.15.1
 + pytest-recording==0.13.4
 + pytest-retry==1.7.0
 + pytest-socket==0.8.0
 + pytest-watcher==0.6.3
 + pytest-xdist==3.8.0
 + python-dateutil==2.9.0.post0
 + pyyaml==6.0.3
 + requests==2.33.0
 + requests-toolbelt==1.0.0
 + rich==14.3.3
 + rsa==4.9.1
 + six==1.17.0
 + sniffio==1.3.1
 + syrupy==5.2.0
 + tenacity==9.1.4
 + typing-extensions==4.15.0
 + typing-inspection==0.4.2
 + urllib3==2.7.0
 + uuid-utils==0.14.1
 + vcrpy==8.1.1
 + watchdog==6.0.0
 + websockets==16.0
 + wrapt==2.1.1
 + xxhash==3.6.0
 + zstandard==0.25.0
============================= test session starts ==============================
platform linux -- Python 3.11.15, pytest-9.0.3, pluggy-1.6.0
codspeed: 4.3.0 (disabled, mode: walltime, callgraph: not supported, timer_resolution: 1.0ns)
benchmark: 5.2.3 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /workspace/libs/genai
configfile: pyproject.toml
plugins: benchmark-5.2.3, asyncio-1.3.0, anyio-4.12.1, mock-3.15.1, recording-0.13.4, langsmith-0.8.0, xdist-3.8.0, retry-1.7.0, syrupy-5.2.0, codspeed-4.3.0, socket-0.8.0
asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 227 items

tests/integration_tests/test_callbacks.py .                              [  0%]
tests/integration_tests/test_chat_models.py ...............F............ [ 12%]
.....xX........F........R..F.FF...F.....................s.ss.            [ 39%]
tests/integration_tests/test_compile.py .                                [ 39%]
tests/integration_tests/test_embeddings.py ................              [ 46%]
tests/integration_tests/test_function_call.py ...                        [ 48%]
tests/integration_tests/test_llms.py ......                              [ 50%]
tests/integration_tests/test_standard.py .......ssss.................... [ 64%]
........ss...s.xx.......ssss...............................s...s.        [ 92%]
tests/integration_tests/test_structured_output_integration.py .......... [ 97%]
....                                                                     [ 99%]
tests/integration_tests/test_tools.py ..                                 [100%]

=================================== FAILURES ===================================
__________ test_chat_google_genai_invoke_thinking_disabled[google_ai] __________
Traceback (most recent call last):
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3166, in _generate
    response: GenerateContentResponse = self.client.models.generate_content(
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 5644, in generate_content
    response = self._generate_content(
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 4306, in _generate_content
    response = self._api_client.request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1401, in request
    response = self._request(http_request, http_options, stream=False)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1235, in _request
    return retry(self._request_once, http_request, stream)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 470, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 371, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 393, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
                                     ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 473, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1214, in _request_once
    errors.APIError.raise_for_response(response)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 134, in raise_for_response
    cls.raise_error(response.status_code, response_json, response)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 159, in raise_error
    raise ClientError(status_code, response_json, response)
google.genai.errors.ClientError: 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 353, in from_call
    result: TResult | None = func()
                             ^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 245, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/logging.py", line 850, in pytest_runtest_call
    yield
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 179, in pytest_runtest_call
    item.runtest()
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py", line 1720, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py", line 166, in pytest_pyfunc_call
    result = testfunction(**testargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/tests/integration_tests/test_chat_models.py", line 650, in test_chat_google_genai_invoke_thinking_disabled
    result = llm.invoke(
             ^^^^^^^^^^^
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 2644, in invoke
    return super().invoke(input, config, stop=stop, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 474, in invoke
    self.generate_prompt(
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1823, in generate_prompt
    return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1630, in generate
    self._generate_with_cache(
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1970, in _generate_with_cache
    result = self._generate(
             ^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3170, in _generate
    _handle_client_error(e, request)
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 169, in _handle_client_error
    raise ChatGoogleGenerativeAIError(msg) from e
langchain_google_genai.chat_models.ChatGoogleGenerativeAIError: Error calling model 'gemini-2.5-flash' (NOT_FOUND): 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}
_______ test_thinking_params_preserved_with_structured_output[google_ai] _______
Traceback (most recent call last):
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3166, in _generate
    response: GenerateContentResponse = self.client.models.generate_content(
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 5644, in generate_content
    response = self._generate_content(
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 4306, in _generate_content
    response = self._api_client.request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1401, in request
    response = self._request(http_request, http_options, stream=False)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1235, in _request
    return retry(self._request_once, http_request, stream)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 470, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 371, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 393, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
                                     ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 473, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1214, in _request_once
    errors.APIError.raise_for_response(response)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 134, in raise_for_response
    cls.raise_error(response.status_code, response_json, response)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 159, in raise_error
    raise ClientError(status_code, response_json, response)
google.genai.errors.ClientError: 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 353, in from_call
    result: TResult | None = func()
                             ^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 245, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/logging.py", line 850, in pytest_runtest_call
    yield
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 179, in pytest_runtest_call
    item.runtest()
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py", line 1720, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py", line 166, in pytest_pyfunc_call
    result = testfunction(**testargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/tests/integration_tests/test_chat_models.py", line 1482, in test_thinking_params_preserved_with_structured_output
    result = structured_llm.invoke("What is 2+2? Just give a brief answer.")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 3333, in invoke
    input_ = context.run(step.invoke, input_, config, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 5881, in invoke
    return self.bound.invoke(
           ^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 2644, in invoke
    return super().invoke(input, config, stop=stop, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 474, in invoke
    self.generate_prompt(
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1823, in generate_prompt
    return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1630, in generate
    self._generate_with_cache(
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1970, in _generate_with_cache
    result = self._generate(
             ^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3170, in _generate
    _handle_client_error(e, request)
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 169, in _handle_client_error
    raise ChatGoogleGenerativeAIError(msg) from e
langchain_google_genai.chat_models.ChatGoogleGenerativeAIError: Error calling model 'gemini-2.5-flash' (NOT_FOUND): 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}
____________________ test_search_with_googletool[google_ai] ____________________
Traceback (most recent call last):
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3166, in _generate
    response: GenerateContentResponse = self.client.models.generate_content(
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 5644, in generate_content
    response = self._generate_content(
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 4306, in _generate_content
    response = self._api_client.request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1401, in request
    response = self._request(http_request, http_options, stream=False)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1235, in _request
    return retry(self._request_once, http_request, stream)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 470, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 371, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 393, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
                                     ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 473, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1214, in _request_once
    errors.APIError.raise_for_response(response)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 134, in raise_for_response
    cls.raise_error(response.status_code, response_json, response)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 159, in raise_error
    raise ClientError(status_code, response_json, response)
google.genai.errors.ClientError: 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 353, in from_call
    result: TResult | None = func()
                             ^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 245, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^
...
[Logs truncated due to log size limitations. For full logs, see https://console.cloud.google.com/logs/viewer?advancedFilter=resource.type%3D%22build%22+AND+resource.labels.build_id%3D%22fd0e45a2-12ba-4aa0-8e64-a400837b5fe9%22+AND+resource.labels.build_trigger_id%3D%22cf0bf1b4-59b2-4ad2-9f4c-4866728a37b1%22&project=llm-integration-tests.]
...
e_ai] _____________________
Traceback (most recent call last):
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3166, in _generate
    response: GenerateContentResponse = self.client.models.generate_content(
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 5644, in generate_content
    response = self._generate_content(
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 4306, in _generate_content
    response = self._api_client.request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1401, in request
    response = self._request(http_request, http_options, stream=False)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1235, in _request
    return retry(self._request_once, http_request, stream)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 470, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 371, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 393, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
                                     ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 473, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1214, in _request_once
    errors.APIError.raise_for_response(response)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 134, in raise_for_response
    cls.raise_error(response.status_code, response_json, response)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 159, in raise_error
    raise ClientError(status_code, response_json, response)
google.genai.errors.ClientError: 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 353, in from_call
    result: TResult | None = func()
                             ^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 245, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/logging.py", line 850, in pytest_runtest_call
    yield
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 179, in pytest_runtest_call
    item.runtest()
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py", line 1720, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py", line 166, in pytest_pyfunc_call
    result = testfunction(**testargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/tests/integration_tests/test_chat_models.py", line 1818, in test_google_maps_grounding
    response = model_with_maps.invoke(
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 5881, in invoke
    return self.bound.invoke(
           ^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 2644, in invoke
    return super().invoke(input, config, stop=stop, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 474, in invoke
    self.generate_prompt(
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1823, in generate_prompt
    return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1630, in generate
    self._generate_with_cache(
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1970, in _generate_with_cache
    result = self._generate(
             ^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3170, in _generate
    _handle_client_error(e, request)
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 169, in _handle_client_error
    raise ChatGoogleGenerativeAIError(msg) from e
langchain_google_genai.chat_models.ChatGoogleGenerativeAIError: Error calling model 'gemini-2.5-flash' (NOT_FOUND): 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}
_____________ test_google_maps_grounding_invoke_direct[google_ai] ______________
Traceback (most recent call last):
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3166, in _generate
    response: GenerateContentResponse = self.client.models.generate_content(
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 5644, in generate_content
    response = self._generate_content(
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 4306, in _generate_content
    response = self._api_client.request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1401, in request
    response = self._request(http_request, http_options, stream=False)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1235, in _request
    return retry(self._request_once, http_request, stream)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 470, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 371, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 393, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
                                     ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 473, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1214, in _request_once
    errors.APIError.raise_for_response(response)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 134, in raise_for_response
    cls.raise_error(response.status_code, response_json, response)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 159, in raise_error
    raise ClientError(status_code, response_json, response)
google.genai.errors.ClientError: 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 353, in from_call
    result: TResult | None = func()
                             ^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 245, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/logging.py", line 850, in pytest_runtest_call
    yield
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 179, in pytest_runtest_call
    item.runtest()
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py", line 1720, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py", line 166, in pytest_pyfunc_call
    result = testfunction(**testargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/tests/integration_tests/test_chat_models.py", line 1882, in test_google_maps_grounding_invoke_direct
    response = model.invoke(
               ^^^^^^^^^^^^^
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 2644, in invoke
    return super().invoke(input, config, stop=stop, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 474, in invoke
    self.generate_prompt(
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1823, in generate_prompt
    return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1630, in generate
    self._generate_with_cache(
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1970, in _generate_with_cache
    result = self._generate(
             ^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3170, in _generate
    _handle_client_error(e, request)
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 169, in _handle_client_error
    raise ChatGoogleGenerativeAIError(msg) from e
langchain_google_genai.chat_models.ChatGoogleGenerativeAIError: Error calling model 'gemini-2.5-flash' (NOT_FOUND): 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}
_______ test_chat_google_genai_invoke_with_generation_params[google_ai] ________
Traceback (most recent call last):
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3166, in _generate
    response: GenerateContentResponse = self.client.models.generate_content(
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 5644, in generate_content
    response = self._generate_content(
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 4306, in _generate_content
    response = self._api_client.request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1401, in request
    response = self._request(http_request, http_options, stream=False)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1235, in _request
    return retry(self._request_once, http_request, stream)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 470, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 371, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 393, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
                                     ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 473, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1214, in _request_once
    errors.APIError.raise_for_response(response)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 134, in raise_for_response
    cls.raise_error(response.status_code, response_json, response)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 159, in raise_error
    raise ClientError(status_code, response_json, response)
google.genai.errors.ClientError: 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 353, in from_call
    result: TResult | None = func()
                             ^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 245, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/logging.py", line 850, in pytest_runtest_call
    yield
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 179, in pytest_runtest_call
    item.runtest()
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py", line 1720, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py", line 166, in pytest_pyfunc_call
    result = testfunction(**testargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/tests/integration_tests/test_chat_models.py", line 2001, in test_chat_google_genai_invoke_with_generation_params
    result_constrained = llm.invoke(
                         ^^^^^^^^^^^
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 2644, in invoke
    return super().invoke(input, config, stop=stop, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 474, in invoke
    self.generate_prompt(
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1823, in generate_prompt
    return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1630, in generate
    self._generate_with_cache(
  File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1970, in _generate_with_cache
    result = self._generate(
             ^^^^^^^^^^^^^^^
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3170, in _generate
    _handle_client_error(e, request)
  File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 169, in _handle_client_error
    raise ChatGoogleGenerativeAIError(msg) from e
langchain_google_genai.chat_models.ChatGoogleGenerativeAIError: Error calling model 'gemini-2.5-flash' (NOT_FOUND): 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}
=============================== warnings summary ===============================
tests/integration_tests/test_chat_models.py::test_logprobs[google_ai]
tests/integration_tests/test_chat_models.py::test_logprobs_with_json_schema[google_ai]
  /workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py:166: UserWarning: WARNING! logprobs is not default parameter.
                  logprobs was transferred to model_kwargs.
                  Please confirm that logprobs is what you intended.
    result = testfunction(**testargs)

tests/integration_tests/test_standard.py::TestGeminiProStandardGoogleAI::test_audio_inputs
  /workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_tests/integration_tests/chat_models.py:2684: UserWarning: LANGCHAIN_TESTS_USER_AGENT environment variable not set. langchain-tests pulls (CC0 License) audio data from wikimedia.org. Consider setting a user agent to identify your requests. See https://foundation.wikimedia.org/wiki/Policy:Wikimedia_Foundation_User-Agent_Policy
    audio_data = _get_base64_from_url(url)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=================================== XPASSES ====================================
__________________ test_logprobs_with_json_schema[google_ai] ___________________

Build Log: https://console.cloud.google.com/logs/viewer?advancedFilter=resource.type%3D%22build%22+AND+resource.labels.build_id%3D%22fd0e45a2-12ba-4aa0-8e64-a400837b5fe9%22+AND+resource.labels.build_trigger_id%3D%22cf0bf1b4-59b2-4ad2-9f4c-4866728a37b1%22&project=llm-integration-tests