test(genai): make grounding_metadata assertions resilient to schema additions#1806
Open
obchain (obchain) wants to merge 1 commit into
Open
Google Cloud Build / langchain-google-genai-us (llm-integration-tests)
failed
May 26, 2026 in 19m 22s
Summary
Build Information
| Trigger | langchain-google-genai-us |
| Build | fd0e45a2-12ba-4aa0-8e64-a400837b5fe9 |
| Start | 2026-05-26T01:20:07-07:00 |
| Duration | 19m21.018s |
| Status | FAILURE |
Steps
| Step | Status | Duration |
|---|---|---|
| integration tests | FAILURE | 19m3.077s |
Details
starting build "fd0e45a2-12ba-4aa0-8e64-a400837b5fe9"
FETCHSOURCE
From https://github.com/langchain-ai/langchain-google
* branch cb8019ea5aad85d6b2502419400b56fcbf9ee0c7 -> FETCH_HEAD
HEAD is now at cb8019e test(genai): make grounding_metadata assertions resilient to schema additions
GitCommit:
cb8019ea5aad85d6b2502419400b56fcbf9ee0c7
BUILD
Pulling image: python:3.11
3.11: Pulling from library/python
f32f49ce655a: Pulling fs layer
8a7504cd2818: Pulling fs layer
b53089dca505: Pulling fs layer
8d6d44b254da: Pulling fs layer
b3085a3e73e9: Pulling fs layer
3dfcdc764bf9: Pulling fs layer
bc5f9aa62a15: Pulling fs layer
bc5f9aa62a15: Waiting
b3085a3e73e9: Verifying Checksum
b3085a3e73e9: Download complete
bc5f9aa62a15: Verifying Checksum
bc5f9aa62a15: Download complete
3dfcdc764bf9: Download complete
8a7504cd2818: Verifying Checksum
8a7504cd2818: Download complete
f32f49ce655a: Verifying Checksum
f32f49ce655a: Download complete
b53089dca505: Verifying Checksum
b53089dca505: Download complete
8d6d44b254da: Verifying Checksum
8d6d44b254da: Download complete
f32f49ce655a: Pull complete
8a7504cd2818: Pull complete
b53089dca505: Pull complete
8d6d44b254da: Pull complete
b3085a3e73e9: Pull complete
3dfcdc764bf9: Pull complete
bc5f9aa62a15: Pull complete
Digest: sha256:58a54fbf1d5320af4f6761f9c6721329320a46ea086b5504640f19d1b7467d6a
Status: Downloaded newer image for python:3.11
docker.io/library/python:3.11
Using pip 24.0 from /usr/local/lib/python3.11/site-packages/pip (python 3.11)
Collecting uv==0.8.13
Obtaining dependency information for uv==0.8.13 from https://files.pythonhosted.org/packages/82/6b/81387a715dd045f7edea452fb76a5896dcfc11b8ecf0db5106f4b0f633ec/uv-0.8.13-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Downloading uv-0.8.13-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (11 kB)
Downloading uv-0.8.13-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (19.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.5/19.5 MB 82.5 MB/s eta 0:00:00
Installing collected packages: uv
Successfully installed uv-0.8.13
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip is available: 24.0 -> 26.1.1
[notice] To update, run: pip install --upgrade pip
Using CPython 3.11.15 interpreter at: /usr/local/bin/python3
Creating virtual environment at: .venv
Resolved 81 packages in 1ms
Building langchain-google-genai @ file:///workspace/libs/genai
Downloading pygments (1.2MiB)
Downloading cryptography (4.3MiB)
Downloading zstandard (5.3MiB)
Downloading pydantic-core (2.0MiB)
Downloading numpy (16.1MiB)
Downloading pydantic-core
Downloading zstandard
Downloading cryptography
Downloading pygments
Downloading numpy
Built langchain-google-genai @ file:///workspace/libs/genai
Prepared 68 packages in 1.80s
warning: Failed to hardlink files; falling back to full copy. This may lead to degraded performance.
If the cache and target directories are on different filesystems, hardlinking may not be supported.
If this is intentional, set `export UV_LINK_MODE=copy` or use `--link-mode=copy` to suppress this warning.
Installed 68 packages in 301ms
+ annotated-types==0.7.0
+ anyio==4.12.1
+ certifi==2026.2.25
+ cffi==2.0.0
+ charset-normalizer==3.4.4
+ cryptography==46.0.7
+ distro==1.9.0
+ execnet==2.1.2
+ filetype==1.2.0
+ freezegun==1.5.5
+ google-auth==2.48.0
+ google-genai==1.65.0
+ h11==0.16.0
+ httpcore==1.0.9
+ httpx==0.28.1
+ idna==3.15
+ iniconfig==2.3.0
+ jsonpatch==1.33
+ jsonpointer==3.0.0
+ langchain-core==1.4.0
+ langchain-google-genai==4.2.3 (from file:///workspace/libs/genai)
+ langchain-protocol==0.0.15
+ langchain-tests==1.1.5
+ langsmith==0.8.0
+ markdown-it-py==4.0.0
+ mdurl==0.1.2
+ numpy==2.4.2
+ orjson==3.11.7
+ packaging==26.0
+ pluggy==1.6.0
+ protobuf==6.33.5
+ py-cpuinfo==9.0.0
+ pyasn1==0.6.3
+ pyasn1-modules==0.4.2
+ pycparser==3.0
+ pydantic==2.13.4
+ pydantic-core==2.46.4
+ pygments==2.20.0
+ pytest==9.0.3
+ pytest-asyncio==1.3.0
+ pytest-benchmark==5.2.3
+ pytest-codspeed==4.3.0
+ pytest-mock==3.15.1
+ pytest-recording==0.13.4
+ pytest-retry==1.7.0
+ pytest-socket==0.8.0
+ pytest-watcher==0.6.3
+ pytest-xdist==3.8.0
+ python-dateutil==2.9.0.post0
+ pyyaml==6.0.3
+ requests==2.33.0
+ requests-toolbelt==1.0.0
+ rich==14.3.3
+ rsa==4.9.1
+ six==1.17.0
+ sniffio==1.3.1
+ syrupy==5.2.0
+ tenacity==9.1.4
+ typing-extensions==4.15.0
+ typing-inspection==0.4.2
+ urllib3==2.7.0
+ uuid-utils==0.14.1
+ vcrpy==8.1.1
+ watchdog==6.0.0
+ websockets==16.0
+ wrapt==2.1.1
+ xxhash==3.6.0
+ zstandard==0.25.0
============================= test session starts ==============================
platform linux -- Python 3.11.15, pytest-9.0.3, pluggy-1.6.0
codspeed: 4.3.0 (disabled, mode: walltime, callgraph: not supported, timer_resolution: 1.0ns)
benchmark: 5.2.3 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /workspace/libs/genai
configfile: pyproject.toml
plugins: benchmark-5.2.3, asyncio-1.3.0, anyio-4.12.1, mock-3.15.1, recording-0.13.4, langsmith-0.8.0, xdist-3.8.0, retry-1.7.0, syrupy-5.2.0, codspeed-4.3.0, socket-0.8.0
asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 227 items
tests/integration_tests/test_callbacks.py . [ 0%]
tests/integration_tests/test_chat_models.py ...............F............ [ 12%]
.....xX........F........R..F.FF...F.....................s.ss. [ 39%]
tests/integration_tests/test_compile.py . [ 39%]
tests/integration_tests/test_embeddings.py ................ [ 46%]
tests/integration_tests/test_function_call.py ... [ 48%]
tests/integration_tests/test_llms.py ...... [ 50%]
tests/integration_tests/test_standard.py .......ssss.................... [ 64%]
........ss...s.xx.......ssss...............................s...s. [ 92%]
tests/integration_tests/test_structured_output_integration.py .......... [ 97%]
.... [ 99%]
tests/integration_tests/test_tools.py .. [100%]
=================================== FAILURES ===================================
__________ test_chat_google_genai_invoke_thinking_disabled[google_ai] __________
Traceback (most recent call last):
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3166, in _generate
response: GenerateContentResponse = self.client.models.generate_content(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 5644, in generate_content
response = self._generate_content(
^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 4306, in _generate_content
response = self._api_client.request(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1401, in request
response = self._request(http_request, http_options, stream=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1235, in _request
return retry(self._request_once, http_request, stream) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 470, in __call__
do = self.iter(retry_state=retry_state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 371, in iter
result = action(retry_state)
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 393, in <lambda>
self._add_action_func(lambda rs: rs.outcome.result())
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 473, in __call__
result = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1214, in _request_once
errors.APIError.raise_for_response(response)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 134, in raise_for_response
cls.raise_error(response.status_code, response_json, response)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 159, in raise_error
raise ClientError(status_code, response_json, response)
google.genai.errors.ClientError: 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 353, in from_call
result: TResult | None = func()
^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 245, in <lambda>
lambda: runtest_hook(item=item, **kwds),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
raise exception
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/logging.py", line 850, in pytest_runtest_call
yield
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
return (yield)
^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
return result.get_result()
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 103, in get_result
raise exc.with_traceback(tb)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
res = yield
^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
return (yield)
^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
res = hook_impl.function(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 179, in pytest_runtest_call
item.runtest()
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py", line 1720, in runtest
self.ihook.pytest_pyfunc_call(pyfuncitem=self)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
raise exception
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
return result.get_result()
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 103, in get_result
raise exc.with_traceback(tb)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
res = yield
^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
res = hook_impl.function(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py", line 166, in pytest_pyfunc_call
result = testfunction(**testargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/tests/integration_tests/test_chat_models.py", line 650, in test_chat_google_genai_invoke_thinking_disabled
result = llm.invoke(
^^^^^^^^^^^
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 2644, in invoke
return super().invoke(input, config, stop=stop, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 474, in invoke
self.generate_prompt(
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1823, in generate_prompt
return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1630, in generate
self._generate_with_cache(
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1970, in _generate_with_cache
result = self._generate(
^^^^^^^^^^^^^^^
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3170, in _generate
_handle_client_error(e, request)
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 169, in _handle_client_error
raise ChatGoogleGenerativeAIError(msg) from e
langchain_google_genai.chat_models.ChatGoogleGenerativeAIError: Error calling model 'gemini-2.5-flash' (NOT_FOUND): 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}
_______ test_thinking_params_preserved_with_structured_output[google_ai] _______
Traceback (most recent call last):
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3166, in _generate
response: GenerateContentResponse = self.client.models.generate_content(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 5644, in generate_content
response = self._generate_content(
^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 4306, in _generate_content
response = self._api_client.request(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1401, in request
response = self._request(http_request, http_options, stream=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1235, in _request
return retry(self._request_once, http_request, stream) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 470, in __call__
do = self.iter(retry_state=retry_state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 371, in iter
result = action(retry_state)
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 393, in <lambda>
self._add_action_func(lambda rs: rs.outcome.result())
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 473, in __call__
result = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1214, in _request_once
errors.APIError.raise_for_response(response)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 134, in raise_for_response
cls.raise_error(response.status_code, response_json, response)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 159, in raise_error
raise ClientError(status_code, response_json, response)
google.genai.errors.ClientError: 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 353, in from_call
result: TResult | None = func()
^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 245, in <lambda>
lambda: runtest_hook(item=item, **kwds),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
raise exception
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/logging.py", line 850, in pytest_runtest_call
yield
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
return (yield)
^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
return result.get_result()
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 103, in get_result
raise exc.with_traceback(tb)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
res = yield
^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
return (yield)
^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
res = hook_impl.function(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 179, in pytest_runtest_call
item.runtest()
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py", line 1720, in runtest
self.ihook.pytest_pyfunc_call(pyfuncitem=self)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
raise exception
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
return result.get_result()
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 103, in get_result
raise exc.with_traceback(tb)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
res = yield
^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
res = hook_impl.function(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py", line 166, in pytest_pyfunc_call
result = testfunction(**testargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/tests/integration_tests/test_chat_models.py", line 1482, in test_thinking_params_preserved_with_structured_output
result = structured_llm.invoke("What is 2+2? Just give a brief answer.")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 3333, in invoke
input_ = context.run(step.invoke, input_, config, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 5881, in invoke
return self.bound.invoke(
^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 2644, in invoke
return super().invoke(input, config, stop=stop, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 474, in invoke
self.generate_prompt(
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1823, in generate_prompt
return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1630, in generate
self._generate_with_cache(
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1970, in _generate_with_cache
result = self._generate(
^^^^^^^^^^^^^^^
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3170, in _generate
_handle_client_error(e, request)
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 169, in _handle_client_error
raise ChatGoogleGenerativeAIError(msg) from e
langchain_google_genai.chat_models.ChatGoogleGenerativeAIError: Error calling model 'gemini-2.5-flash' (NOT_FOUND): 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}
____________________ test_search_with_googletool[google_ai] ____________________
Traceback (most recent call last):
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3166, in _generate
response: GenerateContentResponse = self.client.models.generate_content(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 5644, in generate_content
response = self._generate_content(
^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 4306, in _generate_content
response = self._api_client.request(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1401, in request
response = self._request(http_request, http_options, stream=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1235, in _request
return retry(self._request_once, http_request, stream) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 470, in __call__
do = self.iter(retry_state=retry_state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 371, in iter
result = action(retry_state)
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 393, in <lambda>
self._add_action_func(lambda rs: rs.outcome.result())
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 473, in __call__
result = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1214, in _request_once
errors.APIError.raise_for_response(response)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 134, in raise_for_response
cls.raise_error(response.status_code, response_json, response)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 159, in raise_error
raise ClientError(status_code, response_json, response)
google.genai.errors.ClientError: 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 353, in from_call
result: TResult | None = func()
^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 245, in <lambda>
lambda: runtest_hook(item=item, **kwds),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^
...
[Logs truncated due to log size limitations. For full logs, see https://console.cloud.google.com/logs/viewer?advancedFilter=resource.type%3D%22build%22+AND+resource.labels.build_id%3D%22fd0e45a2-12ba-4aa0-8e64-a400837b5fe9%22+AND+resource.labels.build_trigger_id%3D%22cf0bf1b4-59b2-4ad2-9f4c-4866728a37b1%22&project=llm-integration-tests.]
...
e_ai] _____________________
Traceback (most recent call last):
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3166, in _generate
response: GenerateContentResponse = self.client.models.generate_content(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 5644, in generate_content
response = self._generate_content(
^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 4306, in _generate_content
response = self._api_client.request(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1401, in request
response = self._request(http_request, http_options, stream=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1235, in _request
return retry(self._request_once, http_request, stream) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 470, in __call__
do = self.iter(retry_state=retry_state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 371, in iter
result = action(retry_state)
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 393, in <lambda>
self._add_action_func(lambda rs: rs.outcome.result())
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 473, in __call__
result = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1214, in _request_once
errors.APIError.raise_for_response(response)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 134, in raise_for_response
cls.raise_error(response.status_code, response_json, response)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 159, in raise_error
raise ClientError(status_code, response_json, response)
google.genai.errors.ClientError: 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 353, in from_call
result: TResult | None = func()
^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 245, in <lambda>
lambda: runtest_hook(item=item, **kwds),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
raise exception
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/logging.py", line 850, in pytest_runtest_call
yield
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
return (yield)
^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
return result.get_result()
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 103, in get_result
raise exc.with_traceback(tb)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
res = yield
^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
return (yield)
^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
res = hook_impl.function(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 179, in pytest_runtest_call
item.runtest()
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py", line 1720, in runtest
self.ihook.pytest_pyfunc_call(pyfuncitem=self)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
raise exception
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
return result.get_result()
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 103, in get_result
raise exc.with_traceback(tb)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
res = yield
^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
res = hook_impl.function(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py", line 166, in pytest_pyfunc_call
result = testfunction(**testargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/tests/integration_tests/test_chat_models.py", line 1818, in test_google_maps_grounding
response = model_with_maps.invoke(
^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 5881, in invoke
return self.bound.invoke(
^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 2644, in invoke
return super().invoke(input, config, stop=stop, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 474, in invoke
self.generate_prompt(
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1823, in generate_prompt
return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1630, in generate
self._generate_with_cache(
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1970, in _generate_with_cache
result = self._generate(
^^^^^^^^^^^^^^^
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3170, in _generate
_handle_client_error(e, request)
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 169, in _handle_client_error
raise ChatGoogleGenerativeAIError(msg) from e
langchain_google_genai.chat_models.ChatGoogleGenerativeAIError: Error calling model 'gemini-2.5-flash' (NOT_FOUND): 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}
_____________ test_google_maps_grounding_invoke_direct[google_ai] ______________
Traceback (most recent call last):
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3166, in _generate
response: GenerateContentResponse = self.client.models.generate_content(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 5644, in generate_content
response = self._generate_content(
^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 4306, in _generate_content
response = self._api_client.request(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1401, in request
response = self._request(http_request, http_options, stream=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1235, in _request
return retry(self._request_once, http_request, stream) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 470, in __call__
do = self.iter(retry_state=retry_state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 371, in iter
result = action(retry_state)
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 393, in <lambda>
self._add_action_func(lambda rs: rs.outcome.result())
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 473, in __call__
result = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1214, in _request_once
errors.APIError.raise_for_response(response)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 134, in raise_for_response
cls.raise_error(response.status_code, response_json, response)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 159, in raise_error
raise ClientError(status_code, response_json, response)
google.genai.errors.ClientError: 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 353, in from_call
result: TResult | None = func()
^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 245, in <lambda>
lambda: runtest_hook(item=item, **kwds),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
raise exception
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/logging.py", line 850, in pytest_runtest_call
yield
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
return (yield)
^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
return result.get_result()
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 103, in get_result
raise exc.with_traceback(tb)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
res = yield
^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
return (yield)
^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
res = hook_impl.function(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 179, in pytest_runtest_call
item.runtest()
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py", line 1720, in runtest
self.ihook.pytest_pyfunc_call(pyfuncitem=self)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
raise exception
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
return result.get_result()
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 103, in get_result
raise exc.with_traceback(tb)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
res = yield
^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
res = hook_impl.function(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py", line 166, in pytest_pyfunc_call
result = testfunction(**testargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/tests/integration_tests/test_chat_models.py", line 1882, in test_google_maps_grounding_invoke_direct
response = model.invoke(
^^^^^^^^^^^^^
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 2644, in invoke
return super().invoke(input, config, stop=stop, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 474, in invoke
self.generate_prompt(
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1823, in generate_prompt
return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1630, in generate
self._generate_with_cache(
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1970, in _generate_with_cache
result = self._generate(
^^^^^^^^^^^^^^^
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3170, in _generate
_handle_client_error(e, request)
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 169, in _handle_client_error
raise ChatGoogleGenerativeAIError(msg) from e
langchain_google_genai.chat_models.ChatGoogleGenerativeAIError: Error calling model 'gemini-2.5-flash' (NOT_FOUND): 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}
_______ test_chat_google_genai_invoke_with_generation_params[google_ai] ________
Traceback (most recent call last):
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3166, in _generate
response: GenerateContentResponse = self.client.models.generate_content(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 5644, in generate_content
response = self._generate_content(
^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/models.py", line 4306, in _generate_content
response = self._api_client.request(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1401, in request
response = self._request(http_request, http_options, stream=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1235, in _request
return retry(self._request_once, http_request, stream) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 470, in __call__
do = self.iter(retry_state=retry_state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 371, in iter
result = action(retry_state)
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 393, in <lambda>
self._add_action_func(lambda rs: rs.outcome.result())
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 473, in __call__
result = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/_api_client.py", line 1214, in _request_once
errors.APIError.raise_for_response(response)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 134, in raise_for_response
cls.raise_error(response.status_code, response_json, response)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/google/genai/errors.py", line 159, in raise_error
raise ClientError(status_code, response_json, response)
google.genai.errors.ClientError: 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 353, in from_call
result: TResult | None = func()
^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 245, in <lambda>
lambda: runtest_hook(item=item, **kwds),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
raise exception
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/logging.py", line 850, in pytest_runtest_call
yield
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
return (yield)
^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
return result.get_result()
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 103, in get_result
raise exc.with_traceback(tb)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
res = yield
^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
return (yield)
^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
res = hook_impl.function(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 179, in pytest_runtest_call
item.runtest()
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py", line 1720, in runtest
self.ihook.pytest_pyfunc_call(pyfuncitem=self)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
raise exception
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
teardown.throw(exception)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
return result.get_result()
^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 103, in get_result
raise exc.with_traceback(tb)
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
res = yield
^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
res = hook_impl.function(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py", line 166, in pytest_pyfunc_call
result = testfunction(**testargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/tests/integration_tests/test_chat_models.py", line 2001, in test_chat_google_genai_invoke_with_generation_params
result_constrained = llm.invoke(
^^^^^^^^^^^
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 2644, in invoke
return super().invoke(input, config, stop=stop, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 474, in invoke
self.generate_prompt(
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1823, in generate_prompt
return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1630, in generate
self._generate_with_cache(
File "/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 1970, in _generate_with_cache
result = self._generate(
^^^^^^^^^^^^^^^
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 3170, in _generate
_handle_client_error(e, request)
File "/workspace/libs/genai/langchain_google_genai/chat_models.py", line 169, in _handle_client_error
raise ChatGoogleGenerativeAIError(msg) from e
langchain_google_genai.chat_models.ChatGoogleGenerativeAIError: Error calling model 'gemini-2.5-flash' (NOT_FOUND): 404 NOT_FOUND. {'error': {'code': 404, 'message': 'models/gemini-2.5-flash is not found for API version v1beta, or is not supported for generateContent. Call ModelService.ListModels to see the list of available models and their supported methods.', 'status': 'NOT_FOUND'}}
=============================== warnings summary ===============================
tests/integration_tests/test_chat_models.py::test_logprobs[google_ai]
tests/integration_tests/test_chat_models.py::test_logprobs_with_json_schema[google_ai]
/workspace/libs/genai/.venv/lib/python3.11/site-packages/_pytest/python.py:166: UserWarning: WARNING! logprobs is not default parameter.
logprobs was transferred to model_kwargs.
Please confirm that logprobs is what you intended.
result = testfunction(**testargs)
tests/integration_tests/test_standard.py::TestGeminiProStandardGoogleAI::test_audio_inputs
/workspace/libs/genai/.venv/lib/python3.11/site-packages/langchain_tests/integration_tests/chat_models.py:2684: UserWarning: LANGCHAIN_TESTS_USER_AGENT environment variable not set. langchain-tests pulls (CC0 License) audio data from wikimedia.org. Consider setting a user agent to identify your requests. See https://foundation.wikimedia.org/wiki/Policy:Wikimedia_Foundation_User-Agent_Policy
audio_data = _get_base64_from_url(url)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=================================== XPASSES ====================================
__________________ test_logprobs_with_json_schema[google_ai] ___________________
Loading