Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[generative_ai] Test for embeddings fails on CI #12995

Open
Valeriy-Burlaka opened this issue Dec 13, 2024 · 0 comments
Open

[generative_ai] Test for embeddings fails on CI #12995

Valeriy-Burlaka opened this issue Dec 13, 2024 · 0 comments
Assignees
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. samples Issues that are directly related to samples. triage me I really want to be triaged. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@Valeriy-Burlaka
Copy link
Member

Valeriy-Burlaka commented Dec 13, 2024

In which file did you encounter the issue?

generative_ai/embeddings/test_embeddings_examples.py

Did you change the file? If so, how?

No. The only change I made was in the test file to trigger the CI failure. Please check this commit that reproduces the issue and its build log.

Describe the issue

Test for embeddings test_embeddings_examples.py::test_tune_embedding_model started to fail on CI while still passing locally. Reproduces on main and, with no code changes:

============================= test session starts ==============================
platform linux -- Python 3.9.18, pytest-8.2.0, pluggy-1.5.0 -- /workspace/generative_ai/embeddings/.nox/py-3-9/bin/python
cachedir: .pytest_cache
rootdir: /workspace
configfile: pytest.ini
plugins: asyncio-0.23.6, anyio-4.7.0
asyncio: mode=strict
collecting ... collected 9 items

test_embeddings_examples.py::test_multimodal_embedding_image_video_text PASSED [ 11%]
test_embeddings_examples.py::test_multimodal_embedding_video PASSED      [ 22%]
test_embeddings_examples.py::test_multimodal_embedding_image PASSED      [ 33%]
test_embeddings_examples.py::test_generate_embeddings_with_lower_dimension PASSED [ 44%]
test_embeddings_examples.py::test_create_embeddings PASSED               [ 55%]
test_embeddings_examples.py::test_create_text_embeddings PASSED          [ 66%]
test_embeddings_examples.py::test_text_embed_text PASSED                 [ 77%]
test_embeddings_examples.py::test_code_embed_text PASSED                 [ 88%]
test_embeddings_examples.py::test_tune_embedding_model FAILED            [100%]

=================================== FAILURES ===================================
__________________________ test_tune_embedding_model ___________________________
Traceback (most recent call last):
 
...
# Val: Skipping the pytest/runner stacktrace.
...
    result = testfunction(**testargs)
  File "/workspace/generative_ai/embeddings/test_embeddings_examples.py", line 133, in test_tune_embedding_model
    tuning_job = model_tuning_example.tune_embedding_model(
  File "/workspace/generative_ai/embeddings/model_tuning_example.py", line 42, in tune_embedding_model
    tuning_job = base_model.tune_model(
  File "/workspace/generative_ai/embeddings/.nox/py-3-9/lib/python3.9/site-packages/vertexai/language_models/_language_models.py", line 2344, in tune_model
    return super().tune_model(
  File "/workspace/generative_ai/embeddings/.nox/py-3-9/lib/python3.9/site-packages/vertexai/language_models/_language_models.py", line 367, in tune_model
    return self._tune_model(
  File "/workspace/generative_ai/embeddings/.nox/py-3-9/lib/python3.9/site-packages/vertexai/language_models/_language_models.py", line 422, in _tune_model
    if _is_text_embedding_tuning_pipeline(model_info.tuning_pipeline_uri):
  File "/workspace/generative_ai/embeddings/.nox/py-3-9/lib/python3.9/site-packages/vertexai/language_models/_language_models.py", line 4010, in _is_text_embedding_tuning_pipeline
    return pipeline_uri.startswith(
AttributeError: 'NoneType' object has no attribute 'startswith'
---- generated xml file: /workspace/generative_ai/embeddings/sponge_log.xml ----
=========================== short test summary info ============================
FAILED test_embeddings_examples.py::test_tune_embedding_model - AttributeErro...
========================= 1 failed, 8 passed in 21.42s =========================
nox > Command pytest --junitxml=sponge_log.xml failed with exit code 1
nox > Session py-3.9 failed.
make: *** [Makefile:40: test] Error 1
/workspace/generative_ai/embeddings
PWD: /workspace/generative_ai/embeddings

 Testing failed: Nox returned a non-zero exit code. 
@Valeriy-Burlaka Valeriy-Burlaka added triage me I really want to be triaged. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Dec 13, 2024
@product-auto-label product-auto-label bot added the samples Issues that are directly related to samples. label Dec 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. samples Issues that are directly related to samples. triage me I really want to be triaged. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

No branches or pull requests

2 participants