Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add testmon in pytest #1209

Conversation

jiashenC
Copy link
Member

@jiashenC jiashenC commented Sep 24, 2023

Add testmon in pytest

.circleci/config.yml Outdated Show resolved Hide resolved
@xzdandy xzdandy marked this pull request as ready for review September 25, 2023 06:48
…ts-that-are-affected-by-the-open-pr' of github.com:georgia-tech-db/eva into 1200-use-pytest-testmon-to-run-the-long-integration-tests-that-are-affected-by-the-open-pr
@gaurav274
Copy link
Member

Do we also need to push the .testmondata file?

@jiashenC
Copy link
Member Author

Do we also need to push the .testmondata file?

It will be generated from the staging branch and used for open PRs.

@@ -201,6 +212,11 @@ jobs:
- restore_cache:
keys:
- v1-model_cache-{{ checksum "setup.py" }}

# Restore testmondata
- restore_cache:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not something critical, shall we only restore the cache for the open PR long integration cached test?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I tried a different way to simplify the circle ci config. Please take a look.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I meant it is currently restoring the cache for any test. Is that expected? Do we want to rerun everything for staging and master? Or we should run only affected as well?

Copy link
Member

@gaurav274 gaurav274 Sep 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel we should run all the testcases for staging/master.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. We are running all the testcases for staging/master. We use --testmon-noselect for all staging and master.
  2. Yes. We are restoring the staging cache key for any test, but they are only used for open PRs. It avoids all the when condition to make circle ci simple. PS: I can not make when condition work for restoring cache.

@xzdandy xzdandy mentioned this pull request Sep 26, 2023
5 tasks
…ts-that-are-affected-by-the-open-pr' of github.com:georgia-tech-db/eva into 1200-use-pytest-testmon-to-run-the-long-integration-tests-that-are-affected-by-the-open-pr
@xzdandy
Copy link
Collaborator

xzdandy commented Sep 26, 2023

@jiashenC @gaurav274 This PR is ready. Please review. Thanks.

@xzdandy xzdandy mentioned this pull request Sep 26, 2023
6 tasks
@gaurav274 gaurav274 merged commit 40966de into staging Sep 26, 2023
@gaurav274 gaurav274 deleted the 1200-use-pytest-testmon-to-run-the-long-integration-tests-that-are-affected-by-the-open-pr branch September 26, 2023 22:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature Request ✨ New feature or request
Projects
Development

Successfully merging this pull request may close these issues.

Use pytest-testmon to run the long integration tests that are affected by the open PR
3 participants