Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature]: Make E2E Target Names Easier to Use (i.e. Make _e2e-tests.sh_ Script Easier to Use) #443

Open
1 task done
0xNeshi opened this issue Dec 10, 2024 · 0 comments · May be fixed by #467
Open
1 task done
Labels
effort: low Easy or tiny task that takes less than a day. priority: 0 Nice-to-have. Willing to ship without this. type: ref A code update that doesn't meaningfully change functionality. type: test Changes to the testing suite.

Comments

@0xNeshi
Copy link
Collaborator

0xNeshi commented Dec 10, 2024

What is the feature you would like to see?

E.g. currently there's no way to run only safe-erc20-example e2e tests without modifying the e2e-tests.sh script.
This is because we run our e2e tests from the workspace, which looks at file names to filter out specific tests to run.

Let's say you wanted to run them - you'd probably write something like:

./scripts/e2e-tests.sh safe-erc20*

This will error out with the following log:

error: no test target matches pattern `safe-erc20*`.
Available test targets:
    access_control
    ecdsa
    erc1155
    erc1155-metadata-uri
    erc20
    erc20permit
    erc721-consecutive
    erc721
    erc721
    ownable
    ownable_two_step
    address_with_no_code
    erc20
    erc20_that_always_returns_false
    erc20_that_does_not_return
    usdt_approval_behavior
    vesting-wallet

A couple of problems are immediately noticeable:

  1. No target name indicates a relation to SafeErc20.
  2. There are duplicate target names.
  3. There seem to be missing certain e2e tests.

This may not be necessarily a problem for us who are intimately acquainted with our code base, but it will be a problem for any new contributor, both external, and potentially internal ones.

I propose we make it even easier and clearer how to run e2e tests from the command line using one of the below approaches:

  1. Rename each e2e test file to start with its directory name.
    Pros:
    • No change necessary to e2e-tests.sh script.
    • Filtering tests (e.g. safe-erc20* - note the asterisk) becomes easier.
      Cons:
    • Long and repetitive file names.
  2. Update e2e-tests.sh to run tests withing specific packages.
    Pros:
    • File names remain unchanged.
      Cons:
    • Writing a wrong package name does not list valid packages.
    ~$ ./scripts/e2e-tests.sh non-existent-package
        Finished `release` profile [optimized] target(s) in 0.12s
    error: cannot specify features for packages outside of workspace
    • Need to declare std feature in all examples.

I'm in favor of option 1) just because when a wrong target name is typed, the error log contains all valid targets.

Contribution Guidelines

  • I agree to follow this project's Contribution Guidelines
@0xNeshi 0xNeshi added priority: 0 Nice-to-have. Willing to ship without this. type: test Changes to the testing suite. effort: low Easy or tiny task that takes less than a day. type: ref A code update that doesn't meaningfully change functionality. labels Dec 10, 2024
@0xNeshi 0xNeshi linked a pull request Dec 26, 2024 that will close this issue
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
effort: low Easy or tiny task that takes less than a day. priority: 0 Nice-to-have. Willing to ship without this. type: ref A code update that doesn't meaningfully change functionality. type: test Changes to the testing suite.
Projects
Status: Todo
Development

Successfully merging a pull request may close this issue.

1 participant