Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add nftests custom modules #7201

Open
wants to merge 16 commits into
base: master
Choose a base branch
from
Open

Conversation

nschcolnicov
Copy link
Contributor

PR checklist

Closes #XXX

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the module conventions in the contribution docs
  • If necessary, include test data in your PR.
  • Remove all TODO statements.
  • Emit the versions.yml file.
  • Follow the naming conventions.
  • Follow the parameters requirements.
  • Follow the input/output options guidelines.
  • Add a resource label
  • Use BioConda and BioContainers if possible to fulfil software requirements.
  • Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
    • For modules:
      • nf-core modules test <MODULE> --profile docker
      • nf-core modules test <MODULE> --profile singularity
      • nf-core modules test <MODULE> --profile conda
    • For subworkflows:
      • nf-core subworkflows test <SUBWORKFLOW> --profile docker
      • nf-core subworkflows test <SUBWORKFLOW> --profile singularity
      • nf-core subworkflows test <SUBWORKFLOW> --profile conda

@nschcolnicov nschcolnicov requested a review from a team as a code owner December 12, 2024 20:01
.github/workflows/pytest-workflow.yml Outdated Show resolved Hide resolved
modules/nf-core/custom/matrixfilter/main.nf Outdated Show resolved Hide resolved
modules/nf-core/custom/matrixfilter/main.nf Show resolved Hide resolved
modules/nf-core/custom/matrixfilter/tests/main.nf.test Outdated Show resolved Hide resolved
modules/nf-core/custom/matrixfilter/tests/main.nf.test Outdated Show resolved Hide resolved
modules/nf-core/custom/matrixfilter/tests/nextflow.config Outdated Show resolved Hide resolved
modules/nf-core/custom/tabulartogseacls/main.nf Outdated Show resolved Hide resolved
modules/nf-core/custom/tabulartogseagct/main.nf Outdated Show resolved Hide resolved
@famosab
Copy link
Contributor

famosab commented Dec 16, 2024

You will also need to remove the modules from the pytest config file. Afterwards we should be good to go :)

https://github.com/nf-core/modules/blob/816ff3b787af7e12e75a7d95f76b631f474853ee/tests/config/pytest_modules.yml#L112C1-L120C53

@nschcolnicov
Copy link
Contributor Author

nschcolnicov commented Dec 17, 2024

There is a linting error:
image
But running the command suggested nf-core modules lint --fix reveals that the changes that it is expecting are not correct, it tries to convert all of the output channels into map type:
image

What do you think about this? @famosab

@famosab
Copy link
Contributor

famosab commented Dec 17, 2024

Huh that is interesting can you commit the "fixed" yaml files and push them to this PR so I can have a look?

@nschcolnicov
Copy link
Contributor Author

Huh that is interesting can you commit the "fixed" yaml files and push them to this PR so I can have a look?

Take a look: eb84fde

Comment on lines 72 to 76
- "*R_sessionInfo.log":
type: map
description: |
Groovy Map containing information on experiment.
e.g. [ id:'test' ]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the update was needed because we changed one of the outputs per my request. But why --fix now does these kind of replacements also seems off to me. @mashehu @mirpedrol what do you guys think? is that expected behavior and should we just manually revert these changes or is something wrong with tools?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing the name of the output and using *R_sessionInfo.log is the right change but replacing the type and description is not! We saw the same with other modules, it must be a weird bug with the --fix command, you can revert this.
And the same for the other outputs where this happened, like filtered.csv and tests.csv

Copy link
Contributor

@famosab famosab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are almost done :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there was a new convention recently, maybe you can update to adhere to that: https://nf-co.re/docs/guidelines/components/modules#configuration-of-extargs-in-tests

Copy link
Contributor Author

@nschcolnicov nschcolnicov Dec 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to make sure, you are requesting me to pass the values that I'm providing via ext.args using params instead of the actual value? Didn't you correct me to do it this way on a previous comment? https://github.com/nf-core/modules/pull/7201/files/816ff3b787af7e12e75a7d95f76b631f474853ee#r1888548704 Want to make sure I'm doing it the exact way you are suggesting

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I did then I am sorry. But what is meant is that the params that you need to pass are written directly in the nf.test file which makes its directly understandable for other people that look at your tests and then they do not have to go through multiple nextflow.config files.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, sorry, you are right, I was wrong both times, I just wasn't understanding how you wanted me to structure the config, I think I have now understood and addressed the issues, let me know what you think @famosab 😄

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see comment above

@nschcolnicov
Copy link
Contributor Author

It seems that I'm no longer able to push to this branch:
image
@famosab @mirpedrol Do you know what could be wrong?

@nschcolnicov nschcolnicov force-pushed the add_nftests_custom_modules branch from eb84fde to 50e2138 Compare December 18, 2024 16:32
@famosab
Copy link
Contributor

famosab commented Dec 19, 2024

@nschcolnicov I think you should be able to push again :)

@nschcolnicov
Copy link
Contributor Author

@famosab Tests should be passing but the docker_self_hosted one is having an issue, I don't think it is related to my changes though. I see they all fail with this error:
touch: cannot touch '.command.trace': Permission denied
Also linting still fails due to the issue where it wants to have everything in the meta.yml converted to meta type.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants