Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ScoverageAggregate for different test #124

Open
aesteve opened this issue Nov 4, 2019 · 4 comments
Open

ScoverageAggregate for different test #124

aesteve opened this issue Nov 4, 2019 · 4 comments

Comments

@aesteve
Copy link

aesteve commented Nov 4, 2019

Hi and sorry to create an issue, since it's more of a question than a proper issue, but how do you aggregate coverage when you have different test tasks?

I have a multi-project build.
Say I have two test tasks: test and integrationTest
Then I have (as you said): reportTestScoverage and reportIntegrationTestScoverage => perfect.
I tried aggregateScoverage: it's running both the test and integrationTest tasks, perfect.
But I'd need to have two different aggregateScoverage tasks, one for each test task.

I tried first:

        create<org.scoverage.ScoverageAggregate>("testCoverage") {
            dependsOn("reportTestScoverage")
        }

But it failed saying many properties were missing.

Thus I tried:

        create<org.scoverage.ScoverageAggregate>("testCoverage") {
            dependsOn("reportTestScoverage")
            coverageOutputCobertura.set(true)
            coverageOutputHTML.set(true)
            coverageOutputXML.set(true)
            deleteReportsOnAggregation.set(false)
            coverageDebug.set(false)
            reportDir.set(rootProject.buildDir.resolve("reports"))
        }

But this failed because ScoverageAggregate.runner is null.

So I finally got it working through:

        create<org.scoverage.ScoverageAggregate>("testCoverage") {
            dependsOn("reportTestScoverage")
            coverageOutputCobertura.set(true)
            coverageOutputHTML.set(true)
            coverageOutputXML.set(true)
            deleteReportsOnAggregation.set(false)
            coverageDebug.set(false)
            reportDir.set(rootProject.buildDir.resolve("reports"))
            runner = ScoverageRunner(project.configurations.scoverage.get())
        }

But it doesn't work, it only creates scoverage for the last subProject it builded.

Do you have any pointer? Thank you!

@maiflai
Copy link
Contributor

maiflai commented Nov 5, 2019

I'm not sure this is going to work in the ideal fashion, mainly because of the way that scoverage currently works under the covers.

Coverage of a particular class is measured and written to a location fixed during compilation.

I think this means that you won't be able to easily separate coverage obtained under unit or integration tests in a single build.

You might try using -x test on the command line, which should then only run the integrationTest tasks? You would then need to run another build skipping the test tasks, possibly into a different buildDir if you want to retain the results.

Stu.

@aesteve
Copy link
Author

aesteve commented Nov 10, 2019

Ok so standard aggregateScoverage but with -x option, and two different reports.

I can live with that I think. Mostly, test should have maximum coverage except for the environment-dependent stuff.
Then, integrationTest should just check that given the right set of environment variables, it "boots", and "does some stuff".

So I guess I can merge both "manually" by asserting integration have covered the stuff we need it to cover (env variables and App stuff).

Thanks for your help!

@eyalroth
Copy link
Contributor

You might try using -x test on the command line, which should then only run the integrationTest tasks? You would then need to run another build skipping the test tasks, possibly into a different buildDir if you want to retain the results.

I also think that's the way to go, but with a slight adjustment. Instead of omitting the test task, try omitting the specific report task instead:

gradle aggregateScoverage -x reportTestScoverage
or
gradle aggregateScoverage -x reportIntegrationTestScoverage

Make sure to clean between the two, since the aggregation task will not be able to tell which report files were generated on the first build and which on the second.

@atais
Copy link

atais commented Oct 15, 2020

I used

gradle aggregateScoverage -x reportTestScoverage

and it still fires integrationTest task.

My test config:

apply plugin: "org.scoverage"
apply plugin: "com.github.maiflai.scalatest"

test {
    tags {
        exclude "org.scalatest.tags.Slow"
    }
}

task integrationTest(type: Test) {
    tags {
        include "org.scalatest.tags.Slow"
    }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants