Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add number of passed and failed doctests to test summary #1376

Open
benzwick opened this issue Jul 27, 2020 · 1 comment
Open

Add number of passed and failed doctests to test summary #1376

benzwick opened this issue Jul 27, 2020 · 1 comment

Comments

@benzwick
Copy link

It would be nice to show the number of doctests that passed or failed in the test summary as in other tests, e.g. (https://docs.julialang.org/en/v1/stdlib/Test/#Working-with-Test-Sets-1)

Test Summary: | Pass  Fail  Total
Foo Tests     |    3     1      4
  Animals     |    2            2
  Arrays      |    1     1      2

Currently the output I get is similar to this when all doctests pass:

Test Summary:                             | Pass   Total
  doctest                                 |    1       1
    Doctests: MyPackage                   |    1       1

and this when one or more (in this case two) doctests fail:

Test Summary:                                                          | Pass  Fail  Error  Total
  doctest                                                              |          1             1
    Doctests: MyPackage                                                |          1             1

so it is not possible to see how many doctests have failed. Conversely, I would also like to know how many of my doctests were successful.

The doctest for this example is in runtests.jl as follows:

@time @testset "MyPackage" begin
    @testset "doctest" begin
        Documenter.DocMeta.setdocmeta!(MyPackage, :DocTestSetup, :(using MyPackage); recursive=true)
        Documenter.doctest(MyPackage)
    end
end
@fingolfin
Copy link
Contributor

Perhaps this would be resolved (or at least made a bit better) by adding verbose=true to the @testset generated by Documenter.doctest, and perhaps also your one wrapping that?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants