@contents and any canonical bibliography do not work together #16

kellertuer · 2023-07-13T10:09:54Z

I narrowed my make file down to

Scenario 1

using Manopt
using Documenter: DocMeta, HTML, MathJax3, deploydocs, makedocs
using DocumenterCitations

# (e) ...finally! make docs
bib = CitationBibliography(joinpath(@__DIR__, "src", "references.bib"); style=:authoryear)
print(bib) # displays all my references in a complicated formal -> references.bib found
makedocs(bib;
    format=HTML(; mathengine=MathJax3(), prettyurls=get(ENV, "CI", nothing) == "true"),
    modules=[Manopt],
    sitename="Manopt.jl",
    pages=[
        "Home" => "index.md",
    ],
)
deploydocs(; repo="github.com/JuliaManifolds/Manopt.jl", push_preview=true)

and in my index.md

```@bibliography
Pages = ["index.md"]
Canonical = false
```

This does not find the cites and warns that the links do not exist like

┌ Warning: invalid local link: unresolved path in index.md
│   link.text =
│    1-element Vector{Any}:
│     "Boumal:2023"
│   link.url = "@cite"
└ @ Documenter.Writers.HTMLWriter ~/.julia/packages/Documenter/bYYzK/src/Writers/HTMLWriter.jl:2081

But generates the right bibliography at the end of the index page.

Scenario 2

If I instead set line 2 to

using Documenter

I get

ERROR: LoadError: MethodError: no method matching HTML(; mathengine::MathJax3, prettyurls::Bool)

Closest candidates are:
  HTML(::T) where T got unsupported keyword arguments "mathengine", "prettyurls"
   @ Base docs/utils.jl:30
  HTML(::Any...) got unsupported keyword arguments "mathengine", "prettyurls"
   @ Base docs/utils.jl:33

Stacktrace:
 [1] top-level scope
   @ ~/Repositories/Julia/Manopt.jl/docs/make2.jl:11

If I leave out the format= keyword in the makedocs I am back to references not being found.

Scenario 3

I leave out the Canonical=false in the index.md (maybe one needs at least one canonical one for the links to work?)

The I get

[ Info: Populate: populating indices.
ERROR: LoadError: MethodError: no method matching header_level(::BibInternal.Entry)

Closest candidates are:
  header_level(::Markdown.Header{N}) where N
   @ Documenter ~/.julia/packages/Documenter/bYYzK/src/Utilities/Utilities.jl:380

Stacktrace:
 [1] populate!(contents::Documenter.Documents.ContentsNode, document::Documenter.Documents.Document)
   @ Documenter.Documents ~/.julia/packages/Documenter/bYYzK/src/Documents.jl:451
 [2] populate!(document::Documenter.Documents.Document)
   @ Documenter.Documents ~/.julia/packages/Documenter/bYYzK/src/Documents.jl:413
 [3] runner(#unused#::Type{Documenter.Builder.Populate}, doc::Documenter.Documents.Document)
   @ Documenter.Builder ~/.julia/packages/Documenter/bYYzK/src/Builder.jl:247
 [4] dispatch(#unused#::Type{Documenter.Builder.DocumentPipeline}, x::Documenter.Documents.Document)
   @ Documenter.Utilities.Selectors ~/.julia/packages/Documenter/bYYzK/src/Utilities/Selectors.jl:170
 [5] #2
   @ ~/.julia/packages/Documenter/bYYzK/src/Documenter.jl:273 [inlined]
 [6] cd(f::Documenter.var"#2#3"{Documenter.Documents.Document}, dir::String)
   @ Base.Filesystem ./file.jl:112
 [7] makedocs(components::CitationBibliography; debug::Bool, format::Documenter.Writers.HTMLWriter.HTML, kwargs::Base.Pairs{Symbol, Any, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:modules, :sitename, :pages), Tuple{Vector{Module}, String, Vector{Pair{String, String}}}}})
   @ Documenter ~/.julia/packages/Documenter/bYYzK/src/Documenter.jl:272
 [8] top-level scope
   @ ~/Repositories/Julia/Manopt.jl/docs/make2.jl:11
in expression starting at /Users/ronnber/Repositories/Julia/Manopt.jl/docs/make2.jl:11

The reduced makefile is submitted currently also on the corresponding branch, cf. https://github.com/JuliaManifolds/Manopt.jl/blob/kellertuer/documentercitations/docs/make2.jl

and I am running with

  [e30172f5] Documenter v0.27.25
  [daee34ce] DocumenterCitations v1.0.0

And I am by now a bit lost how to get this to run, I can either not have working links (but a correct bibliography at the end) or error messages. Is there some magic I am missing?

The text was updated successfully, but these errors were encountered:

goerz · 2023-07-13T10:42:45Z

You have to have one (or at least one, with caveats) canonical bibliography. By definition, that's where all citation links go.

Is "Scenario 3" identical to "Scenario 1" except for removing Canonical=false?

Scenarios 2 seems unrelated. I don't think HTML is exported in Documenter. I have using Documenter and then Documenter.HMTL(…) in make.jl.

I'm not really sure what's going on in Scenario 3. It could be some weird interaction with the Markdown parser, similar to #14?

goerz · 2023-07-13T10:43:45Z

Just for fun, try removing the colon from your citation keys

kellertuer · 2023-07-13T11:23:29Z

Removed the colon already, all 3 scenarios are the same then.

Yes 3 is just 1 without Canonical=false.

kellertuer · 2023-07-13T12:00:20Z

Ah yes I missed the caveat that a canonical reference list has to exist and added such a page (just covering the canonical one). Then I still get (concistently) The error from 3 now also in 1.

kellertuer · 2023-07-13T12:12:15Z

Havin only one @bibliography in references.md (and that not even yet entered into the menu) like

# Literature

This is all literature mentioned / referenced in the `Manopt.jl` documenation.
Usually you will find a small reference section at the end of every documentation page that contains references.

```@bibliography
```

still yields

ERROR: LoadError: MethodError: no method matching header_level(::BibInternal.Entry)

Closest candidates are:
  header_level(::Markdown.Header{N}) where N
   @ Documenter ~/.julia/packages/Documenter/bYYzK/src/Utilities/Utilities.jl:380

Stacktrace:
 [1] populate!(contents::Documenter.Documents.ContentsNode, document::Documenter.Documents.Document)
   @ Documenter.Documents ~/.julia/packages/Documenter/bYYzK/src/Documents.jl:451
 [2] populate!(document::Documenter.Documents.Document)
   @ Documenter.Documents ~/.julia/packages/Documenter/bYYzK/src/Documents.jl:413
 [3] runner(#unused#::Type{Documenter.Builder.Populate}, doc::Documenter.Documents.Document)
   @ Documenter.Builder ~/.julia/packages/Documenter/bYYzK/src/Builder.jl:247
 [4] dispatch(#unused#::Type{Documenter.Builder.DocumentPipeline}, x::Documenter.Documents.Document)
   @ Documenter.Utilities.Selectors ~/.julia/packages/Documenter/bYYzK/src/Utilities/Selectors.jl:170
 [5] #2
   @ ~/.julia/packages/Documenter/bYYzK/src/Documenter.jl:273 [inlined]
 [6] cd(f::Documenter.var"#2#3"{Documenter.Documents.Document}, dir::String)
   @ Base.Filesystem ./file.jl:112
 [7] makedocs(components::CitationBibliography; debug::Bool, format::Documenter.Writers.HTMLWriter.HTML, kwargs::Base.Pairs{Symbol, Any, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:modules, :sitename, :pages), Tuple{Vector{Module}, String, Vector{Pair{String, String}}}}})
   @ Documenter ~/.julia/packages/Documenter/bYYzK/src/Documenter.jl:272
 [8] top-level scope
   @ ~/Repositories/Julia/Manopt.jl/docs/make2.jl:10
in expression starting at /Users/ronnber/Repositories/Julia/Manopt.jl/docs/make2.jl:10

but no warnings that the cites do not exist before, still also no single rendered output html page. If I remove said references file (still not bibliography command in the index.md) I only get warnings about non existing links of course.

I am not sure what this error is trying to tell me. Seems like it wants to treat a BibTeX entry as a heading?

edit: The error appears as soon as I have a reference to be linked/added in the literature of a canonical bibliography. So If

I have no single @cite all works fine (and the bibliography global block is empty).
If I have a single cite and a canonical bibliography block (e.g. in my references.md) the error above appears
If I have the @bibliography block (either in references.md or index.md) set to non canonical, it renders, but no links work

kellertuer · 2023-07-13T13:18:51Z

If I do

import Documenter.Utilities.header_level
using BibInternal
header_level(::BibInternal.Entry) = 0

I get a bit further until I hit

ERROR: LoadError: type Entry has no field text
Stacktrace:
  [1] getproperty(x::BibInternal.Entry, f::Symbol)
    @ Base ./Base.jl:37
  [2] domify(ctx::Documenter.Writers.HTMLWriter.HTMLContext, navnode::Documenter.Documents.NavNode, contents::Documenter.Documents.ContentsNode)
    @ Documenter.Writers.HTMLWriter ~/.julia/packages/Documenter/bYYzK/src/Writers/HTMLWriter.jl:1513
  [3] (::Documenter.Writers.HTMLWriter.var"#78#79"{Documenter.Writers.HTMLWriter.HTMLContext, Documenter.Documents.NavNode, Documenter.Documents.Page})(elem::Markdown.Code)
    @ Documenter.Writers.HTMLWriter ~/.julia/packages/Documenter/bYYzK/src/Writers/HTMLWriter.jl:1453

but I fear I might be missing something tremendously before that I get the first error actually? I am just not sure what and I am lost by now.

fredrikekre · 2023-07-13T15:08:55Z

Not sure if related, but note that Documenter works with all markdown files in src not just files in the pages = ... keyword argument. This

(and that not even yet entered into the menu

got me thinking perhaps you have some garbage files left from previous tries in src?

kellertuer · 2023-07-13T15:11:49Z

I am aware that Documenter renders all markdown,

But, no, sadly not that I am aware of. Everything works fine as long as long as I have either zero cites or no canonical bibliography. As soon as I have both, it seems fail to reference the bib entries since those do not have a header level?

goerz · 2023-07-13T15:17:47Z

Can you try using this .bib file? Just to test it? I’m thinking there’s something in your .bib file that’s tripping up the system.

kellertuer · 2023-07-13T15:20:10Z

Sure, will try. My current file is already really minimal and just

@book{AbsilMahonySepulchre:2008,
  title = {Optimization Algorithms on Matrix Manifolds},
  author = {Absil, P.-A. and Mahony, R. and Sepulchre, R.},
  year = {2008},
  publisher = {Princeton University Press},
  doi = {10.1515/9781400830244}
}

goerz · 2023-07-13T15:22:27Z

Try removing the colon from the key name

kellertuer · 2023-07-13T15:23:07Z

I did try that a lot of times already, that does not change anything.

goerz · 2023-07-13T15:23:46Z

Ok, very weird… I’ll try to dig into this when I’m back at my computer

kellertuer · 2023-07-13T15:24:38Z

No problem, I will try to narrow it down to a as-few-as-possible-file/lines example later today.

fredrikekre · 2023-07-13T15:25:02Z

Did you restart Julia in between? Perhaps the caching is causing the problem if it has already cached some bad entry?

kellertuer · 2023-07-13T15:27:02Z

I usually start my documenter run from terminal, since there I can decide with --quarto whether to render quarto stuff before running documenter or not. So Julia is indeed specifically started to run Documenter.

kellertuer · 2023-07-13T15:30:31Z

Can you try using this .bib file? Just to test it? I’m thinking there’s something in your .bib file that’s tripping up the system.

The error also appears when I switch from my references.bib to your refs.bib (ignoring the missing cite keys I get) and additionally writing [Evans1983](@cite) in my index.md (running make2.jl, my minimal file atop). So it is most probably not the bib file.

edit: even when my index md is the only file with citation code and is minimised to

[Evans1983](@cite)

```@bibliography
Pages = ["index.md"]
```

goerz · 2023-07-13T15:33:31Z

Okay, that’s good to know, at least

kellertuer · 2023-07-13T16:49:07Z

So! I went first for the approach of an MWE, but that suddenly worked, so I slowly in a binary fashion deleted files from (a copy of) my relatively large documentation to find the page that issues the error. 😴
But then 🤓

If you do a src/references.bib as

@book{AbsilMahonySepulchre:2008,
  title = {Optimization Algorithms on Matrix Manifolds},
  author = {Absil, P.-A. and Mahony, R. and Sepulchre, R.},
  year = {2008},
  publisher = {Princeton University Press},
  doi = {10.1515/9781400830244}
}

and an src/index.md as

```@contents
Depth = 3
```

[AbsilMahonySepulchre:2008](@cite)

```@bibliography
```

and a make.jl like (make sure to have Documenter, DocumenterCitations and here Manopt in your environment)

using Pkg
Pkg.activate(@__DIR__)
using Manopt, Documenter, DocumenterCitations

# (e) ...finally! make docs
bib = CitationBibliography(joinpath(@__DIR__, "src", "references.bib"); style=:authoryear)
makedocs(bib;
    modules=[Manopt],
    sitename="Manopt.jl",
    pages=[
        "Home" => "index.md",
    ],
)
deploydocs(; repo="github.com/JuliaManifolds/Manopt.jl", push_preview=true)

That one fails!

So it seems to be some interplay between table of contents and bibliography!

Luckily at least for some time I can ignore the @contents (in the long run for accessibility I would prefer to have that).
On the other hand – luckily – it is not Quarto or something causing this, but indeed Documenter itself.

goerz · 2023-07-13T17:38:25Z

Ah, so this is the "old" issue ali-ramadhan/DocumenterCitations.jl#24

Well, good to know that's still an issue, I guess.

We'll have to really figure out why that happens and fix it at some point.

I'm sorry this caused you so much trouble. Quite a rabbit hole!

For now, my recommendation would be to have your canonical reference on a separate page, and specifically not in index.md, where you probably have @contents.

kellertuer · 2023-07-13T17:42:03Z

Well, it was not that nice to start with this.

It does not matter whether @contents and the literature are on the same markdown file. my @contents was on a separate lists.md file somewhere far away from my references.md (canonical) or my local ones I will to per page.

And I was not aware that this was even an old issue, well. Now I wasted this afternoon. It is what it is, and at least now it seems to work and I just have to rewrite my two styles of references into one (but hopefully only do such thing once).

goerz · 2023-07-13T17:49:53Z

Yeah, I understand. I'm sorry about the frustrating experience.

We'll definitely look into improving this situation, but it might take some time.

kellertuer · 2023-07-13T17:51:27Z

Sure, no problem :) Thanks for providing this package, I am looking forward to new docs with nicer references (and might let you know once I have finished that).

goerz · 2023-07-13T18:49:38Z

It does not matter whether @contents and the literature are on the same markdown file.

Right! So based on ali-ramadhan/DocumenterCitations.jl#24 (comment) and ali-ramadhan/DocumenterCitations.jl#33 (comment), as well as my own testing just now, the current workaround is to have your canonical bibliography on a separate page, and exclude that page from the Pages list of any @contents block.

Definitely not an ideal situation.

goerz · 2023-07-13T18:55:10Z

Now that I can reproduce this, I'm not sure if this is really a bug in DocumenterCitations.jl, or in Documenter.jl.

The error gets triggered by the code handling the @contents block. It processes the content of the listed pages and for some reason gets to a point where it wants to know the header_level of a bibtex entry.

@fredrikekre Should we open an issue for this at https://github.com/JuliaDocs/Documenter.jl/issues? Or maybe we should figure out exactly why this happens, first.

mortenpi · 2023-07-13T23:30:48Z

Does it also error when you use #3 and Documenter#master? With all the changes to the AST-related internals, maybe this has fixed itself.

goerz · 2023-09-15T06:18:09Z

Does it also error when you use #3 and Documenter#master

Unfortunately, yes. But in the latest iteration of #3 I've added a @test_broken case to track this. So I should be able to get to the bottom of this, soonish.

goerz · 2023-09-20T17:40:45Z

Alright, I looked into this, and the problem was that we were storing the Anchors for the bibliographic references in doc.internal.headers.

However, the internals of Documenter rightfully assume that any Anchor.object in doc.internal.headers is an instance of MarkdownAST.Heading, which has a level attribute. In our case, Anchor.object is an instance of Bibliography.Entry.

The correct thing to do here is for DocumenterCitations to maintain its own AnchorMap for bibliographic references, which doesn't interact with any table of contents.

Fixed in #42

kellertuer changed the title ~~Strange behaviour od DocumenterCitations.jl?~~ Strange behaviour of DocumenterCitations.jl? Jul 13, 2023

kellertuer changed the title ~~Strange behaviour of DocumenterCitations.jl?~~ Canonical=true block throws error Jul 13, 2023

kellertuer changed the title ~~Canonical=true block throws error~~ @contents and any canonical bibliography do not work together Jul 13, 2023

goerz mentioned this issue Jul 13, 2023

Docs compilation error ali-ramadhan/DocumenterCitations.jl#24

Open

goerz added the bug Something isn't working label Jul 14, 2023

goerz mentioned this issue Sep 11, 2023

Compatibility with Documenter 1.0 #3

Merged

goerz mentioned this issue Sep 20, 2023

Fix incompatibility of @bibliography and @contents #42

Merged

goerz closed this as completed in #42 Sep 22, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

@contents and any canonical bibliography do not work together #16

@contents and any canonical bibliography do not work together #16

kellertuer commented Jul 13, 2023 •

edited

Loading

goerz commented Jul 13, 2023

goerz commented Jul 13, 2023

kellertuer commented Jul 13, 2023

kellertuer commented Jul 13, 2023

kellertuer commented Jul 13, 2023 •

edited

Loading

kellertuer commented Jul 13, 2023

fredrikekre commented Jul 13, 2023

kellertuer commented Jul 13, 2023

goerz commented Jul 13, 2023

kellertuer commented Jul 13, 2023

goerz commented Jul 13, 2023

kellertuer commented Jul 13, 2023

goerz commented Jul 13, 2023

kellertuer commented Jul 13, 2023

fredrikekre commented Jul 13, 2023

kellertuer commented Jul 13, 2023

kellertuer commented Jul 13, 2023 •

edited

Loading

goerz commented Jul 13, 2023

kellertuer commented Jul 13, 2023 •

edited

Loading

goerz commented Jul 13, 2023

kellertuer commented Jul 13, 2023

goerz commented Jul 13, 2023

kellertuer commented Jul 13, 2023

goerz commented Jul 13, 2023

goerz commented Jul 13, 2023

mortenpi commented Jul 13, 2023

goerz commented Sep 15, 2023

goerz commented Sep 20, 2023

@contents and any canonical bibliography do not work together #16

@contents and any canonical bibliography do not work together #16

Comments

kellertuer commented Jul 13, 2023 • edited Loading

Scenario 1

Scenario 2

Scenario 3

goerz commented Jul 13, 2023

goerz commented Jul 13, 2023

kellertuer commented Jul 13, 2023

kellertuer commented Jul 13, 2023

kellertuer commented Jul 13, 2023 • edited Loading

kellertuer commented Jul 13, 2023

fredrikekre commented Jul 13, 2023

kellertuer commented Jul 13, 2023

goerz commented Jul 13, 2023

kellertuer commented Jul 13, 2023

goerz commented Jul 13, 2023

kellertuer commented Jul 13, 2023

goerz commented Jul 13, 2023

kellertuer commented Jul 13, 2023

fredrikekre commented Jul 13, 2023

kellertuer commented Jul 13, 2023

kellertuer commented Jul 13, 2023 • edited Loading

goerz commented Jul 13, 2023

kellertuer commented Jul 13, 2023 • edited Loading

goerz commented Jul 13, 2023

kellertuer commented Jul 13, 2023

goerz commented Jul 13, 2023

kellertuer commented Jul 13, 2023

goerz commented Jul 13, 2023

goerz commented Jul 13, 2023

mortenpi commented Jul 13, 2023

goerz commented Sep 15, 2023

goerz commented Sep 20, 2023

kellertuer commented Jul 13, 2023 •

edited

Loading

kellertuer commented Jul 13, 2023 •

edited

Loading

kellertuer commented Jul 13, 2023 •

edited

Loading

kellertuer commented Jul 13, 2023 •

edited

Loading