Any Plans to support i18n?

[Roger-luo]

Hi JuliaDocs team,

Our old Chinese document (v0.3) in JuliaCN is out of date, we are planning to have a new Chinese document for Julia. Since, currently, julia source is using Documenter.jl, I'm wondering if you have any suggestions on i18n with Documenter.jl? (the Julia-i18n approach may not work for Chinese users, since that web service could be hard to reach in China and it would be more convenient to let user access docs inside REPL).

the original English documents collects docs from each stdlib, so we have to edit julia source (the doc directory and doc directory in stdlib), it may mess the original julia fork. It might be cleaner to have separated files/dir for Chinese docs. Previously, since all the docs are in doc with readthedocs services, we simply copied it, but it seems not working anymore with Markdown in the latest version.

We would like to hear your suggestions.

Bests,
JuliaCN team

[mortenpi]

Definitely no objections to having Documenter help with i18n as much as possible, although there's nothing specific on the roadmap at the moment. Depending on what the specific features might be that you need, it might make sense to implement them as a plugin through a separate package (DocumenterI18N?).

Some quick thoughts:

• For the Julia manual pages, setting up a separate repository with the translated Markdown pages should be quite easy. And then you can build the translated docs with Documenter from scratch. Some of the .md files of the manual live under stdlib/ in the Julia repo, so you'd need to copy those over as well.

• Docstrings are a bit harder since they come from the binary. But you can always programmatically access and edit docstrings on the fly, so it shouldn't be hard to overwrite them before calling makedocs.

• The docstrings could probably be overwritten in a package as well, which would allow i18n of the REPL (e.g. the user would start a session with e.g. using JuliaZH, and that would load translated docstrings into Base). To facilitate this, some of this work might have to happen in Base (I am not that familiar with the docsystem code in Base).

• I guess it would be nice if Documenter(I18N) would keep track of whether a page / docstring has gotten out of date though?

[Roger-luo]

We currently using this JuliaCN.jl (still work in progress) to overwrite docstring in binary so we could support i18n in REPL, and translate documents in docs and use current functions of Documenter.jl to generate Chinese documentations. I guess we had the same idea.

But it would be great if there is buttons that can be generated by Documenter, like in readthedocs, you can switch English or other languages by click buttons.

And yes, it would be very convenient to track out of date, since not all translator knows well about the doc system and git.

[Roger-luo]

@mortenpi Okay, I found another problem here. When I was trying to add translation against its English version, I cannot use HTML comments at the moment due to #674

Since this is used quite frequently, and for future convenience I'm think adding a syntax for i18n could make this easier, e.g

```@lang en
original markdown
```

```@lang zh
Chinese translation
```

```@lang fr
Franch translation
```

And by default, the outer markdown contents is @lang en (means the default language tag is en):

(some English contents)

will be equivalent to

```@lang en
(some English contents)
```

And the default language can be set by @meta:

```@meta
Language = :zh
```

When making the docs, there should be an option: language, it can be set to :all/:default and then all the contents will be built (different language will be in different folders, e.g English is in html/en, Chinese is in html/zh), or just a single language en (default), then it builds with current behavior.

# this builds only contents with tag zh
makedocs(
    modules = [A, B, ...],
    language=:zh,
)

# this builds all contents in separated folder
makedocs(
    modules = [A, B, ...],
    language=:all,
)

# this builds all contents in separated folder
makedocs(
    modules = [A, B, ...],
    # language is set to :en by default
)

[mortenpi]

My initial reaction is that this gets a bit unwieldy. Having separate .md files for separate languages might be easier. But, of course, ideally you would want to rewrite only the text parts and not duplicate non-text content. But how do you decide which blocks get the language tag and which ones are universal? (I don't have any better suggestions either though.)

Nevertheless, if you would like to try to implement this: it would need a new expander (src/Expanders.jl) that would search for the at-lang blocks, and the expander would then modify the documenter tree somehow. But that's about as much guidance I can give at the moment. You are welcome to bug me here, on Slack or on Gitter though.

It would be helpful if you would try implementing this as a Documenter plugin though and report back about what should be refactored in Documenter and how. It would probably take a while to get to a really working extension package, but it would really help guide Documenter's development I think.

[Roger-luo]

Oh, sorry I forgot to mention, we are working on it here: https://github.com/Roger-luo/Localize.jl

The basic methods have changed. We are using transifex to track the original markdown files now. we are planning to figure out how to dump docstrings to a file format that is supported by transifex:

Roger-luo/Localize.jl#7
JuliaCN/JuliaZH.jl#18

I totally agree this should be some kind of extension to Documenter.jl. We will use English for Localize.jl's development. And please feel free to help us make such an extension and I will check how to make an expander too.

Currently, the JuliaZH.jl simply use module system. However, if a package want to support multiple languages with Documenter in their gh-pages, this does not seem possible at the moment.

I was thinking about to have an environment variable to describe the current language in ENV (or maybe just read it from the system for users)

By separately run make.jl with different environment variables, we could have multiple language support in different folders in the gh-pages branch (or the build folder), like build/en_US, build/zh_CN, etc. without modifying the original Documenter I guess.

[odow]

Closing as a duplicate of #658. Let's keep all discussion for translations to one issue.