Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Part 1: doc restructure, introduce categories #1364

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

tomschr
Copy link

@tomschr tomschr commented Nov 12, 2023

Based on the discussions from #1360, this PR contains the following changes for part 1:

  • Introduce the categories "First steps", how-to guides (recipes), addendum, and references based on the Diátaxis doc framework
  • Clean up the first page and move some resources into resources.md file.
  • Add a high level overview on the first page. Tell the reader how the Lark documentation is organized.
  • Minor changes in titles to adhere to the overall doc type

I've compiled it with make html but got some warnings. However, it seems, they exist in the master branch as well.

Let me know what you think, @erezsh.

Open questions

When I worked on this PR, I stumbled upon the following issues:

  • Do we want a table of content in the first page although we have already one on the left sidebar? Should we keep the first page clean? Only difference is the depth of the TOC.
  • Should we present the features on the first page instead of using the link?
  • The doc is a mixture of RST and Markdown. Probably there is a reason for that ("we are get used to Markdown/don't like RST"?).
    Using different formats in one doc makes it hard to stay consistent. Both have a different set of syntax and features. What works with one doesn't work for the other.
    IMHO, this makes it also hard to contribute. But this is for a different PR (if any), but I would really recommend to use only one format.

Result

Screenshot_20231112_171942

Based in the Diátaxis doc framework, we identified
tutorials, how-to guides/recipes, addednum, and
references.

This commit deals with moving the topics into the
"right" category.

Related to issue lark-parser#1360
The first page should be clean and simple. It acts as
a "hub" to let readers select what they need.

Move all links into an additional resources.md file and
add it to addendum.
@tomschr tomschr force-pushed the 1360-doc-restructure branch from 432eda0 to 7bf9f9a Compare November 13, 2023 07:16
Tell the reader how the Lark documentation is organized.
This makes it easier for the reader to choose the necessary
topics.
@tomschr tomschr marked this pull request as ready for review November 13, 2023 08:56
@MegaIng
Copy link
Member

MegaIng commented Nov 15, 2023

  • TOC: Unsure. If it's just mostly the same as the left nav, it's pointless. If it has more depth (and this depth is actually useful. Doesn't look like it for the JSON tutorial for example), then I would say don't get rid of it. Also, can't then left nav get more depth and expand if you click on it?
  • Features list: No opinion from me.
  • With regard to markdown vs rst, check here: Please use Sphinx for documentation #640 . So in some sense it's mostly historic.

Copy link
Member

@MegaIng MegaIng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of this looks quite good, just a few wording choices I would suggest to think a bit more about if we are already modifying them.

docs/how_to_develop.md Outdated Show resolved Hide resolved
docs/how_to_use.md Outdated Show resolved Hide resolved
docs/index.rst Outdated
Comment on lines 30 to 40
-------

How the documentation is organized
----------------------------------

A high-level overview of how it’s organized will help you know where to look for certain things:

.. _Sublime Text & TextMate: https://github.com/lark-parser/lark_syntax
.. _Visual Studio Code: https://github.com/lark-parser/vscode-lark
.. _Intellij & PyCharm: https://github.com/lark-parser/intellij-syntax-highlighting
.. _Vim: https://github.com/lark-parser/vim-lark-syntax
.. _Atom: https://github.com/Alhadis/language-grammars
* Tutorials take you by the hand through a series of steps how to get familiar with Lark. Start here if you’re new to Lark.
* How-to guides are bite-sized, problem-specific solutions to common tasks.
* Addendums covers background information to Lark.
* References contain syntax and semantics reference material for Lark.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I personally don't really like if documentation feels the need to explain how it works. The naming of sections should be good enough that it's obvious. This is IMO the case for Tutorials, Addendum and References. How-to is a bit less clear, Recipes is a better name there IMO.

Even if we keep this list, the first sentence that explains that this is an explanations of sections is definitely not needed and just represents noise.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a fair point. Actually, I had stolen this part from the Django documentation. Maybe for the Django docs it's justified as their work is huge.

I have no problem to remove it, but let's give it a bit more time to see what others think about this.

@tomschr
Copy link
Author

tomschr commented Nov 16, 2023

  • TOC: Unsure. If it's just mostly the same as the left nav, it's pointless. If it has more depth (and this depth is actually useful. Doesn't look like it for the JSON tutorial for example), then I would say don't get rid of it. Also, can't then left nav get more depth and expand if you click on it?

Exactly. I think we can configure Sphinx to show more content for the left sidebar/TOC. In that case, the TOC in the content would be useless and could be removed.

After I researched this issue, it can be done by adding the following lines:

diff --git i/docs/conf.py w/docs/conf.py
index aacf8bb..405af4d 100644
--- i/docs/conf.py
+++ w/docs/conf.py
@@ -102,7 +102,8 @@ html_theme = 'sphinx_rtd_theme'
 # documentation.
 #
 html_theme_options = {
-    'prev_next_buttons_location': 'both'
+    'prev_next_buttons_location': 'both',
+    'collapse_navigation': False,
 }

I see. As a first time contributor, I was puzzled. Usually, writers (and probably contributors as well) don't want to deal with two formats. It's just makes live harder as you have to switch context (MD <-> RST).

I've read the issue you gave me, but I'm still a bit confused. It's for historical reasons, okay. Are we forced to keep it? If not, can we use one format? Or are there any objections?

At this point, we have the following options:

  1. Convert the .md files to .rst to make it consistent.
  2. Stop here and merge the existing PR after all reviews are done. Make the conversion in another PR or keep it as is.

Apart from the above change, I realized there might be another question to consider. The current structure under docs/ is flat. With the current split into these four doc types, we maybe should reflect this change in the structure as well.

For example, we could create docs/references, docs/tutorials/ directories and move the files there. That would make contribution a bit clearer, hopefully. However, it would make the PR a bit bigger. But this idea is independent from the two options above and could be done also in another PR (or not, if you prefer the current structure).

It's your project, I'll give you just some ideas to think about. 🙂 If you think this would be beneficial, I'm willing to work on it.

What do you think?

Remove TOC in content as the left TOC sidebar contains [+] markers
to unfold/fold the deeper structure.
@MegaIng
Copy link
Member

MegaIng commented Nov 16, 2023

I've read the issue you gave me, but I'm still a bit confused. It's for historical reasons, okay. Are we forced to keep it? If not, can we use one format? Or are there any objections?

We can convert the markdown files to rst, I have no objections (erezsh might). The point is more that we don't/didn't consider it worth the effort to convert them. But I don't think that should happen in this PR.

The current structure under docs/ is flat. With the current split into these four doc types, we maybe should reflect this change in the structure as well.

Sure. But IMO this should also be a separate PR to keep them all manageably small.

Let's wait for erezsh to add their opinion.

@tomschr
Copy link
Author

tomschr commented Nov 16, 2023

Thanks MegaIng for your comments!

I agree with you. If we want to go this extra mile, I can do that in separate PRs. I'll wait for erezsh's opinion.

Thanks! 👍

@tomschr
Copy link
Author

tomschr commented Nov 21, 2023

I see no advantages in having two formats at the same time. Just some additional thoughts:

  • Consistency is key. This is the overall parenthesis that holds it together.
  • We should make the life of potential contributors easier, not harder.
  • Having one format, we:
    • can use the full set of syntax. We do not have to deal with a subset or work with limitations of one format.
    • can get rid of some dependencies which are useless after the conversion.
    • looks consistent and clean
  • Conversion is a matter of calling pandoc and fixing the leftovers (we have to do it once).

@tomschr
Copy link
Author

tomschr commented Dec 15, 2023

@MegaIng / @erezsh Anything that I can do to make your life easier? Or anything that's missing?
I know you are probably quite busy before the holiday season, but it would help me if I'm on the right track and if there is still interest to proceed further. 💚

Thank you and have a great vacation.

@erezsh
Copy link
Member

erezsh commented Dec 15, 2023

@tomschr Thanks! Sorry to keep you waiting. I've actually been on vacation these past 3 weeks, and might have a bit more free time in a week or two.

@tomschr
Copy link
Author

tomschr commented Dec 15, 2023

Thank you! Hope the vacation was fantastic! 😄 No problem, have a good day.

@erezsh
Copy link
Member

erezsh commented Dec 23, 2023

Hi @tomschr ,

I got to looking at your branch. There were some changes I liked, and some that I didn't understand. More importantly, looks like your changes from the docs. This is how it looks when I build your branch locally:

image

Also, looks like you intentionally removed all the links from the index page. I think that's wrong. It's arguable how much links should be there, but definitely there should be a few "call to actions" for the most common pages users would like.

@tomschr
Copy link
Author

tomschr commented Dec 23, 2023

@erezsh Thank you for your time! I'm sorry, it seems I had accidentally committed my last experiment. That shouldn't be the case, I've corrected the TOC now. Could you check again?

@tomschr
Copy link
Author

tomschr commented Jan 31, 2024

@erezsh Any additions, corrections, or suggestions from your side?

@erezsh
Copy link
Member

erezsh commented Feb 2, 2024

Sorry, I've been busy. If it was as simple as just merging it, I would have already done so.

I will need to take the time to write my thoughts in detail, and explain what I think needs changing and why.

I will try to get to it soon!

@erezsh
Copy link
Member

erezsh commented Feb 12, 2024

@tomschr Sorry again for the long wait. At an overview, I agree with these changes, but not with all of the choices you made.

A few points:

  • Examples don't belong in "first steps". The way I see it, first steps should be reserved for tutorials, beginner guides, and installation instructions (if we had any). Maybe it should contain the "How to use Lark" section too.
    It definitely shouldn't contain a whole section called "Grammar Composition", which is an advanted topic, and definitely isn't important for beginners.

( I might be persuaded that it should contain the beginner examples, but I think examples should be together. Probably in their own separate section )

  • I think "How to use Lark" is a better title than "Working with Lark"

  • The portion you moved to "Resources" needs editing to adjust to the new structure

  • Now that I think about it, "Parsers" section belongs in reference.

  • Possibly "features" also belongs in the reference section.

@tomschr
Copy link
Author

tomschr commented Feb 12, 2024

Thank you for your response!

After such a long time it's hard to remember all the details that I made and the reasons behind it. 😆

Examples don't belong in "first steps". The way I see it, first steps should be reserved for tutorials, beginner guides, and installation instructions (if we had any).

That is true, but as we don't have these guides yet I had to put at least something. Keep in mind, we are just migrating from one structure to a new one. It's an intermediate step, it's not the end. We can always adjust it later.

If you prefer to move these things, I can do that. But then we maybe end up with only one article under "First steps".

I think "How to use Lark" is a better title than "Working with Lark"

I did it on purpose, because "working" is a verb and means action. The problem with starting with "How to..." is, it propagates quite fast and you have other articles with very similar titles. That makes it harder for the reader to distinguish them. The difference in each title appears too late. In my work as a professional writer, I try to avoid starting with "How to..." if possible.

Now that I think about it, "Parsers" section belongs in reference.
Possibly "features" also belongs in the reference section.

That shouldn't be a problem to move them. I'll do that later.

The portion you moved to "Resources" needs editing to adjust to the new structure.

I'm not sure I got that. Any examples?

Thanks for your time!

@erezsh
Copy link
Member

erezsh commented Feb 12, 2024

No need to thank me! I'm glad you're trying to contribute to Lark, and like I said, I'm sorry it took so long.

After such a long time it's hard to remember all the details that I made and the reasons behind it

Now you get a glimpse of how I feel about docs I wrote 4 years ago :)

But then we maybe end up with only one article under "First steps".

That is acceptable. I think it's better than a mis-categorization, that might confuse a newcomer.

We can also include the "How to write a DSL" tutorial as an external link.

I try to avoid starting with "How to..." if possible.

Maybe there's a third option. Like "A guide to using Lark" or "best-practices for using Lark", etc.

I'm not sure I got that. Any examples?

Well, I wrote it as a "quick-links" section. Since you moved it as-is, it contains links to sections that are not really resources, like "philosophy", "features", and "examples". Also, some of these "resources" are either guides, or examples, or something in between, and would probably fit there better than to "resources".

* Move some sections
  - parsers -> Reference
  - features -> Reference
  - examples/index -> How to guides
@tomschr
Copy link
Author

tomschr commented Feb 20, 2024

No need to thank me! I'm glad you're trying to contribute to Lark, and like I said, I'm sorry it took so long.

Haha, no problem. Now I had to say sorry as I wanted to work on it last week.

Now you get a glimpse of how I feel about docs I wrote 4 years ago :)

Haha, very true!

[... suggestions to move sections...]

This is done in commit c3470e1 now.

That is acceptable. I think it's better than a mis-categorization, that might confuse a newcomer.

Okay, no problem.

Maybe there's a third option. Like "A guide to using Lark" or "best-practices for using Lark", etc.

Hmn. 🤔 My idea was all about consistency: References and conceptual topics are in nominal style ("Grammar Reference", "JSON parser" etc.) whereas task-oriented topics uses verbs ("Contributing to Lark", "Working with Lark" etc.). Perhaps "Working" isn't the best verb and it can be improved.

Your suggestion "A guide to using Lark" or "Best-practices for using Lark" just adds more fluff to the title. 😉 From my point of view, it's just too wordy. 😉 Whether it's a guide or a section doesn't really matter to the user. It's an implementation detail. The readers want to know what is the topic about and they want to understand it fast. Anything that distracts the user reduces their understanding. 😉

Just wanted to tell you my reasoning and why I think it matters. If you really want to go with your suggestion, I can do that change of course. 🙂

I'm not sure I got that. Any examples?

Well, I wrote it as a "quick-links" section. Since you moved it as-is, it contains links to sections that are not really resources, like "philosophy", "features", and "examples". Also, some of these "resources" are either guides, or examples, or something in between, and would probably fit there better than to "resources".

Yes, I moved it "as-is" to reduce the impact of the changes. Remember, it's part 1. In this part I deal only with restructuring, maybe correcting titles here or there, but not touching the overall text. This could be done at a later point.

But perhaps I misunderstood you.

@erezsh
Copy link
Member

erezsh commented Feb 20, 2024

My idea was all about consistency

I appreciate consistency, but I think clarity is even more important.

"Working with Lark" isn't a terrible title, but I feel that it doesn't communicate well what the content might be. If I was looking for best-practices, I might miss it if I was trying to scan the menu.

Anyway, I don't feel that strongly about it.

Yes, I moved it "as-is" to reduce the impact of the changes

It's good to keep the changes small. But I don't want the docs to end up looking wrong or broken, even if it's temporary.

If you're still confused, I can try to fix that page myself. But it needs doing.

@tomschr
Copy link
Author

tomschr commented Feb 20, 2024

"Working with Lark" isn't a terrible title, but I feel that it doesn't communicate well what the content might be. If I was looking for best-practices, I might miss it if I was trying to scan the menu.

I see what you mean. But the titles work in combination with the guide title. I think we are talking about different things:

  • Task-oriented/"recipes"/"first steps": more for beginners. Explains the topic more in a step-by-step style (if possible). Usually more task-based. Could be a "Working with Lark"
  • "Best practices": more for medium users to experts. Explains the topic in a broader context. Could be something like "Optimizing your grammar" (hypothetical).

From what I see, I think these are different guides. If you have this distinction, you can focus on your titles. Beginner will be attracted by "First steps", more advanced users will gravitate the Best Practices or other guides.

But I don't want the docs to end up looking wrong or broken, even if it's temporary.

Sure, absolutely. 👍

If you're still confused, I can try to fix that page myself. But it needs doing.

If it's not too much to ask for, please go ahead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants