Add profiling for incremental compilation

[hooyuser]

Motivation

Incremental compilation is a critical part of the writing and editing workflow. Profiling incremental compilation can help identify hotspots, optimize code, and debug issues like unintentional failures in Typst's caching system (e.g. typst/typst#5046).

Since Typst CLI already supports this feature with typst watch --timings, it would be beneficial for Tinymist to achieve functional parity. This feature can significantly improve the user experience, especially for most ongoing projects where most compilation time is incremental.

Description

Introduce a profiling mechanism for incremental compilation in Tinymist. To maintain compatibility, the existing profiling behavior (referred to as "Once Mode") can be retained. In addition, a new "Incremental Mode" can be introduced, enabling users to profile and monitor incremental compilations more effectively.

To make the feature user-friendly, new buttons such as "Update Profile" and "Stop Profile" could be added next to the existing "Profile" button (final naming to be determined, e.g., "Profile++" for brevity) .

To optimize UI space, these buttons would dynamically appear only after clicking "Profile" when profiling becomes active, and would be dynamically hidden after clicking "Stop Profile" when profiling ends.

Functionality Overview:

• Profile: Clears existing profiling data, creates a new compilation environment, and conducts an initial compilation with profiling information.
• Update Profile: Conducts incremental compilation with profiling information, displaying the profiling data in a newly opened window.
• Stop Profile: Clears the profiling compilation environment.

Or alternatively, we can only keep two buttons: "Profile++" and "Stop Profile".

Functionality Overview:

• Profile++: If profiling is not active, this button initializes a new compilation environment and performs an initial compilation with profiling information. If profiling is already active, it conducts incremental compilation with profiling information. Then display the profiling result in a newly opened window.
• Stop Profile: Clears the profiling compilation environment and ends the profiling session.

This mechanism makes large document editing more future-proof, with opportunities for further enhancements.

Examples

• A user is editing a large document and notices slow incremental compilation. They use the "incremental mode" profiling to pinpoint bottlenecks and improve the workflow.
• During editing, the profiling highlights inefficiencies in Typst's caching mechanism, enabling developers to debug and optimize.

Questions

• How should users switch between "Once Mode" and "Incremental Mode"? As a first implementation, would it be sufficient to provide a toggle in the settings?
• What are the best naming conventions for the new buttons to maintain clarity and conciseness in the UI?
• Should we design a mechanism to persistently store profiling data, silently generate profiling data, and implement functionality similar to typst watch --timings? How can we achieve better profiling data management overall?
• Is there any chance to enable click-to-jump functionality for navigating to source code when viewing the flame graph of profiling results?

[Myriad-Dreamin]

What about not distinguishing "Once Mode" and "Incremental Mode" for low-level design?

• Start Action: When we click the "start" button or trigger the profile command, it starts. It should open the trace UI beside, which provides two additional buttons, "update" and "stop".
• Update Action: When we click the "update" button, it trigger an incremental compilation to trace.
• Stop Action: When we click the "stop" button, it finishes collection and opens the perfetto.

We can provide an "Once Mode" only in VS Code, which triggers the "start" and then triggers "stop" at once.

This would help developing customized interface in other editors.

[hooyuser]

I think your proposed low-level design with start, update, and stop actions is really neat and well-structured.

As for the high-level UI, my proposed design may not be fully optimized—it mainly focuses on minimizing deviations from the old UI of the legacy profiling system. That could make sense as a starting point. If we're open to not perfectly replicating the legacy behavior, there are definitely more possibilities to be explored.

[Myriad-Dreamin]

Specifically, the low-level actions are the interfaces at language server side to expose to editors.
If so, for high-level actions at edtior side, we can:

• codelens: we still provide an only "Profile" codelen. The difference is that we add a drop down to select which mode: "Once" or "Incremental". The default selected drop down is "Once", so people can press enter to profile in "Once Mode".
• commands: Beside old "Profile" command, we add an extra "Profile Incrementally" command, which correspond to profile in "Once Mode" and "Incremental Mode".
• update/stop: the trace UI includes an extra "update/stop" page. In the page, the user can update (step) or stop incremental compilation. After stopping the incremental compilation, the trace UI goes to the old page, which shows perfetto for us.

In short, we'll keep perfect compatibility and add an extra dropdown to select mode, an extra command to enter incremental mode by command, and an extra page in trace UI to control incremental compilation.

---

Should we design a mechanism to persistently store profiling data, silently generate profiling data, and implement functionality similar to typst watch --timings? How can we achieve better profiling data management overall?

It should be an separated feature request. Since we've already have typst watch --timings to generate the trace file, it is not necessary to provide exact same (duplicated) function in the editor. At least it sounds like a low-priority feature which doesn't give much benefit.

---

Is there any chance to enable click-to-jump functionality for navigating to source code when viewing the flame graph of profiling results?

It should be an separated feature request as well. But answer it shortly here, it is probably hard if perfetto doesn't provide customization points to navigate source code.

[hooyuser]

Yes, I think this is a great starting point to enable Tinymist to profile incremental compilation for the first time! Could be a real lifesaver.

I also have some ideas that may not be fully developed yet. That's all for a more enhanced version of profiling in Tinymist, though they might require more effort and may not be feasible for the initial iteration.

We can use the "Start Action" to generate profiling results for the first full compilation and the "Update Action" to generate profiling results for incremental compilations, reflecting changes in the Typst file since the last "Start Action / Update Action." It would be very useful to display the compilation time for each "Start Action / Update Action" in the trace UI. Additionally, I believe it would be more flexible for users to have the option to open the Perfetto viewer for specific profiling results from either action.

A common use case is investigating exceptionally slow incremental compilations through random editing — basically just some naive attempts in suspicious code region. If a user notices abnormal compilation times, they might want to open the corresponding profiling result immediately to analyze the issue without stopping the entire profiling session. And he can also compares it with the profiling results that exhibit normal compilation times. Ideally, the Perfetto viewer should work independently of the profiling session. So users can view flame graphs and other profiling data without stopping the profiling process.

Allowing keeping multiple profiling results also raises some concerns about the memory footprint of profiling. Incremental compilations are significantly faster than full compilations—often by at least an order of magnitude. But as observed in typst watch --timings, the sizes of produced tracing files (record-n.json) typically don't differ that much. For large Typst files, each record-n.json can take up 100-500MB, making it not so practical to keep all tracing data in memory. If in-memory storage is necessary, some mechanisms like LRU cache could be implemented to manage memory more efficiently. Otherwise, following what typst watch --timings does by storing profiling data on disk seems like a reasonable and scalable approach.

Just my two cents. This functionality can totally surpass what Typst CLI can offer, not only because of better IDE integration but also due to its more flexible controls. I’ve noticed a lot of complaints about performance in the Typst community (e.g., some recent Tinymist issue), even though typst --timings has been available for a long time. One reason for this might be that the CLI’s profiling functionality is never user-friendly for the average Typst user, which makes it useful only "in theory" but not in practice.

A robust and easy-to-use perf profiler is a critical infrastructure for Typst ecosystem because it directly ties into one of Typst's most appealing features: real-time preview. Any improvements to this infrastructure can encourage more users to understand and address performance issues effectively. As Tinymist continues to establish itself as the de facto standard Typst IDE, I strongly believe this will have a profound impact on the entire Typst community.

[Myriad-Dreamin]

Great. I've read all your message, the ideas inside sound interesting, but I don't give much comment on far ideas since I'm a bit tired. Let's do them step by step.

record-n.json can take up 100-500MB

I think perfetto accepts .json.gz directly. Surely, the tracing data can be heavily compressed. And this is an easy but effect optimization we can perform.