xenops not working with org mode 9.7.11 (Emacs 29.3)

[RaghuNaik]

I get the following error when I open an org file

> File mode specification error: (error Selecting deleted buffer) 
> Error during redisplay: (jit-lock-function 1) signaled (error "Selecting deleted buffer")

When I start emacs from the command line with emacs --debug-init after putting

(jit-lock-debug-mode 1)
(setq debug-on-error t)

in my init file, I get the following backlog trace:

Debugger entered--Lisp error: (error "Selecting deleted buffer") org-element--substring((src-block (:standard-properties [4 4 nil nil 86 1 nil planning element t nil nil nil nil #<killed buffer> nil nil (section (:standard-properties [4 4 4 86 86 0 nil section element t nil 4 86 nil #<killed buffer> nil nil (headline ...)]))] :language "R" :switches nil :parameters nil :number-lines nil :preserve-indent nil :retain-labels t :use-labels t :label-fmt nil :value [org-element-deferred org-element--unescape-substring (14 71) t])) 14 71) org-element--unescape-substring((src-block (:standard-properties [4 4 nil nil 86 1 nil planning element t nil nil nil nil #<killed buffer> nil nil (section (:standard-properties [4 4 4 86 86 0 nil section element t nil 4 86 nil #<killed buffer> nil nil (headline ...)]))] :language "R" :switches nil :parameters nil :number-lines nil :preserve-indent nil :retain-labels t :use-labels t :label-fmt nil :value [org-element-deferred org-element--unescape-substring (14 71) t])) 14 71) apply(org-element--unescape-substring (src-block (:standard-properties [4 4 nil nil 86 1 nil planning element t nil nil nil nil #<killed buffer> nil nil (section (:standard-properties [4 4 4 86 86 0 nil section element t nil 4 86 nil #<killed buffer> nil nil (headline ...)]))] :language "R" :switches nil :parameters nil :number-lines nil :preserve-indent nil :retain-labels t :use-labels t :label-fmt nil :value [org-element-deferred org-element--unescape-substring (14 71) t])) (14 71)) org-element--property(:value (src-block (:standard-properties [4 4 nil nil 86 1 nil planning element t nil nil nil nil #<killed buffer> nil nil (section (:standard-properties [4 4 4 86 86 0 nil section element t nil 4 86 nil #<killed buffer> nil nil (headline ...)]))] :language "R" :switches nil :parameters nil :number-lines nil :preserve-indent nil :retain-labels t :use-labels t :label-fmt nil :value [org-element-deferred org-element--unescape-substring (14 71) t])) nil nil) org-element-property(:value (src-block (:standard-properties [4 4 nil nil 86 1 nil planning element t nil nil nil nil #<killed buffer> nil nil (section (:standard-properties [4 4 4 86 86 0 nil section element t nil 4 86 nil #<killed buffer> nil nil (headline ...)]))] :language "R" :switches nil :parameters nil :number-lines nil :preserve-indent nil :retain-labels t :use-labels t :label-fmt nil :value [org-element-deferred org-element--unescape-substring (14 71) t]))) org-babel--normalize-body((src-block (:standard-properties [4 4 nil nil 86 1 nil planning element t nil nil nil nil #<killed buffer> nil nil (section (:standard-properties [4 4 4 86 86 0 nil section element t nil 4 86 nil #<killed buffer> nil nil (headline ...)]))] :language "R" :switches nil :parameters nil :number-lines nil :preserve-indent nil :retain-labels t :use-labels t :label-fmt nil :value [org-element-deferred org-element--unescape-substring (14 71) t]))) org-babel-get-src-block-info(light (src-block (:standard-properties [4 4 nil nil 86 1 nil planning element t nil nil nil nil #<killed buffer> nil nil (section (:standard-properties [4 4 4 86 86 0 nil section element t nil 4 86 nil #<killed buffer> nil nil (headline ...)]))] :language "R" :switches nil :parameters nil :number-lines nil :preserve-indent nil :retain-labels t :use-labels t :label-fmt nil :value [org-element-deferred org-element--unescape-substring (14 71) t]))) xenops-src-parse-at-point() funcall(xenops-src-parse-at-point) xenops-util-first-result(funcall (xenops-math-parse-block-element-at-point xenops-math-parse-inline-element-at-point xenops-math-parse-table-at-point xenops-image-parse-at-point xenops-footnote-parse-at-point xenops-minted-parse-at-point xenops-src-parse-at-point)) xenops-parse-any-element-at-point((xenops-math-parse-block-element-at-point xenops-math-parse-inline-element-at-point xenops-math-parse-table-at-point xenops-image-parse-at-point xenops-footnote-parse-at-point xenops-minted-parse-at-point xenops-src-parse-at-point)) xenops-apply-parse-next-element(653 (xenops-math-parse-block-element-at-point xenops-math-parse-inline-element-at-point xenops-math-parse-table-at-point xenops-image-parse-at-point xenops-footnote-parse-at-point xenops-minted-parse-at-point xenops-src-parse-at-point)) xenops-apply-handlers-over-region((xenops-math-render xenops-image-render xenops-footnote-render) 1 653 nil nil) xenops-apply-operations((render)) xenops-render() xenops-render-if-cached() xenops-mode() run-hooks(change-major-mode-after-body-hook text-mode-hook outline-mode-hook org-mode-hook) apply(run-hooks (change-major-mode-after-body-hook text-mode-hook outline-mode-hook org-mode-hook)) run-mode-hooks(org-mode-hook) org-mode() set-auto-mode-0(org-mode nil) set-auto-mode--apply-alist((("\\.Rmd" . poly-quarto-mode) ("\\.Rmd" . poly-markdown+r-mode) ("\\.Rnw" . poly-noweb+r-mode) ("\\.Snw" . poly-noweb+r-mode) ("\\.md" . poly-markdown-mode) ("\\.djhtml\\'" . web-mode) ("\\.ctp\\'" . web-mode) ("\\.ejs\\'" . web-mode) ("\\.eco\\'" . web-mode) ("\\.hbs\\'" . web-mode) ("\\.handlebars\\'" . web-mode) ("\\.mustache\\'" . web-mode) ("\\.erb\\'" . web-mode) ("\\.eex\\'" . web-mode) ("\\.as[cp]x?\\'" . web-mode) ("\\.[gj]sp\\'" . web-mode) ("\\.htm\\'" . web-mode) ("\\.html\\'" . web-mode) ("\\.twig\\'" . web-mode) ("\\.tpl\\'" . web-mode) ("\\.phtml\\'" . web-mode) ("SConscript\\'" . python-mode) ("SConstruct\\'" . python-mode) ("\\.pug$" . pug-mode) ("\\.\\(pum\\|puml\\)\\'" . plantuml-mode) ("\\.epub\\'" . nov-mode) ("\\.mdx\\'" . markdown-mode) ("\\.mdk\\'" . markdown-mode) ("\\.mkd\\'" . markdown-mode) ("Cask\\'" . emacs-lisp-mode) ("\\*.el\\'" . emacs-lisp-mode) ("\\.[Jj][Mm][Dd]\\'" . ess-jags-mode) ("\\.[Jj][Oo][Gg]\\'" . ess-jags-mode) ("\\.[Rr]out" . R-transcript-mode) ("\\.Sout" . S-transcript-mode) ("\\.ado\\'" . STA-mode) ("\\.do\\'" . STA-mode) ("\\.lsp\\'" . XLS-mode) ("\\.hat\\'" . omegahat-mode) ("\\.omg\\'" . omegahat-mode) ("CITATION\\'" . R-mode) ("NAMESPACE\\'" . R-mode) ("\\.[rR]profile\\'" . R-mode) ("\\.[sS]nw\\'" . Snw-mode) ("\\.[rR]nw\\'" . Rnw-mode) ("\\.[rR]\\'" . R-mode) ("\\.SSC\\'" . S-mode) ("\\.ssc\\'" . S-mode) ("\\.[qsS]\\'" . S-mode) ("/R/.*\\.q\\'" . R-mode) ...) nil nil) set-auto-mode() normal-mode(t) after-find-file(nil t) find-file-noselect-1(#<buffer 20240913T124523--impulse-response-from-johnston__cointegration_r_timeseries_vars.org> "/mnt/hgfs/r_projects/review/notes/20240913T124523-..." nil nil "/mnt/hgfs/r_projects/review/notes/20240913T124523-..." (28177 54)) #<subr find-file-noselect>("/mnt/hgfs/r_projects/review/notes/20240913T124523-..." nil nil nil) apply(#<subr find-file-noselect> "/mnt/hgfs/r_projects/review/notes/20240913T124523-..." (nil nil nil)) editorconfig--advice-find-file-noselect(#<subr find-file-noselect> "/mnt/hgfs/r_projects/review/notes/20240913T124523-..." nil nil nil) apply(editorconfig--advice-find-file-noselect #<subr find-file-noselect> ("/mnt/hgfs/r_projects/review/notes/20240913T124523-..." nil nil nil)) find-file-noselect("/mnt/hgfs/r_projects/review/notes/20240913T124523-..." nil nil nil) find-file("/mnt/hgfs/r_projects/review/notes/20240913T124523-...") find-file-existing("/mnt/hgfs/r_projects/review/notes/20240913T124523-...") (lambda (&rest ignore) (find-file-existing "/mnt/hgfs/r_projects/review/notes/20240913T124523-..."))((push-button :args nil :value "/mnt/hgfs/r_projects/review/notes/20240913T124523-..." :action (lambda (&rest ignore) (find-file-existing "/mnt/hgfs/r_projects/review/notes/20240913T124523-...")) :mouse-face highlight :follow-link "\15" :button-prefix "" :button-suffix "" :button-face nil :format "%[%t%]" :button-overlay #<overlay from 507 to 625 in *spacemacs*> :from #<marker (moves after insertion) at 507 in *spacemacs*> :to #<marker at 625 in *spacemacs*>) nil) widget-apply-action((push-button :args nil :value "/mnt/hgfs/r_projects/review/notes/20240913T124523-..." :action (lambda (&rest ignore) (find-file-existing "/mnt/hgfs/r_projects/review/notes/20240913T124523-...")) :mouse-face highlight :follow-link "\15" :button-prefix "" :button-suffix "" :button-face nil :format "%[%t%]" :button-overlay #<overlay from 507 to 625 in *spacemacs*> :from #<marker (moves after insertion) at 507 in *spacemacs*> :to #<marker at 625 in *spacemacs*>) nil) widget-button-press(507) (if (not (re-search-forward (concat "^ +" nr-string "[0-9]* +. ") (save-excursion (goto-char (window-end)) (forward-line -1) (point)) 'noerror)) (progn (goto-char prev-point) (let (message-log-max) (message "Couldn't find startup list number: %s" spacemacs-buffer--startup-list-number))) (message "Opening file/dir: %s" (widget-value (widget-at (point)))) (widget-button-press (point))) (let ((prev-point (point))) (goto-char (window-start)) (if (not (re-search-forward (concat "^ +" nr-string "[0-9]* +. ") (save-excursion (goto-char (window-end)) (forward-line -1) (point)) 'noerror)) (progn (goto-char prev-point) (let (message-log-max) (message "Couldn't find startup list number: %s" spacemacs-buffer--startup-list-number))) (message "Opening file/dir: %s" (widget-value (widget-at (point)))) (widget-button-press (point)))) spacemacs-buffer/jump-to-line-starting-with-nr-space("2") spacemacs-buffer/stop-waiting-for-additional-numbers() apply(spacemacs-buffer/stop-waiting-for-additional-numbers nil) timer-event-handler([t 0 0 400000 nil spacemacs-buffer/stop-waiting-for-additional-numbers nil idle 0 nil])

I had earlier reported this error to the Spacemacs team here

Is anyone else experiencing this? Thanks

[SuChengHao]

I have the same problem

[dandavison]

Hi, I'm sorry I don't have much time for xenops development. Xenops was never really intended for use with org-mode: it is intended as a standalone emacs LaTeX editor.

[fnussbaum]

I have turned my attempted fix from syl20bnr/spacemacs#16577 (comment) into a PR (#74). It is not really tested, as I don't use xenops-mode myself. Perhaps you could try this out? @RaghuNaik @SuChengHao

@dandavison Of course feel free to disregard this if you prefer to not support Org anymore. I have also opened #75 which fixes a minor annoyance I had when trying to test #74.

[RaghuNaik]

Thanks @fnussbaum. I did test out the fix. It does not solve the issues. Though the errors go away, equations are not processed and formatted now. Also emacs freezes on startup immediately after introducing the fix.

[dandavison]

Hi @fnussbaum @RaghuNaik again, sorry I'm not being much use here, and thanks for the efforts. Note that xenops does have a test suite (make test), although it looks like I have not activated GitHub CI.

if you prefer to not support Org anymore

I shouldn't have promised that it would work in org; during development I was using it exclusively as a standalone emacs LaTeX editor.

[RaghuNaik]

Hi @dandavison thanks for your excellent package. I must say that xenops was working perfectly fine before the last org update 9.7.11. I think today org 9.7.12 has been pushed out. Possibly this may solve the issues.

[fnussbaum]

Though the errors go away, equations are not processed and formatted now.

Previewing formulas as svg images seems to work fine for me with #74 applied, with both Org 9.7.11 and 9.7.12. Did you mean that by processing and formatting, or something else?

I used the following snippet to test:

#+begin_src latex
Equation environment:
\begin{equation}
\alpha + \beta
\end{equation}
Inline math: $\alpha - \beta$
#+end_src

After moving the cursor in and out of the environment and inline math, they are replaced by correctly rendered images for me.

Also emacs freezes on startup immediately after introducing the fix.

I cannot reproduce this either.

In case you would like to further debug this on your system, I would suggest, for example:

1. Remove xenops-mode from your org-mode-hook. This should fix the freezing on startup if that was indeed caused by some xenops/org interaction.
2. Manually enable xenops-mode in an org buffer. If this freezes, you could try getting a backtrace with debug-on-quit and hitting C-g repeatedly.

Note that xenops does have a test suite (make test)

Sounds good, thanks, I'll try to get it to run on my system when I find some time.

[RaghuNaik]

I commented out the hooks and then enabled xenops-mode manually initially. The formatted equations svg images come through now.

Now commenting out the hooks is not required for getting the formatted equations.

Thanks @fnussbaum.