Julia Snail

Snail is a development environment and REPL interaction package for Julia in the spirit of Common Lisp’s SLIME and Clojure’s CIDER. It enables convenient and dynamic REPL-driven development.

Snail works on platforms which support libvterm or Eat, which currently means Unix-like systems. It should also work on Windows using WSL.

Refer to the changelog for release notes.

• Features
• Demo
• Installation
  • vterm
  • Eat
  • Snail
  • Other considerations
• Configuration
  • use-package setup
  • Manual setup
  • display-buffer-alist notes
  • Other customizations
• Usage
  • Basics
  • Multiple Julia versions
  • Multiple REPLs
  • Remote REPLs
    • julia-snail-executable and the remote shell path
    • Remote environment setup
    • Errors mentioning No file notification program found
  • Docker container REPLs
  • Extra Julia command-line arguments
  • Module-nested includes
  • Documentation lookup
  • Multimedia and plotting
  • code-cells integration (notebook mode)
• Extensions
  • REPL history
  • Formatter
  • Ob-Julia
  • Debug
• Future improvements

Features

• REPL display: Snail uses advanced terminal emulators (libvterm with Emacs bindings or Eat) to display Julia’s native REPL. As a result, the REPL has good performance and far fewer display glitches than attempting to run the REPL in an Emacs-native term.el buffer.
• REPL interaction: Snail provides a bridge between Julia code and a Julia process running in a REPL. The bridge allows Emacs to interact with and introspect the Julia image. Among other things, this allows loading entire files and individual functions into running Julia processes.
• Remote REPLs: Julia sessions on remote machines work transparently with Snail using SSH and Emacs Tramp.
• Multimedia and plotting: Snail can display Julia graphics in graphical Emacs instances using packages like Plots and Gadfly.
• Cross-referencing: Snail is integrated with the built-in Emacs xref system. When a Snail session is active, it supports jumping to definitions of functions and macros loaded in the session.
• Completion: Snail is also integrated with the built-in Emacs completion-at-point facility. Provided it is configured with the company-capf backend, company-mode completion will also work (this should be the case by default).
• Parser: Snail uses CSTParser, a full-featured Julia parser, to infer the structure of source files and to enable features which require an understanding of code context, especially the module in which a particular piece of code lives. This enables awareness of the current module for completion and cross-referencing purposes.

Demo

screencast-2021-08-06.mp4

Installation

Julia versions >1.6.0 should all work with Snail.

Snail’s Julia-side dependencies will automatically be installed when it starts, and will stay out of your way using Julia’s LOAD_PATH mechanism.

On the Emacs side, you must install one of the supported high-performance terminal emulators to use with the Julia REPL: either vterm or Eat.

vterm

1. Make sure you have Emacs 26.2 or later, compiled with module support (--with-modules). Check the value of module-file-suffix: it should be non-nil. (This is currently a default compile-time option Emacs distributed with Homebrew.)
2. Install libvterm. It is available in Homebrew and Ubuntu 19.10, and in source form on other systems.
3. Install emacs-libvterm using your Emacs package manager. It is available from MELPA as vterm, so use something like (package-install 'vterm) or (use-package vterm :ensure t). It is important to do this step separately from the julia-snail installation, as you may run into problems with the Emacs package manager and byte-compiler!
4. Verify that vterm works by running M-x vterm to start a shell. It should display a nice terminal buffer. You may find it useful to customize and configure vterm.

Eat

Eat requires Emacs 28.1 or later. Install and configure it. The following use-package example should help:

(add-to-list 'package-archives '("nongnu" . "https://elpa.nongnu.org/nongnu/") t)

(use-package eat
  :pin nongnu
  :custom
  (eat-kill-buffer-on-exit t)
  :config
  (delete [?\C-u] eat-semi-char-non-bound-keys) ; make C-u work in Eat terminals like in normal terminals
  (delete [?\C-g] eat-semi-char-non-bound-keys) ; ditto for C-g
  (eat-update-semi-char-mode-map)
  ;; XXX: Awkward workaround for the need to call eat-reload after changing Eat's keymaps,
  ;; but reloading from :config section causes infinite recursion because :config wraps with-eval-after-load.
  (defvar eat--prevent-use-package-config-recursion nil)
  (unless eat--prevent-use-package-config-recursion
    (setq eat--prevent-use-package-config-recursion t)
    (eat-reload))
  (makunbound 'eat--prevent-use-package-config-recursion)
  )

Snail

Install julia-snail using your Emacs package manager (see below for a sample use-package invocation). It is available on MELPA and MELPA Stable.

Other considerations

Optionally, install markdown-mode to improve documentation buffer display.

Because Julia supports Unicode identifiers and uses them for mathematical symbols, it is also a good idea to double-check that your Emacs is cleanly set up to handle Unicode. The article Working with Coding Systems and Unicode in Emacs explains the settings (though Emacs 27 seems to do the right thing by default).

Configuration

use-package setup

If you plan to use vterm, then make sure to install vterm first! (See the Installation section.) This (unfortunately) means that the order of use-package invocations below matters:

(use-package vterm
  :ensure t)
;; Now run `M-x vterm` and make sure it works!

(use-package julia-snail
  :ensure t
  :hook (julia-mode . julia-snail-mode))

To use Eat instead of vterm, set julia-snail-terminal-type to :eat:

(use-package julia-snail
  :ensure t
  :custom
  (julia-snail-terminal-type :eat)
  :hook
  (julia-mode . julia-snail-mode))

Manual setup

Install dependencies as noted in the Package-Requires line of julia-snail.el. Then make sure vterm works, as described in the Installation section.

(add-to-list 'load-path "/path/to/julia-snail")
(require 'julia-snail)
(add-hook 'julia-mode-hook #'julia-snail-mode)

Then configure key bindings in julia-snail-mode-map as desired.

display-buffer-alist notes

Window and buffer display behavior is one of the worst defaults Emacs ships with. Please refer to remarks on the subject written elsewhere. Some packages go to great lengths to provide clean custom window management (e.g., Magit), but Snail does not have the resources to implement such elaborate schemes.

Snail uses the Emacs display-buffer system to pop up windows, and tries to do so in the most sane manner possible. Most information pop-ups (except the REPL) can be dismissed by pressing q, and this should restore the window configuration in most cases. With Emacs defaults, Snail should also reuse existing windows as much as possible, i.e., calling julia-snail from a source buffer will switch an already-visible REPL window, and calling julia-snail from a REPL window will switch back to a source buffer and reuse already-visible windows.

However, display-buffer will (by default) split windows if the target buffer is not visible. To prevent splits altogether, try the following:

(add-to-list 'display-buffer-alist
             '("\\*julia" (display-buffer-reuse-window display-buffer-same-window)))

It is likely that most users will want the default REPL pop-up behavior to split the window vertically, but the default split-window-sensibly implementation only splits that way if split-height-threshold is smaller than the current window height. split-height-threshold defaults to 80 (lines), and relatively few windows will be that tall. Therefore, consider adding something like the following to your configuration:

(customize-set-variable 'split-height-threshold 15)

Other customizations

• julia-snail-use-emoji-mode-lighter (default t) — attempt to use a 🐌 emoji in the Emacs modeline lighter if the display supports it. Set to nil to use the ASCII string "Snail" instead (a :diminish override in use-package should also work).
• julia-snail-repl-display-eval-results (default nil) — print the result of evaluating code sent from Emacs to the REPL.
• julia-snail-popup-display-eval-results (default :command) — show the result of evaluating code sent from Emacs to the REPL in the source buffer. Set to nil to deactivate, to :command to have the popup disappear at the next command, or to :change for when the buffer contents change. When set to :change, the popup display is limited to a single line.
• julia-snail-imenu-style (default :module-tree) — control Imenu integration, especially module detection handling. When set to :module-tree, the Imenu is a tree with modules as nodes and functions, macros, and types as the leaves. This works well with modern Imenu display commands like consult-imenu and helm-imenu, and allows the imenu-list package to show a nice tree. However, this may interfere with the simpler imenu Emacs built-in command as it forces hierarchical navigation to reach leaves. The :flat setting disables Imenu hierarchies and instead puts the full module path in the identifier. To disable Snail's Imenu integration completely and fall back to the julia-mode regexp-based default, set julia-snail-imenu-style to nil.

Usage

The following describes basic Snail functionality. Refer to the Snail project wiki for additional information, including a Tips and Tricks section.

Basics

Once Snail is properly installed, open a Julia source file. If julia-mode-hook has been correctly configured, julia-snail-mode should be enabled in the buffer (look for the Snail lighter in the modeline).

Start a Julia REPL using M-x julia-snail or C-c C-z. This will load all the Julia-side supporting code Snail requires, and start a server. The server runs on a TCP port (10011 by default) on localhost. You will see JuliaSnail.start(<port>) execute on the REPL.

NB: If the REPL does not start successfully, this means the julia binary invocation failed. A common reason for this is failure to find the julia binary. Check that julia-snail-executable is on your Emacs exec-path or set to an absolute path. It may be useful to do this in a .dir-locals.el so it can be set per-project. It may also happen that Snail bootstrapping fails, in which case the error buffer may flash too quickly to see. To debug this problem, switch to the command line and run /path/to/julia -L /path/to/julia-snail/JuliaSnail.jl, which should show the error.

If the REPL buffer is set to use libvterm mode (the default), then libvterm configuration and key bindings will affect it. If the REPL buffer is set to use Eat, then Eat configuration and key bindings will also take effect.

If the Julia program uses Pkg, then run M-x julia-snail-package-activate or C-c C-a to enable it. (Doing this using REPL commands like ] also works as normal.)

Load the current Julia source file using M-x julia-snail-send-buffer-file or C-c C-k. Notice that the REPL does not show an include() call, because the command executed across the Snail network connection. Among other advantages, this minimizes REPL history clutter.

Users of Revise should load it normally into the session, and do not need to use julia-snail-send-buffer-file.

Once some Julia code has been loaded into the running image, Snail can begin introspecting it for purposes of cross-references and identifier completion.

The julia-snail-mode minor mode provides a key binding map (julia-snail-mode-map) with the following commands:

key	command and description
C-c C-z	julia-snail start a REPL; flip between REPL and source
C-c C-a	julia-snail-package-activate activate the project using Project.toml
C-c C-d	julia-snail-doc-lookup display the docstring of the identifier at point
C-c C-l	julia-snail-send-line evaluate current line in the current module (or in Main with prefix arg; or copy directly to REPL with two prefix args)
C-c C-r	julia-snail-send-region evaluate active region in the current module (or in Main with prefix arg; or copy directly to REPL with two prefix args)
C-c C-e	julia-snail-send-dwim if region active, evaluate it in current module; else if on top-level block, evaluate it in current module; else evaluate current line
C-c C-c	julia-snail-send-top-level-form evaluate end-terminated block around the point in the current module
C-M-x	julia-snail-send-top-level-form ditto
C-c C-k	julia-snail-send-buffer-file include() the current buffer’s file
C-c C-R	julia-snail-update-module-cache update module-nested include cache (mainly for Revise)

Several commands include the note “in the current module”. This means the Julia parser will determine the enclosing module...end statements, and run the relevant code in that module. If the module has already been loaded, this means its global variables and functions will be available.

In addition, most xref commands are available (except xref-find-references). xref-find-definitions, by default bound to M-., does a decent job of jumping to function and macro definitions. Cross-reference commands are current-module aware where it makes sense.

Completion also works. Emacs built-in completion features, as well as company-complete, will do a reasonable job of finding the right completions in the context of the current module (though will not pick up local variables). Completion is current-module aware.

Experimental feature: To interrupt a Julia task started from the Emacs side (e.g. a long-running computation started with julia-snail-send-line), use julia-snail-interrupt-task. When only one task is running, Snail will simply try to terminate it. With multiple tasks, the user will be prompted for a request ID. This is currently an opaque identifier, and the interface will be improved in the future.

Multiple Julia versions

The julia-snail-executable variable can be set at the file level or at the directory level and point to different versions of Julia for different projects. It should be a string referencing the executable binary path.

NB: On a Mac, the Julia binary is typically Contents/Resources/julia/bin/julia inside the distribution app bundle. You must either make sure julia-snail-executable is set to an absolute path, or configure your Emacs exec-path to correctly find the julia binary.

Multiple REPLs

To use multiple REPLs, set the local variables julia-snail-repl-buffer and julia-snail-port. They must be distinct per-project. They can be set at the file level, or at the directory level. The latter approach is recommended, using a .dir-locals.el file at the root of a project directory. (Emacs provides numerous interactive helper functions to help deal with file and directory variable scope: add-dir-local-variable, delete-dir-local-variable, copy-dir-locals-to-file-locals, copy-dir-locals-to-file-locals-prop-line, and copy-file-locals-to-dir-locals. Users of Projectile have additional tools at their disposal: projectile-edit-dir-locals and projectile-skel-dir-locals.)

For example, consider two projects: Mars and Venus, both of which you wish to work on at the same time. They live in different directories.

The Mars project directory contains the following .dir-locals.el file:

((julia-mode . ((julia-snail-port . 10050)
                (julia-snail-repl-buffer . "*julia Mars*"))))

The Venus project directory contains the following .dir-locals.el file:

((julia-mode . ((julia-snail-port . 10060)
                (julia-snail-repl-buffer . "*julia Venus*"))))

(Be sure to refresh any buffers currently visiting files in Mars and Venus using find-alternate-file or similar after changing these variables.)

Now, source files in Mars will interact with the REPL running in the *julia Mars* buffer, and source files in Venus will interact with the REPL running in the *julia Venus* buffer.

Remote REPLs

Snail can use a Julia REPL instance running on a remote host using SSH tunneling and Emacs Tramp, subject to the following conditions:

1. A full Julia environment must be installed on the remote host.
2. The code under development is likewise on the remote host. Emacs must open source files using Tramp.
3. SSH access to the remote host must be configured with no-password access. Several ways exist to configure SSH to do this, all beyond the scope of the Snail README (hints: either use an agent or the ControlMaster setting; do not just copy a no-passphrase key to the remote host!).

If all these things are true, visit a remote Julia source file using Tramp, and start julia-snail. It should transparently start a fully-functional remote REPL.

Just as with local Julia sessions, Snail can be configured using a remote .dir-locals.el file or another method for setting variables. In particular, julia-snail-executable may need to be changed.

The SSH tunnel will, by default, open from julia-snail-port on the remote host to the same port on localhost. The remote host’s port can be changed by setting julia-snail-remote-port.

NB: To use .dir-locals.el over Tramp, you must set enable-remote-dir-locals to t!

julia-snail-executable and the remote shell path

NB: It is simpler to set julia-snail-executable in your project to the Julia binary’s absolute path than to wrangle your shell path. This section gives a bit of assistance if you disregard this advice.

A subtle problem may occur if julia-snail-executable is set to a value you expect to find on the remote host’s shell path. When Snail connects to the remote host using SSH, it will launch Julia in a non-interactive, non-login shell. This means that, depending on (1) your remote shell, (2) how you set your path, and (3) which shell startup files you rely on, the path may not be what you have in your ordinary remote shell sessions.

• Zsh: .zshenv is always executed; .zshrc is only executed by interactive shells. Make sure you set your path in .zshenv.
• Bash: .bashrc is only executed by non-interactive shells; .bash_profile is only executed by interactive shells. To run your setup regardless of shell type, put everything in .bashrc and source .bashrc from .bash_profile.

Remote environment setup

You may encounter a situation in which your remote host’s shell is configured to read startup files you do not control, and this setup does not occur when you attempt to launch a remote REPL. This may happen, e.g., in a computing cluster, where some startup files are required to correctly set up your environment (libraries, paths, and so forth) but are not being read by non-interactive non-login shells. In that case, it may be useful to force those extra scripts to load. Try adding the following code in your remote .bashrc or .zshenv scripts:

# bash (replace /etc/profile in both places with the file you need):
[[ ! $(shopt -q login_shell) && $- != *i* && -f /etc/profile ]] && . /etc/profile

# zsh (replace /etc/profile in both places with the file you need):
[[ ! -o login && ! -o interactive && -f /etc/profile ]] && . /etc/profile

Errors mentioning No file notification program found

A remote REPL may fail to start with a File notification error. Fixing this Tramp-related error requires (1) installing inotifywait on the remote host (on Debian-derived systems, the inotify-tools package should help), and (2) forcing Tramp to cleanup its cache using M-x tramp-cleanup-all-connections. Restarting the Tramp session may be needed afterwards (delete all buffers prefixed *tramp).

Docker container REPLs

Snail can use a Julia REPL instance running inside a Docker container. Like SSH remote REPLs, this uses Tramp. To make this work:

1. The docker-tramp package must be installed in the local Emacs instance. (Snail does not automatically install this dependency because the container feature is optional.)
2. A full Julia environment is available in-container.
3. The code under development is available in-container (using volumes or some other mechanism).
4. Ports must be appropriately mapped in-container.

A sample container invocation should help understand the requirements:

docker run --name julia-1 --rm -it -p 127.0.0.1:10011:10011 -v "${HOME}/work:/work" julia bash

Visit an in-container Julia source file using Tramp, and start julia-snail. It should transparently start an in-container REPL.

Snail configuration, in particular port mapping, works the same as for remote SSH REPLs.

Extra Julia command-line arguments

The julia-snail-extra-args variable can be set to include additional arguments to the Julia binary. It can be set to nil (the default), a string, or a list of strings.

This variable is buffer-local, so it can be kept distinct per-project using .dir-locals.el. The following example starts Julia with a custom image and automatically activates a specific project:

((julia-mode . ((julia-snail-extra-args . ("--sysimage" "/path/to/image"
                                           "--project=/path/to/project")))))

Module-nested includes

Consider the following file, call it alpha.jl:

module Alpha

include("alpha-1.jl")
include("alpha-2.jl")

end

Everything in the files alpha-1.jl and alpha-2.jl is inside the Alpha module, but neither of these files will mention that module explicitly. Snail supports this by using the Julia parser to track include(...) calls and their module context. This feature works with nested modules.

Using this feature requires some care. The root file which contains the module declaration (alpha.jl in this example) must be loaded using julia-snail-send-buffer-file first (or, for Revise users, julia-snail-update-module-cache). Alternatively, you could run julia-snail-analyze-includes, which does not evaluate the code in the root file but analyzes and remembers the structure of include statements, and then you need to manually load the package of the root file with a normal import or using statement in the REPL. If this does not happen, the parser will not have the opportunity to learn where alpha-1.jl and alpha-2.jl fit in the module hierarchy, and will assume their parent module is Main. The same applies to any deeper nesting of files (i.e., if alpha-1.jl then does include("alpha-1.1.jl"), then julia-snail-send-buffer-file or julia-snail-update-module-cache must be executed from alpha-1.jl).

Furthermore, if alpha-1.jl is refactored to sit outside the Alpha module, or moved in the directory structure, Snail must be informed. To do this, call the julia-snail-clear-caches command.

Documentation lookup

julia-snail-doc-lookup shows the documentation string of the identifier at point. If the current Emacs session has markdown-mode installed, it will be turned on with markup hiding enabled.

Multimedia and plotting

Snail supports making diagrams by plugging into Julia’s multimedia I/O system. Any plot back-end which generates SVG or PNG output can display in an Emacs buffer, provided the Emacs instance itself supports images.

To enable Emacs-Julia multimedia integration, either (1) set local variable julia-snail-multimedia-enable to t, preferably in .dir-locals.el, or (2) after the Julia REPL connects to Emacs, call the function julia-snail-multimedia-toggle-display-in-emacs.

With Emacs multimedia display turned on, plotting commands in packages like Plots and Gadfly will display an Emacs buffer.

The following variables control multimedia integration. It is best to set these in the project’s .dir-locals.el.

• julia-snail-multimedia-enable: When set before starting a REPL, this turns on Emacs multimedia integration.
• julia-snail-multimedia-buffer-autoswitch: Controls whether Emacs should automatically switch to the image buffer after a plotting command, or if it should only display it. Defaults to nil (off).
• julia-snail-multimedia-buffer-style: Controls how the multimedia display buffer works. When :single-reuse (default), it uses one buffer, and overwrites it with new images as they come in from Julia. When set to :single-new, Snail will open a new buffer for each plot. When set to :multi, Snail uses a single buffer but appends new images to it rather than overwriting them. Note that :multi inserts image objects, but does not enable image-mode in the buffer, thus limiting zoom capabilities.

As a simple example, activate Emacs plotting and try run this code in the REPL:

Pkg.add("Gadfly")
import Gadfly
Gadfly.plot(sin, 0, 2π)

NB: One complication to keep in mind: calls to Gadfly.plot and Plots.plot will return plot objects instead of displaying them when called across the Emacs-Julia bridge using commands such as julia-snail-send-line (but not when called directly in the REPL). In this case, explicitly call display on the plot object:

display(Gadfly.plot(cos, 0, 2π))

code-cells integration (notebook mode)

Snail can be used with the code-cells package for (Jupyter) notebook-style work. Sample configuration follows:

(use-package code-cells
  :hook (julia-mode . code-cells-mode)
  :config
  (add-to-list 'code-cells-eval-region-commands '(julia-snail-mode . julia-snail-send-code-cell)))

Extensions

Snail supports opt-in extensions. The julia-snail-extensions variable controls which extensions load in a given Snail session. As most other Snail configuration variables, it is best set at a project level using .dir-locals.el:

((julia-mode . ((julia-snail-extensions . (repl-history formatter)))))

Extensions will be enabled at Snail startup, and will install their own Julia-side dependencies as needed into their own individual Pkg environments.

An annoyance: because Snail extensions do not use Elisp autoloading, customizing their keybindings must be performed in an explicit with-eval-after-load form. This cannot (currently) be done in a use-package :bind directive. To change extension keybindings, use the following pattern (which can be placed in a use-package :config directive):

(with-eval-after-load 'julia-snail/repl-history
  (define-key julia-snail/repl-history-mode-map (kbd "H-y") #'julia-snail/repl-history-search-and-yank))

REPL history

This extension provides access Julia REPL history from julia-snail-mode buffers using the following commands (the letters in the key sequences stand for julia repl history):

• julia-snail/repl-history-search-and-yank provides a search interface (C-c j r h C-s)
• julia-snail/repl-history-yank yanks recent history entries (1 by default) (C-j r h C-y)
• julia-snail/repl-history-buffer opens a buffer with the history (C-c j r h C-o)

Formatter

This extension uses JuliaFormatter.jl to modify source buffer text (the letters in the key sequences stand for julia formatter):

• julia-snail/formatter-format-region modifies the current region (C-c j f r)
• julia-snail/formatter-format-buffer modifies the entire current buffer (C-c j f b)

Ob-Julia

This extension lets julia-snail be used in Org Mode src blocks. This implementation does not closely observe the usual functional conventions of org babel langauges, and instead more closely mirrors emacs-jupyter's behaviour. This mode is not very mature yet, but it does support rich multimedia display of images and plots, and also allows one to choose the evaluation module with a :module session parameter (default is Main).

To use it, enable the ob-julia extension, either globally with M-x customize-variable, or by putting the following snippet as the first line of your Org file:

-*- julia-snail-extensions: (ob-julia) -*-

Then re-open the Org file (using find-alternate-file or a similar command. Note that other Snail configuration variables may also be set in this block. See the Emacs manual's section on file-local variables for syntax details.

Once the extension is enabled, Org Babel commands should work on Julia code as expected. Completion support is available through the Emacs completion-at-point system.

Limitations: no xref support currently available.

Customization variables:

• julia-snail/ob-julia-use-error-pane t : If true, use julia-snail's popup error pane. Otherwise, display errors inline
• julia-snail/ob-julia-mirror-output-in-repl t : If true, all output from code evaluated in ob-julia will also be shown in the julia REPL.
• julia-snail/ob-julia-capture-io t : If true, all intermediate printing during evaluation will be captured by ob-julia and printed into your org notebook
• julia-snail/ob-julia-resource-directory "./.ob-julia-snail/": Directory used to store automatically generated image files for display in org buffers. By default this is a local hidden directory, but it can be changed to e.g. /tmp/ if you don't want to keep the image files around.

Debug

This extension uses DebugAdapter.jl and dape to allow debugging inside the REPL.

Use one of the following macros to initiate the debug session from the REPL:

• @run pause on first breakpoint.
• @enter pause on entry.

using .JuliaSnail.Extensions.Debug

@enter(println("hello world"))

Future improvements

• The libvterm dependency forces the use of recent Emacs releases, forces Emacs to be build with module support, complicates support for Windows, and is generally quite gnarly. The Eat alternative requires Emacs 28. It would be much better to re-implement the REPL in Elisp and make sure it works on older Emacs versions.
• Completion does not pick up local variables.
• A real eldoc implementation would be great, but difficult to do with Julia’s generic functions.
• A real test suite which fully drives both Julia and Emacs and runs in a CI environment (like GitHub Actions) wouldn’t hurt, either.