Install via Quicklisp instructions are unclear

[Symbolics]

I've got all the prerequisites installed, but am unclear on the Quicklisp install instructions:

Install Quicklisp and use (ql:add-to-init-file). If you already have Quicklisp installed you may need to update your distribution with (ql:update-dist "quicklisp") to resolve package conflicts.

    To install an image based user kernel evaluate (cl-jupyter:install-image)
    To install a non-image based user kernel evaluate (cl-jupyter:install)
    To install a Quicklisp/ASDF based system which uses system or user packages available via ql:quickload or asdf:load-system evaluate (cl-jupyter:install :system t :local t :bundle t :prefix "pkg/"). Afterward copy the contents of the pkg directory to the system root. For instance in bash sudo cp -r pkg/* /
    To install a Quicklisp bundle based system evaluate (cl-jupyter:install :system t :local t :bundle t :prefix "pkg/"). Afterward copy the contents of the pkg directory to the system root. For instance in bash sudo cp -r pkg/* /

I assume that since common-lisp-jupyter is in quicklisp:

CL-USER> (ql:system-apropos "common-lisp-jupyter")
#<SYSTEM common-lisp-jupyter / common-lisp-jupyter-20201016-git / quicklisp 2020-10-16>

I can install or clone it and then load with (ql:quickload :common-lisp-jupyter), however I'm not clear on the difference between the options:

1. image based user kernel
2. non-image based user kernel
3. Quicklisp/ASDF based system which uses system or user packages
4. Quicklisp bundle based system

It sounds like if I want the notebook to use Quicklisp or ASDF I need option 3 or 4, which have the same installation instructions. Can you elaborate a bit on each of these four options?

[yitzchak]

User kernels are installed in the user's Jupyter folder, i.e. ~/.local/jupyter/kernels/... whereas system kernels are installed in the system-wide Jupyter folder, i.e. /usr/share/jupyter/kernels/... or /usr/local/share/jupyter/kernels...

An image based kernel uses uiop:dump-image to load the kernel and alll dependencies. A non-image based kernel just loads the kernel via ql:quickload if quicklisp is available and falls back to asdf:load-system otherwise.

When installing a system kernel you can create a quicklisp bundle. This collects all dependencies into /usr/share/common-lisp-jupyter so that a user does not need quicklisp installed. A non-bundled system kernel will rely on the specific user installing quicklisp before the kernel is started.

If you are installiing common-lisp-jupyter on a machine in which you are the only user you should just do a user based kernel. If your Lisp impllementation loads very slowly and supports uiop:dump-image then you could use the image method.

If these additional details help clarify the differences then I'll add them to the README. I know that the current details are a little sparse.

[Symbolics]

I see, thanks. I think it would help to add to the README. I'd suggest perhaps explaining it in two parts:

• The selection of the kernel image type
• The location of the lisp image when installing into jupyter (along with any considerations for MS Windows)

That way it will be easier for folks who have custom installations put things in the right place and select the best kernel for them. It might be good to mention why a dumped image might be useful too; we used to dump images years ago to run with, but these days things are so fast I just load everything up from source/fasl.

As an FYI on Windows, I've now got a ~/.jupyter directory following the installation of JuypterLab, but nothing in AppData/Local, which is where I'd normally expect to find things. In fact nothing in my home directory at all except .jupyter. I did an MSYS2 installation of JupyterLab, so it's probably over in that area. I'll have a look around tomorrow and update this thread on what I needed to do on Windows (it sounds like this isn't a platform you are using much)

[yitzchak]

The Windows installation stuff should work but hasn't been tested for several months since my Appveyor CI broke. I have started transitioning to GitHub actions but haven't setup the Windows stuff yet.

The main obstacle to Windows usage is the fact that pzmq and static-vectors both use cffi grovel which means you need a complete development system. I have working branches of pzmq and statiic-vectors that avoid grovelling via a combination of calls to zmq to determine message allocation info and declaring some types explicitly. I am waiting for cffi to be updated in quicklisp before I submit them as pull requests.

I would like to support Windows without requiring msys. I think it is possible, but the pieces aren't quite there yet.

And yes you are right, there isn't much testing going on in Windows right now. I am using Linux and the CLASP/CANDO people are using Linux and MacOS.

[Symbolics]

Do you recommend the Windows Python or the MSYS version for JupyterLab? Which one is better tested?

[yitzchak]

I think that I have only tried the MSYS one since I had to install MSYS anyways.

[Symbolics]

Okay. The reason I ask is because the Windows wiki instructions seem to indicate paths that are used in the Windows package install, not MSYS.

I'm trying for a Quicklisp install using a MSYS python. I'll let you know how I go and, if you're open to pull requests, will update the Windows instructions for the same. I've not used Roswell, although I do see it more frequently over the last year or so.