Jupyter-Emacs Sessions with org-mode

The first thing to do is start the jupyter session on the remote machine. Since I'm doing this with nikola I should note that you want to start the session in the same location as the file you're editing in emacs, because all your file references will be based on that directory (so if you, for instance, create an image and want to place it in the files folder, you will need to note where that stands relative to the file you are editing and where you start the jupyter kernel).

In my case I'm editing a file in ~/projects/In-Too-Deep/posts/fastai/.

cd ~/projects/In-Too-Deep/posts/fastai/
jupyter kernel

This will start the kernel and show you the file that you need to copy to your local machine (where you are running emacs). Here's an example output of that command.

(In-Too-Deep) hades@erebus ~/p/I/p/fastai (fastai-restart| Dirty:4)> jupyter kernel
[KernelApp] Starting kernel 'python3'
[KernelApp] Connection file: /home/hades/.local/share/jupyter/runtime/kernel-ae33a6cd-f607-450e-a03b-01abe2a3b232.json
[KernelApp] To connect a client: --existing kernel-ae33a6cd-f607-450e-a03b-01abe2a3b232.json

The important thing to note is the line with Connection file ([KernelApp] Connection file: /home/hades/.local/share/jupyter/runtime/kernel-ae33a6cd-f607-450e-a03b-01abe2a3b232.json). You will need to copy that file to the machine that you are running emacs on. Where do you put it? Check your jupyter location on your local machine (where you're running emacs, not where you're running jupyter).

jupyter --runtime-dir

Change into whatever directory is output by that command and then copy the json file from the machine with the running jupyter kernel onto your local machine.

cd ~/.local/share/jupyter/runtime
scp Hades:/home/hades/.local/share/jupyter/runtime/kernel-ae33a6cd-f607-450e-a03b-01abe2a3b232.json .

Now that you've copied over the information for the jupyter session you can start a console for it. I'll assume you're still in the directory with the json file in it, so I won't pass in the full path.

jupyter --console kernel-ae33a6cd-f607-450e-a03b-01abe2a3b232.json --ssh Hades

Note the second argument where I passed in the SSH alias for my remote machine. If you don't have an alias set up then replace it with something that looks like <username>@<IP Address> (this assumes, of course, that the machine with the jupyter session running on it also has an SSH server running). This command sets up our session to forward our jupyter commands to the remote machine. When you execute this command it should tell you that you can connect to the kernel using a slightly modified JSON file name:

[ZMQTerminalIPythonApp] Forwarding connections to 127.0.0.1 via Hades
[ZMQTerminalIPythonApp] To connect another client via this tunnel, use:
[ZMQTerminalIPythonApp] --existing kernel-ae33a6cd-f607-450e-a03b-01abe2a3b232-ssh.json

Note that --existing argument for the next session (it should be the same as the original json file but with -ssh added to the end of the name).

Since this is org-mode-based the first thing you should do is connect your emacs buffer to the console. Add this to the top of your buffer (the file where you intend to run python).

#+PROPERTY: header-args :session /home/athena/.local/share/jupyter/runtime/kernel-ae33a6cd-f607-450e-a03b-01abe2a3b232-ssh.json

If you have this in your file when you open it you don't need to do anything special, but otherwise C-c C-c on it to load the session. What this does is allow subsequent python org-mode blocks to use the remote jupyter session when you execute them, without needing to specify a session.