Kernels are a simple but powerful abstraction in the Jupyter architecture. They encapsulate language interpreters and make them accessible through a standardized interface. This is the key to Jupyter’s remarkable versatility, with over 100 supported languages. Embedding a kernel in your custom application can seamlessly expose it to the Jupyter ecosystem. Not only can you attach JupyterLab to your program for e.g. state inspection, as you would do with a debugger, but you can truly extend your application with all the power of the Jupyter ecosystem.

Applications: GUI and API

Several applications such as FreeCAD, Blender, or QGIS provide a Python API to access their functionalities programmatically, and include an embedded Python console which allows using this API from within the application. Users can both use the graphical user interface and operate on the application in a scripted fashion depending on their needs.

If you think about it, these are the kinds of workflows that Jupyter has always been about, but Jupyter has evolved in the opposite direction. Instead of adding scripting to GUIs, Jupyter added GUIs to scripting. At first, Jupyter was IPython, a console for interactive computing. Then it went to the Web and became IPython Notebook, renamed to Jupyter Notebook. With the browser it became possible to display richer content and create small GUIs, using Jupyter Widgets. Then JupyterLab was created and allowed to create fully-fledged desktop-like applications in the browser, like 3-D CAD modelers.

Jupyter put a lot of attention on building a good experience interacting with the Python REPL, which is not the main focus of domain-specific desktop applications. The Python console embedded in desktop applications is often quite raw. Allowing users to connect from Jupyter can enable a much better experience.

What if there was a bridge that allowed you to connect both worlds, your application on one side and Jupyter on the other side? What is the common denominator allowing code to interact with an application, while also being shared with Jupyter? In the Jupyter infrastructure, we call it a kernel.

Jupyter kernels

Kernels provide a standardized way to expose language interpreters to applications, in a language-agnostic fashion. You can think of it as a server that accepts requests such as “execute this piece of code” and that replies with a result. The result is actually a data representation, it can be as simple as “3” for an input of “1+2”, or a complex interactive plot. A kernel usually lives in a separate process on your machine, but it can also run in the same process, or even remotely on another machine. Someone wanting to connect to this kernel will need the connection information such as the IP address and the ports of the different communication “channels”. It is stored in what is called a connection file.

Anyone who has access to this connection file can then execute code in the kernel. This is “shared code”, meaning that every client to this kernel eventually runs code in the same interpreter. If one client sets a variable “a=1”, other clients will see this variable. They are all running the “same program”. If the kernel runs in the same process as the application, then the kernel can interact directly with it. This way of sharing the execution engine serves as a bridge between the application and Jupyter.

Now it is just a matter of exposing the connection file so that e.g. JupyterLab can see it. The application launching the kernel can save the connection file to a specific directory, and give the path to JupyterLab. And that’s it, JupyterLab can run a notebook using this kernel, allowing it to interact with the application.

Getting started

The xeus-qt-python repository is a great place to start and see how to embed a Python kernel inside a Qt application. Under the hood, it uses xeus-qt, an integration of the xeus kernel architecture in the Qt event loop. The Qt application actually runs JupyterLab in a Qt WebEngine window. Users can run code in a notebook and interact with the application, which in turn can interact with the notebook. Here, we create a Qt button from a notebook. When clicking on the button, a callback prints a message in the notebook. This demonstrates a two-way binding between the Qt application and JupyterLab.

SlicerJupyter from Kitware was one of the first applications to take advantage of this approach. You can read more in this blog post. We hope that this technology will unlock a new brand of use-cases and allow applications to reach a new audience through the Jupyter ecosystem. Please reach out to us if you need help!

About the author

David Brochart is a technical director at QuantStack. He mostly works on the Jupyter backend stack. He contributed to the development of real-time collaboration in JupyterLab, and created the Jupyverse project, a next-generation implementation of the Jupyter server.

Acknowledgments

This work was made possible thanks to several contributors. Mariana Meireles was the initial author of xeus-qt. Thorsten Beier is a scientific software developer at QuantStack and contributes to the xeus kernel ecosystem. He is the initial creator of the xeus-qt-python demonstration. Johan Mabille is a technical director at QuantStack and he is the main author of xeus. Afshin Darian isa technical director at QuantStack and worked on the UI in JupyterLab 4.0 allowing to use an external kernel.