Diving into the implementation of xwidgets

Interactive widgets allow Jupyter users to create user interfaces inline in their notebooks, and to turn them into standalone applications with tools such as Voilà.

Language backends for Jupyter interactive widgets exist in Python (with ipywidgets), and C++ (with xwidgets, and the xeus-cling C++ Jupyter kernel), reusing the same frontend implementation of the widgets in JavaScript.

Simple interactive widgets at play in JupyterLab with the xeus-cling C++ kernel

In this article, we dive into some of the C++ techniques used in the implementation of the xwidgets library, including

• discussions on value semantics and RAII (Resource Acquisition Is Initialization),
• an original application of CRTP (Curiously Recurring Template Pattern),
• an original implementation of the observer pattern, xproperty,
• a plea for allowing the overloading of the dot operator in C++, to enable better proxy types.

For readers interested in knowing more about interpreted C++, we recommend the following posts:

- Interactive workflows for C++ with Jupyter (Jupyter blog), by Sylvain Corlay, Loic Gouarin, Johan Mabille, and Wolf Vollprecht.
- Interactive C++ for Data Science (LLVM blog), by Vassil Vassilev, David Lange, Simeon Ehrig, and Sylvain Corlay
- Interpreted C++ for GIS with Jupyter (Jupyter blog), by Martin Renou

Using the RAII pattern for a widget library

Jupyter widgets are special objects that trigger the creation of a counterpart JavaScript model object in the Jupyter frontend upon creation. The state of the object in the backend is synchronized with the state of the JavaScript frontend object. Views of that widget model are instantiated upon display.

The MVC (Model View Controller) architecture of Jupyter widgets, and synchronization with the backend

This MVC (Model-View-Controller) architecture for Jupyter interactive widgets allowed us to reuse all of the frontend implementation, by simply providing an alternative backend in C++, implementing the same messaging protocol.

In order to tie the lifetime of the kernel and frontend objects, we decided to use the RAII (Resource Acquisition Is Initialization) pattern. RAII is a common programming idiom that consists of tying the lifetime of an object with the holding of a resource. Most typically, the resource is

• acquired in the constructor of the object,
• released in the destructor of the object.

The RAII pattern is not common in garbage-collected languages because unlike in C++, the time when objects are destroyed is not deterministic. The Python programming language mitigates that issue by introducing context managers, often used to e.g. open and close files.

A consequence of relying on object lifetime for resource management in C++ is to adopt the "value semantics" for widget instances, instead of "reference semantics" which is more typical for widget frameworks like Qt.

Note: value semantics vs reference semantics

If you are not familiar with the concepts of value and reference semantics in C++, I recommend reading the excellent FAQ of isocpp.org.

To summarize:

- with value semantics,objects hold actual values, and copying an object copies their attributes. With value semantics, one should provide implementations of copy, and move constructors, as well as copy and move assignment operators. Besides, values should not have virtual methods.

- with reference semantics, objects are manipulated through references (or pointers) and never by value. Reference semantics is typically used for polymorphic programming with virtual methods. With reference semantics, it is recommended to delete copy and move constructors, as well as copy and move assignment operators to avoid accidental copies when passing objects to functions taking arguments by values, causing object slicing. (They can also be made private). Explicit cloning of a reference semantics object is generally allowed via a call to a clone virtual method.

In a C++ codebase, the existence of public copy or move constructors alongside virtual methods in the same class is generally a sign of a bad design.

The use of value semantics for xwidgets provides clear lifetime management for associated resources. Careful use of the move semantics provides fine-grained control.

Illustration of the move semantics for xwidgets

Another advantage of using value semantics in xwidgets is that C++ beginners who are typical users of the Jupyter notebook (often used by instructors) can easily manipulate "widgets as values" without having to deal with manual memory allocation etc.

With value semantics, addressing the lifetime of objects becomes the responsibility of the framework author, in a carefull implementation of (copy and move) constructors, destructors, and (copy and move) assignment operators.

Static polymorphism and the CRTP pattern

While the use of value semantics provides fine-grained control over the lifetime of widgets, it prevents the use of virtual methods in their implementation. Code reuse is achieved with static polymorphism techniques and specifically the CRTP pattern.

The Curiously Recurring Template Pattern (CRTP) is the practice of making a class X derive from a class template instantiation using X itself as a template argument.

class X : public base<X>

CRTP is commonly used by matrix or tensor algebra libraries making use of expression templates (such as xtensor, eigen, or blitz) to prevent the overhead of virtual function dispatch for operations likely to be performed in a loop, such as element access.

Placing common code in a CRTP base allows code reuse without the overhead of virtual dispatch. However, the template base class is specialized for each final type increasing the resulting binary size. This can be mitigated by factoring as much of the code in a base class that would not be templated by the derived type.

CRTP in xwidgets — closing the recursion

In order to allow for code reuse without virtual inheritance, xwidgets' class hierarchy is entirely based on CRTP. Our naming scheme is that CRTP bases, which should not be instantiated directly are prefixed with the letter x and final concrete widget types are not.

Upon construction of e.g. the button widget, constructors of base types are called in the order of inheritance, which is why we need to establish the connection with the frontend in the constructor of the most derived type, after all attributes have been initialized, and send a message to the frontend with all the values.

The pattern for creating the most derived type being always the same, we defined the xmaterialize template class closing the CRTP hierarchy with

using button = xmaterialize<xbutton>;

The xmaterialize template class is defined as final, to prevent further inheritance. The constructors and assignment operators forward to those of the CRTP bases and implement the RAII pattern.

template <template <class> class B, class... P>
class xmaterialize final : public B<xmaterialize<B, P...>>
{
public:

using self_type = xmaterialize<B, P...>;
    using base_type = B<self_type>;

template <class... A>
    xmaterialize(A&&...)
        : base_type(std::forward<A>(args)...)
    {
        this->open();      //  RAII: create the frontend model.
    }    template <template <class> class B, class... P>
    inline xmaterialize<B, P...>::~xmaterialize()
    {
        if (!this->moved_from())
        {
            this->close();  // RAII: delete the frontend model.
        }
    }    ...};

The full implementation of xmaterialize (as of xwidgets 0.25) is available here. Beyond the logic described in this section, it also includes the handling of the move semantics and the method chaining API which is the subject of a later section.

Precompilation and binary size optimization

Even though xwidgets is fully based on template types, we decided to precompile all final widget types for faster interactive use with the xeus-cling kernel. This is achieved with

• an extern declaration in the header (here in xbutton.hpp)

extern template class xmaterialize<xbutton>;

• and in the source file (here in xbutton.cpp), an instruction for the precompilation

template class XWIDGETS_API xmaterialize<xbutton>;

Doing this for all widget types of the library initially resulted in a large compiled binary size. Using the button widget as an example, we see that base types are templated by the final type in the class hierarchy button -> xbutton<button> -> xwidgets<button> -> xobject<button>, and therefore, their binary representation is duplicated for each final type.

A strategy for reducing the binary size has been to factor out as much of the logic of xobject<D> in a non-template base xcommon improving compilation speed and preventing binary code duplication.

Xproperty: an implementation of the observer pattern

In order to update the frontend upon changes of widget properties, xwidgets relies on an implementation of the observer pattern called xproperty. xproperty is to xwidgets what traitlets are to ipywidgets.

In order to trigger observers and validators in the owner object upon assignment of new values, xproperty relies on the overload of the assignment operator =.

template <class T, class O>
template <class V>
inline auto xproperty<T, O>::operator=(V&& value) -> reference
{
    // Before assigning the new value, invoke validators
    // which may also mutate the value.
    m_value = owner()->template invoke_validators<T>(m_name, std::forward<V>(value));    // Call class-level observer.
    owner()->notify(m_name, m_value);    // Call registered observers for that attribute
    owner()->invoke_observers(m_name);    // Return the new value 
    return m_value;
}

Method chaining for widget initialization

Jupyter interactive widgets have many attributes that may be specified at construction time.

In the Python implementation, this is handled with keyword arguments, but the C++ programming language does not support keyword arguments. There exist various approaches to enable this feature with advanced metaprogramming techniques. In the case of xwidgets, this need is limited to the initialization of xproperty attributes, which allowed us to adopt a more scoped approach: a method chaining API for property initialization:

auto slider = slider<double>::initialize()
  .min(-1.0)
  .max(1.0)
  .description("Another slider")
.finalize();

this was enabled by overriding the function call operator () on xproperties to pass an intial value (only when the said property is an rvalue). A static initialize method is used instead of the default constructor to prevent the initialization of the JavaScript counterpart while all attributes may not have been set yet. The frontend counterpart is only acquired with the finalize() call.

Building upon xwidgets

Jupyter interactive widgets are not limited to the controls available in the core package. In fact, there is a rich ecosystem of widget libraries built upon the core framework: ipyvolume (3-D plotting), ipyleaflet (maps visualization), bqplot (2-D plotting), ipygany (3-D mesh visualization), ipycanvas (generic drawing), ipywebrtc (streaming video and audio), and many many more.

For the C++ programming language, we have already provided:

• xleaflet (the C++ equivalent to ipyleaflet, reusing the same frontend),
• xwebrtc (the C++ equivalent to ipywebrtc, reusing the same frontend).

Screencast of xleaflet in JupyterLab, loading and visualizing a GeoJSON dataset in a C++ notebook.

Potentially, C++ backend to all Jupyter interactive widget packages could be provided, creating a huge opportunity for interactive data visualization in C++.

C++ should allow overloading the dot operator

xwidgets and xproperty makes heavy use of value semantics, and proxy objects.

• At the moment, to access an attribute or method of a value held in an xproperty object, we must first call the function call operator () to access the undelying object first, which is cumbersome.
• This is also an issue in other places in the xwidgets stack, when making use of e.g. reference proxies.

Being able to automatically map all methods of the underlying type to be accessible in the xproperty would be incredibly powerful, and remove the need for explicitly accessing the underlying.

Interestingly, the C++ standard does have the equivalent operator overload when it comes to pointer semantics, with the arrow operator ->. If overloading the dot operator . was allowed, this could enable this kind of usecase:

struct V
{
    void f();
};

struct X
{
    V& operator.() { return m_value; }
    V m_value;
};

in which case, X::f would call V::f .

Allowing the overloading of the dot operator . would also enable usecases such as smart references (similar to smart pointers, but with value semantics) and fully-fledged reference proxies (such as the return type of operator[] for std::vector<bool>).

Try it online!

Thanks to MyBinder, you can try xwidgets without the need of installing anything on your computer. Just follow this link:

About the author

Sylvain Corlay is the founder and CEO of QuantStack, an open-source software development studio comprising maintainers of key projects of the scientific computing ecosystem.

As an open-source developer, Sylvain is very active in the Jupyter project, contributing to the Xeus stack, Jupyter interactive widgets, Voilà dashboards. He is a member of the Jupyter steering committee and was the vice chair of JupyterCon 2020. Sylvain also contributes to the conda-forge project and he is the co-creator of the Xtensor C++ tensor algebra library.