Introducing PyScript (summary of PyCon keynote)

Welcome to LWN.net The following subscription-only content has been made available to you by an LWN subscriber. Thousands of subscribers depend on LWN for the best news from the Linux and free software communities. If you enjoy this article, please consider subscribing to LWN. Thank you for visiting LWN.net!

In a keynote at PyCon 2022 in Salt Lake City, Utah, Peter Wang introduced another entrant in the field of in-browser Python interpreters. The Python community has long sought a way to be able to write Python—instead of JavaScript—to run in web browsers, and there have been various efforts to do so over the years. Wang announced PyScript as a new framework, built atop one of those earlier projects, to allow Python scripting directly within the browser; those programs have access to much of the existing Python ecosystem as well as being able to interact with the browser document object model (DOM) directly. In addition, he gave some rather eye-opening demonstrations as part of the talk.

Wang began by introducing himself and the company that he runs, Anaconda, which he co-founded with Travis Oliphant ten years ago. Oliphant was the creator of NumPy and one of the founders of SciPy, both of which are cornerstones of the Python scientific-computing ecosystem. Anaconda has created a number of different tools that are used widely in the community, as well as founding the NumFOCUS non-profit and the PyData conferences.

There were a number of reasons why he and Oliphant chose to focus their efforts around Python, including that the language is approachable, even by those who lack a computer-science background. Another point in its favor is that the Python community is generally welcoming and pleasant to work in. That is a "really big deal if you want to keep growing the user base".

But there is another aspect of the language that makes it so desirable from his standpoint: it can be extended with binary extensions that use an API that is written in C, but can be accessed from other languages. He likens Python to "a Honda Civic with mounting bolts for a warp drive". So the language can be picked up by kids who can then pop open the trunk "and bolt on warp nacelles" that allows the code to run faster than C or C++ in some cases, Wang said.

That aspect is sometimes overlooked, but it means that Python can be used in ways that other, similar languages cannot. "It's not just like Node, it's not just an alternative to Ruby". The reason Python was picked up by Wall Street firms ten or 15 years ago was because of this warp-drive capability, he said.

What sucks

"Here among friends" we can talk about what sucks about the language as well, he said. Even though Anaconda is in the business of providing a Python distribution, he "will be the first to say" that installing everything that is needed for Python is too hard. There are an enormous number of packages on the Python Package Index (PyPI), but getting them to work together is difficult. There are lots of different tools to help solve that problem, but all of them are at about 80%, he said, so people are having a bad experience 20% of the time, which is "not great".

It is odd that, for the world's most popular language, as Python is reported to be, it is difficult to write and distribute applications with a user interface. For example, you cannot write iOS apps with Python. You cannot create an application for Windows—the most popular corporate desktop—with a user interface; even if you use a web front-end, you have to write JavaScript, CSS, and HTML, he said. It's "kind of weird" that you cannot easily do all of that, but at the same time, it's "kind of interesting".

Meanwhile, though, the consequences of those two points, difficulties in packaging and building user interfaces, make it hard to share our work with others, he said. To those who would point to Docker as a solution for that, he said that when you package an application with Docker, you are "zipping up a hard drive and shipping it to someone". That "cannot be our way of getting millions of people to use this stuff".

To a great extent, Python is a victim of its own success. It is an excellent glue language, but that means that it got stuck to all of those things. So much of what we do in computing is tied to ideas and architectures from the 1970s and 1980s, he said, starting with the C language and Unix process model; that also includes things like the toolchains and internetworking protocols like TCP/IP.

The basics of the Python language proper can be taught to anyone in a weekend, he said, but it takes a whole lot more effort to get them to the point where they can create an executable for Windows or an iOS app for an iPad. "Can we unshackle Python from all of this?"

Enter WebAssembly

The web browser has clearly won the operating system wars, Wang said. He does not know if 2022 will be the year of the Linux desktop, [it won't be -ed.] but he does know that there will be a lot of browsers on desktops. JavaScript is the king of some language-popularity surveys because it is the native language of the browser. So, if we want to move into that realm, he said, WebAssembly (or Wasm) is clearly the right answer.

WebAssembly "fundamentally changes the game". It is a virtual CPU instruction set that recently became a W3C standard; it has a 32-bit address space and can do 64-bit arithmetic. There is a compiler tool, Emscripten, that can be used to compile most C and C++ code to WebAssembly, which can then run in the browser. WebAssembly is well-supported by browsers, including mobile browsers, Wang said.

CPython is, of course, a C program and much of the Python numerical stack is written in C or C++. Over the past few years, projects like Pyodide (which we looked at a little over a year ago) and JupyterLite have been compiling large parts of the Python scientific and numerical stack to target WebAssembly.

If you go to the Pyodide site, you can get a Python read-eval-print loop (REPL) in your browser. From those "three little friendly angle brackets", you can import NumPy and pandas. From the JupyterLite site, you can get a notebook in the browser running JupyterLab all on your local system.

Python core developer Christian Heimes has been giving talks and doing a lot of work on getting CPython working with WebAssembly. It will soon be a tier-2 supported platform for CPython, Wang said. WebAssembly simply provides a another computer architecture, beyond x86, Arm, and others, that the CPython project can target

PyScript

So he and others at Anaconda have been looking at the work that is being done and have been thinking about ways to make it all more accessible to "many many more people". To that end, he announced PyScript, but he did so by live-coding a "hello world" demo from the keynote stage. It was his first PyCon keynote, "maybe my last", he said with a chuckle as he typed in a short HTML file that loaded a pyscript.js file from pyscript.net in a <script> tag; the body simply contained:

    <py-script>
        print("Hello PyCon 2022!")
    </py-script>

He proceeded to double-click the file and the greeting appeared in a browser tab; that was met with a round of applause. But this is all just HTML, he said, so he surrounded the code above with a <blink> tag and reloaded. These days, of course, the <blink> tag has been removed from HTML, perhaps sadly; "now I have to explain to kids that there is no <blink> tag".

So he proceeded to add blinking functionality to the PyScript code, and demonstrated a few other things along the way. He created a <div> with a name, that he then targeted for writing the string by accessing the DOM to retrieve the object for the <div>; he also used the asyncio module to sleep for a second, then cleared the <div>, and put that all in a "forever" loop. That worked like a charm, of course. With a laugh, he said: "What the W3C takes away, PyScript gives you back."

So PyScript is a "framework for creating rich Python apps in the browser". It allows interleaving Python and HTML, provides full access to the DOM, and gives the code access to JavaScript libraries—in both directions. The Python code can call JavaScript or be called from JavaScript. So all of the application logic and code can be in one language, in the browser—no web server is needed. You can put the HTML file on a USB stick and give it to your friend. There is the need to download PyScript itself, but that is done from the HTML file using the <script> tag.

PyScript is not some kind of fork of CPython, it is the same code as attendees were running on their laptops and servers, Wang said, just compiled for Wasm. It includes all of the work that Pyodide has done to get the core numerical, scientific, and big data packages working for Wasm as well. PyScript is an "opinionated framework" that provides a foreign function interface (FFI) to talk to JavaScript and the DOM; Python has already wrapped C, C++, and Fortran, so JavaScript can also be added to the list. "This is truly serverless computing."

More demos

He gave some other small demonstrations, many of which come from the PyScript examples page. He started with a REPL, typing a few simple Python statements before typing in a print() call to display an <iframe> HTML tag, which proceeded to play a video in the browser window. "This is the most number of people that I've simultaneously Rickrolled", he said to laughter and applause.

He also showed a simple To-Do application and poked around in the code a bit. The HTML file has some boilerplate, then sets up the text field input and "add task" button. The button has a PyScript-specific pys-onClick attribute that has the name of a Python function to call when the button is clicked. For convenience, the Python code lives in a separate todo.py file that contains the function.

The application itself allows adding things to the list, which also puts a checkbox next to them. When the box is clicked, the list entry also gets a strike-through line added to it. All of which can be handled fairly easily in Python by manipulating the DOM, as can be seen in the code.

The PyScript JavaScript and CSS files can be loaded from a local source or from pyscript.net. The output from Python can be routed to different places, which means that stdout and stderr can go to different locations on the page, as is done in the REPL2 example (note that shift-enter is used to execute like in Jupyter and elsewhere). Since all of Python is being loaded, much more sophisticated applications can be run directly in the browser, with no installation required.

He also showed this interactive demo; note that it takes a rather longer time to load and run than the earlier ones. The code for it shows a bunch of stuff in the HTML <head> section, which will be cleaned up eventually, he said, but the core is just standard Python data-handling code using pandas, scikit-learn for performing k-means clustering, Panel for creating a dashboard, and so on. He reiterated that the file could just be handed off to a colleague to run on their own system—albeit after a moderately lengthy delay to gather and initialize all of the pieces.

Beyond that, there are lots of useful JavaScript libraries available, such as the "powerful visualization system", deck.gl. He showed a PyScript example that displayed data from the New York City taxicab data set using deck.gl and dashboard created with Panel (seen below). It showed a 3D histogram of the island of Manhattan, with the height of the hexagonal bins representing the number of pickups and dropoffs in that part of the city. The visualization can be rotated, zoomed, and animated over time; in addition, the bin size can be changed to increase or decrease the geographic granularity and clicking on a point shows the actual trip details. All of this is done locally in the browser using Python and JavaScript.

But the demo also has Python REPL in its interface, which gives access to the data more directly. The pandas dataframe can be accessed as df and the data can be filtered based on various criteria, such as "trips less than five miles" (in Python), which is then reflected in the display that can be interacted with as usual. Once again, the HTML file is all that is needed to run it. PyScript provides a "dramatic simplification" of what has been a difficult problem for Python: bundling up and distributing applications to users.

Python has never been about reimplementing the world, he said; instead, Python binds together existing tools and libraries. So one of PyScript developers built a wrapper around the D3.js library for "data-driven documents" in two days. "A lot of Python data scientists have been lusting for D3 for a very long time."

Python can also act as the glue between JavaScript libraries using PyScript. He mashed up a JavaScript Mario game with a JavaScript gesture-recognition library, so he had the game running in the browser along with a feed from his laptop camera that was being analyzed for hand gestures. By opening his hand, he could make Mario jump, for example.

He showed the code, which had a lot of JavaScript that needed to be loaded for the game and gesture-recognition library. But the core of the application was in Python in the HTML file. It took the return from the gesture library and determined what that meant in terms of Mario's movement, which it passed down to the game. So the Python was used to bind the two JavaScript libraries together to do "real-time camera video recognition to play a game". All of that is "very cool".

The future

"The most accessible language for the web should be the language we already love and know; let's make that happen." Beyond that, though, traditional web applications are complex, not just because JavaScript is "kind of a terrible language" (though he is willing to discuss that assertion over beer), but because the existing architecture splits the application state between the client and the server. The current web-development landscape is a hodgepodge of tools, languages, frameworks, and so on that is "much more complex than I think it needs to be".

That complexity also means that clients send lots of information back to the server; in the 1980s or 1990s, that state would stay on the client side in the application itself. Sending all of that information to the server is "the root of a lot of the modern evils in tech". He wondered if using PyScript in the browser could eliminate the need for talking to servers in that way, though there would still be a need to access databases and such; that stuff all needs to be figured out, he said. But doing so could perhaps "bring about a version of web 3—without the NFTs". It would allow building applications that are "properly client-server", he said to applause.

There is still an enormous amount of work to be done, Wang said. That includes things like wrapping all of the JavaScript libraries and making them "even cooler", possibly adding support for other languages like Julia or R, and more. There are also lots of interesting problems to solve, like what is the nicest native binding for React or how memory transfers back and forth between JavaScript and Python can be more efficient. The PyScript project is looking for more developers to collaborate on solving these and other problems in the future.

There are also other Python implementations written in C and C++ that could be compiled for Wasm. Since many of the CPython extensions have been built for Wasm, that has the potential to level the playing field for the other Python implementations, most of which are not able to use the C-based extensions. That giant ecosystem of extensions is part of what has kept adoption low for the alternatives, but that could perhaps change.

There are a ton of projects, but the community needs to be thoughtful about how it approaches developing them. There is a risk of burning out developers that needs to be considered. But Wang thinks that most everyone agrees there is an "absolutely massive prize" to be won. It would be a way to solve the most important problem: programming for the 99%.

Because Python has long focused on being easy to learn, and thus is approachable as a teaching language, it can be—is—learned by lots of non-programmers. Python can be used by people who do not consider themselves programmers but who can do a bit with their computers. It is this dynamic that has helped drive Python to a dominant position among computer languages over the last ten or 15 years, he said.

He noted that the number of programmers in the world is around 25 million, which is 0.3% of the population. So the rest of humanity has to rely on this tiny sliver for all of the code that is increasingly pervading everything. "This is not a good state of affairs." It will continue to be that way unless we do something about it.

When he lived in Boston, he was inspired by the huge engraving on the Boston Public Library that says: "The commonwealth requires the education of the people as the safeguard of order and liberty." In his view, the democratization and literacy, including computational and data literacy, are "foundational to assure an open and free future for humankind". He would like to see PyScript play a role in that: "computing for the people and for their communities".

He hopes that kids get their first introduction to programming using PyScript. They do not need to install anything on their tablet or laptop—or they can use the computer at the library. The existing educational materials on HTML, CSS, and Python can be used, mostly as they are today. The idea would be to focus on productivity and quality of life for casual programmers rather than experienced software developers.

His vision is of a web that is "a friendly, hackable place where anyone can make interesting things". They can then share those interesting things with others easily. The remix aspect of our community has gotten lost over the last 20 years, he said, as the stacks have gotten more and more complicated.

He wants to see a return to the quirky, creative aspects of the web, and to "put the joy back into it". His final "demo" was a PyScript REPL, where he typed "import antigravity", which led to the classic xkcd about Python (note that the import is something of an Easter egg in CPython as well). But when he typed "antigravity.fly()", the flying figure rose higher in the sky in a little mini-animation. "Let's do more of that", he said to thunderous applause.

It was an interesting and fairly inspiring keynote that I was sadly unable to write up until now—almost two months after it happened. Interested readers will find it worth viewing the YouTube video of the talk to get a look at some of the demos. Playing with the examples will be enlightening as well. It will be interesting to see where PyScript goes from here.

[I would like to thank LWN subscribers for supporting my trip to Salt Lake City for PyCon.]

---

(Log in to post comments)