Extent-Local Variables (Incubator)

Summary

Introduce extent-local variables to the Java Platform. Extent-local variables provide a way to share immutable data within and across threads. They are preferred to thread-local variables, especially when using large numbers of virtual threads. This is an incubating API.

Goals

• Ease of use — Provide a programming model to share data both within a thread and with child threads, so as to simplify reasoning about data flow.
• Comprehensibility — Make the lifetime of shared data visible from the syntactic structure of code.
• Robustness — Ensure that data shared by a caller can be retrieved only by legitimate callees.
• Performance — Treat shared data as immutable so as to allow sharing by a large number of threads, and to enable runtime optimizations.

Non-Goals

• It is not a goal to change the Java programming language.
• It is not a goal to require migration away from thread-local variables or to deprecate the ThreadLocal API.

Motivation

Large Java programs typically consist of distinct and complementary components that need to share data between themselves. For example, a web framework might include a server component, implemented in the _blank" rel="nofollow noopener" data-shared-secret="1659720814541-0.33629620563250284">thread-per-request style, and a data access component, which handles persistence. Throughout the framework, user authentication and authorization relies on a Principal object shared between components. The server component creates a Principal for each thread that handles a request, and the data access component refers to a thread's Principal to control access to the database.

The diagram below shows the framework handling two requests, each in its own thread. Request handling flows upward, from the server component (Server.serve) to user code (Application.handle) to the data access component (DBAccess.open). The data access component has responsibility for determining whether the thread is permitted to access the database, as follows:

• In Thread 1, the ADMIN principal created by the server component allows database access. The dashed line indicates the principal is to be shared with the data access component, which inspects it and proceeds to call DBAccess.newConnection.

• In Thread 2, the GUEST principal created by the server component does not allow database access. The data access component inspects the principal, realizes that user code must not proceed, and throws an InvalidPrincipalException.

Thread 1                                 Thread 2
--------                                 --------
8. DBAccess.newConnection()              8. InvalidPrincipalException()
7. DBAccess.open() <----------+          7. DBAccess.open() <----------+
   ...                        |             ...                        |
   ...                  Principal(ADMIN)    ...                  Principal(GUEST)
2. Application.handle(..)     |          2. Application.handle(..)     |
1. Server.serve(..) ----------+          1. Server.serve(..) ----------+

Normally, data is shared between caller and callee by passing it as method arguments, but this is not viable for a Principal shared between the server component and the data access component because the server component calls untrusted user code first. The server component needs a better way to share data with the data access component than wiring it into a cascade of untrusted method invocations.

Thread-local variables for sharing

Developers have traditionally used thread-local variables, introduced in Java 1.2, to help components share data without resorting to method arguments. A thread-local variable is a variable of type <code class="prettyprint" data-shared-secret="1659720814541-0.33629620563250284">ThreadLocal</code>. Despite looking like an ordinary variable, a thread-local variable has multiple incarnations, one per thread; the particular incarnation that is used depends on which thread calls ThreadLocal.get/ThreadLocal.set to read/write the variable. Code in one thread automatically reads and writes its incarnation, while code in another thread automatically reads and writes its incarnation. Typically, a thread-local variable is declared as a final static field so it can easily be reached from many components.

Here is an example of how the server component and the data access component, both running in the same request-handling thread, can use a thread-local variable to share a Principal. The server component first declares a thread-local variable, PRINCIPAL (1). When Server.serve is executed in a request-handling thread, it writes a suitable Principal to the thread-local variable (2), then calls user code. If and when user code calls DBAccess.open, the data access component reads the thread-local variable (3) to obtain the Principal of the request-handling thread. Only if the Principal indicates suitable permissions is database access permitted (4).

class Server {
    final static ThreadLocal<Principal> PRINCIPAL = new ThreadLocal<>();  // 1

    void serve(Request request, Response response) {
        var level     = (request.isAuthorized() ? ADMIN : GUEST);
        var principal = new Principal(level);
        PRINCIPAL.set(principal);  // 2
        Application.handle(request, response);
    }
}

class DBAccess {
    DBConnection open() {
        var principal = Server.PRINCIPAL.get();  // 3
        if (!principal.canOpen()) throw new InvalidPrincipalException();
        return newConnection(...);  // 4
    }
}

Using a thread-local variable avoids the need to pass a Principal as a method argument when the server component calls user code, and when user code calls the data access component. The thread-local variable serves as a kind of "hidden method argument": A thread which calls PRINCIPAL.set(...) in Server.serve and PRINCIPAL.get() in DBAccess.open will automatically see its own incarnation of the PRINCIPAL variable. In effect, the ThreadLocal field serves as a key that is used to look up a Principal value for the current thread.

Problems with thread-local variables

Unfortunately, thread-local variables have numerous design flaws that are impossible to avoid:

• Unconstrained mutability — Every thread-local variable is mutable: Any code that can call ThreadLocal.get on a thread-local variable can call ThreadLocal.set at any time. The ThreadLocal API allows this in order to support a fully general model of communication, where data can flow in any direction between components. However, this can lead to spaghetti-like data flow, and to code where it is hard to discern who updates shared state and in what order. The more common need, shown in the example, is a simple one-way broadcast of data from one component to others.

• Unbounded lifetime — Once a thread's incarnation of a thread-local variable is written with ThreadLocal.set, the incarnation is retained for the lifetime of the thread, or until code in the thread calls ThreadLocal.remove. Unfortunately, developers often forget to call remove, so per-thread data is often retained for longer than necessary. In addition, for programs that rely on the unconstrained mutability of thread-local variables, there may be no clear point at which it is safe for a thread to call remove; this can cause a long-term memory leak, as per-thread data will not be garbage-collected until the thread exits. It would be better if the writing and reading of per-thread data occurred in a bounded period during execution of the thread, avoiding the possibility of leaks.

• Expensive Inheritance — The overhead of thread-local variables may be worse when using large numbers of threads, because thread-local variables of a parent thread can be inherited by child threads. (A thread-local variable is not, in fact, "local" to one thread.) When a developer chooses to create a child thread that inherits thread-local variables, the child thread has to allocate storage for every thread-local variable previously written in the parent thread. This can add significant memory footprint. Child threads cannot share the storage used by the parent thread because thread-local variables are mutable, and the ThreadLocal API requires that mutation in one thread is not seen in the other thread. This is unfortunate because child threads rarely call ThreadLocal.set on their inherited thread-local variables.

Toward lightweight sharing

The problems of thread-local variables have become more pressing with the availability of virtual threads. Virtual threads are a lightweight implementation of threads provided by the JDK. Many virtual threads share the same OS thread, allowing for very large numbers of virtual threads. In addition to being plentiful, virtual threads are cheap enough to represent any concurrent unit of behavior. This means that a web framework could dedicate a new virtual thread to the task of handling a request, and still be able to process thousands or millions of requests at once. In the ongoing example, the methods Server.serve, Application.handle, and DBAccess.open would all execute in the same virtual thread.

It would obviously be useful for these methods to be able to share data whether they execute in virtual threads or traditional platform threads. Because virtual threads are instances of Thread, a virtual thread can have thread-local variables; in fact, the short-lived non-pooled nature of virtual threads makes the problem of long-term memory leaks, mentioned above, less acute. (Calling ThreadLocal.remove is unnecessary when a thread terminates quickly, as termination automatically removes its thread-local variables.) However, if each of a million virtual threads has mutable thread-local variables, the memory footprint may be significant.

In summary, thread-local variables have more complexity than is usually needed for sharing data, and significant costs that cannot be avoided. It would be ideal if the Java Platform provided a way to have per-thread data for thousands or millions of virtual threads that is immutable and, given the low cost of forking virtual threads, inheritable. Because these per-thread variables would be immutable, their data could be shared by child threads efficiently. Further, the lifetime of these per-thread variables should be bounded: any data shared via a per-thread variable should become unusable once the method that initially shared the data is finished.

Description

An extent-local variable allows data to be safely and efficiently shared between components in a large program without resorting to method arguments. It is a variable of type <code class="prettyprint" data-shared-secret="1659720814541-0.33629620563250284">ExtentLocal</code>. Typically, it is declared as a final static field so it can easily be reached from many components.

Like a thread-local variable, an extent-local variable has multiple incarnations, one per thread. The particular incarnation that is used depends on which thread calls the methods of ExtentLocal. Unlike a thread-local variable, an extent-local variable is written once and is then immutable for a bounded period during execution of the thread.

An extent-local variable is used as shown below. Some code calls ExtentLocal.where to write a value to the current thread's incarnation of the variable; this binds the extent-local variable for the lifetime of the call to ExtentLocal.run in that thread. The lambda expression passed to run, and any method called directly or indirectly from the lambda expression, can read the extent-local variable with ExtentLocal.get. After run finishes, the binding is destroyed and where/run can be called again.

final static ExtentLocal<...> V = new ExtentLocal<>();

// In some method
ExtentLocal.where(V, <value>).run(() -> { ... V.get() ... call methods ... });

// In a method called directly or indirectly from the lambda expression
... V.get() ...

The syntactic structure of the code delineates the period when a thread can read its incarnation of an extent-local variable. This bounded lifetime, combined with immutability, greatly simplifies reasoning about thread behavior. The one-way broadcast of data from caller to callees (both direct and indirect) is obvious at a glance. There is no ExtentLocal.set method that lets faraway code change the extent-local variable at any time. Immutability also helps performance: reading an extent-local variable with get is usually as fast as reading a local variable, regardless of the call stack distance between where and get.

The meaning of "extent"

The term extent-local variable draws on the idea of an extent in the Java Virtual Machine, to appear in the JVM Specification as follows:

It is often useful to describe the situation where, in a given
thread, a given method m1 invokes a method m2, which invokes a
method m3, and so on until the method invocation chain includes
the current method mn. None of m1..mn have yet completed; all
of their frames are still stored on the Java Virtual Machine
stack. Collectively, their frames are called an _extent_. The
frame for the current method mn is called the _top most frame_
of the extent. The frame for the given method m1 is called the
_bottom most frame_ of the extent.

Accordingly, an extent-local variable that is written in m1 (associated with the bottom most frame of the extent) can be read in m2, m3, and so on up to mn (associated with the top most frame of the extent).

The diagram shown earlier, of two threads executing code from different components, depicts two extents. In both extents, the bottom most frame belongs to the method Server.serve. In the extent for Thread 1, the top most frame belongs to DBAccess.newConnection, while for Thread 2 the top most frame belongs to the constructor of InvalidPrincipalException.

Web framework example with extent-local variables

The framework code shown earlier can easily be rewritten to use an extent-local variable instead of a thread-local variable. At (1), the server component declares an extent-local variable instead of a thread-local variable. At (2), the server component calls ExtentLocal.where and ExtentLocal.run instead of ThreadLocal.set.

class Server {
    final static ExtentLocal<Principal> PRINCIPAL = new ExtentLocal<>();  // 1

    void serve(Request request, Response response) {
        var level     = (request.isAdmin() ? ADMIN : GUEST);
        var principal = new Principal(level);
        ExtentLocal.where(PRINCIPAL, principal)  // 2
                   .run(() -> Application.handle(request, response));
    }
}

class DBAccess {
    DBConnection open() {
        var principal = Server.PRINCIPAL.get();  // 3
        if (!principal.canOpen()) throw new  InvalidPrincipalException();
        return newConnection(...);
    }
}

Together, where and run provide the one-way sharing that the server and data access components need. The where call binds the extent-local variable for the lifetime of the run call, so PRINCIPAL.get() in any method called from run will read the value bound by where. Accordingly, when Server.serve calls user code, and user code calls DBAccess.open, the value read from the extent-local variable (3) is the value written by Server.serve earlier in the thread.

Note that the binding established by where is usable only in code called from run. If PRINCIPAL.get() appeared in Server.serve after the call to run, an exception would be thrown because PRINCIPAL is no longer bound in the thread.

Rebinding of extent-local variables

The immutability of extent-local variables means that a caller can use an extent-local variable to reliably communicate a constant value to its callees in the same thread. However, there are occasions when one of the callees might need to use the same extent-local variable to communicate a different value to its callees in the thread. The ExtentLocal API allows a new binding to be established for nested calls.

As an example, consider a third component of the web framework: a logging component with a method void log(Supplier<String> formatter). User code passes a lambda expression to the log method; if logging is enabled, the method calls formatter.get() to evaluate the lambda expression, and prints the result. Although the user code may have permission to access the database, the lambda expression should not, since it only needs to format text. Accordingly, the extent-local variable that was initially bound in Server.serve should be rebound to a guest Principal for the lifetime of formatter.get():

8. InvalidPrincipalException()
7. DBAccess.open() <--------------------------+  X---------+
   ...                                        |            |
   ...                                  Principal(GUEST)   |
4. Supplier.get()                             |            |
3. Logger.log(() -> { DBAccess.open(); }) ----+      Principal(ADMIN)
2. Application.handle(..)                                  |
1. Server.serve(..) ---------------------------------------+

Here is the code for log with rebinding. It obtains a guest Principal (1) and rebinds the extent-local variable PRINCIPAL to the guest value (2). For the lifetime of the call call (3), PRINCIPAL.get() will read this new value. Thus, if the user code passes a lambda expression to log that performs DBAccess.open(), the check in DBAccess.open will read the guest Principal from PRINCIPAL and throw an InvalidPrincipalException.

class Logger {
    void log(Supplier<String> formatter) {
        if (loggingEnabled) {
            var guest = Principal.createGuest();  // 1
            var message = ExtentLocal.where(Server.PRINCIPAL, guest)  // 2
                                     .call(() -> formatter.get());  // 3
            write(logFile, "%s %s".format(timeStamp(), message));
        }
    }
}

(ExtentLocal.call is used instead of ExtentLocal.run because the result of the lambda expression is needed.)

The syntactic structure of where and call means that rebinding is only visible in the nested extent. The body of log cannot change the binding seen by log itself, but can change the binding seen by log's callees, such as the call method. This guarantees a bounded lifetime for sharing of the new value.

Inheritance of extent-local variables

The web framework example dedicates a thread to handling each request, so the same thread may execute framework code from the server component, then user code from the application developer, then more framework code from the data access component. However, user code will often want to exploit the lightweight nature of virtual threads by creating its own virtual threads and running its own code in them. These virtual threads will be child threads of the request-handling thread.

Data shared by a component running in the request-handling thread needs to be available to components running in child threads. Otherwise, when user code running in a child thread calls the data access component, that component -- now also running in the child thread -- will be unable to check the Principal shared by the server component running in the request-handling thread. To enable cross-thread sharing, extent-local variables can be inherited by child threads.

The preferred mechanism for user code to create virtual threads is the Structured Concurrency API, notably the class StructuredTaskScope. Extent-local variables in the parent thread are automatically inherited by child threads created with StructuredTaskScope. Code in a child thread can use bindings established for an extent-local variable in the parent thread with minimal overhead. Unlike with thread-local variables, there is no copying of a parent thread's extent-local bindings to the child thread.

Here is an example of extent-local inheritance occurring behind the scenes in user code (a variant of the Application.handle method called from Server.serve). The user code calls StructuredTaskScope.fork (1,2) to run the findUser() and fetchOrder() methods concurrently, in their own virtual threads. Each method calls the data access component (3), which as before consults the extent-local variable PRINCIPAL (4). Further details of the user code are not discussed here; see JEP 428 for information.

class Application {
    Response handle() throws ExecutionException, InterruptedException {
        try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
            Future<String>  user  = scope.fork(() -> findUser());    // 1
            Future<Integer> order = scope.fork(() -> fetchOrder());  // 2
            scope.join().throwIfFailed();  // Wait for both forks
            return new Response(user.resultNow(), order.resultNow());
        }
    }

    String findUser() {
        ... DBAccess.open() ...  // 3
    }
}

class DBAccess {
    DBConnection open() {
        var principal = Server.PRINCIPAL.get();  // 4
        if (!principal.canOpen()) throw new  InvalidPrincipalException();
        return newConnection(...);
    }
}

StructuredTaskScope.fork ensures that the binding of the extent-local variable PRINCIPAL made in the request-handling thread -- when <code class="prettyprint" data-shared-secret="1659720814541-0.33629620563250284">Server.serve</code> called <code class="prettyprint" data-shared-secret="1659720814541-0.33629620563250284">ExtentLocal.where</code> -- is automatically visible to PRINCIPAL.get() in the child thread. The following diagram shows the cross-thread extent of the extent-local variable:

Thread 1                           Thread 2
--------                           --------
                                   8. DBAccess.newConnection()
                                   7. DBAccess.open() <----------+
                                   ...                           |
                                   ...                     Principal(ADMIN)
                                   4. Application.findUser()     |
3. StructuredTaskScope.fork(..)                                  |
2. Application.handle(..)                                        |
1. Server.serve(..) ---------------------------------------------+

The "fork/join" model offered by StructuredTaskScope means that the value bound by ExtentLocal.where has a determinate lifetime. The Principal is available while the child thread is running, and the call to StructuredTaskScope.join ensures that child threads terminate and thus no longer use it. This avoids the problem of unbounded lifetime seen when using thread-local variables.

Migrating to extent-local variables

Extent-local variables are likely to be useful and preferable in many scenarios where thread-local variables are used today. Beyond serving as "hidden method arguments", extent-local variables may assist with:

• Re-entrant code — Sometimes it is desirable to be able to detect recursion, perhaps because a framework is not re-entrant or because it is desirable to limit recursion in some way. An extent-local variable provides a way to do this: set it up as usual, with ExtentLocal.where and ExtentLocal.run, and then deep in the call stack, call ExtentLocal.isBound to check if it has a binding for the current thread. More elaborately, the extent-local variable can model a recursion counter by being repeatedly rebound.

• Nested transactions — The detection of recursion would also be useful in the case of flattened transactions: any transaction started while a transaction is in progress becomes part of the outermost transaction.

• Graphics contexts — Another example occurs in graphics, where there is often a drawing context to be shared between parts of the program. Extent-local variables, because of their automatic cleanup and re-entrancy, are better suited to this than thread-local variables.

In general, migration is advised when the purpose of a thread-local variable aligns with the goal of an extent-local variable: one-way broadcast of unchanging data. If a codebase uses thread-local variables in a two-way fashion -- where a callee deep in the call stack transmits data to a faraway caller with ThreadLocal.set -- or in a completely unstructured fashion, then migration is not advised.

There are a few scenarios that favor thread-local variables. An example is caching objects that are expensive to create and use, such as java.text.DateFormat. Notoriously, a DateFormat object is mutable, so it cannot be shared between threads without synchronization. Giving each thread its own DateFormat object, via a thread-local variable that persists for the lifetime of the thread, is often a practical approach.

Alternatives

It is possible to emulate many of the features of extent-local variables with thread-local variables, albeit at some cost in memory footprint, security, and performance.

We experimented with a modified version of the class ThreadLocal that supports some of the characteristics of extent-local variables. However, carrying the additional baggage of thread-local variables results in an implementation that is unduly burdensome, or an API that returns UnsupportedOperationException for much core functionality, or both. It is better, therefore, not to modify ThreadLocal but to give extent-local variables an entirely separate identity.

The idea for extent-local variables was inspired by the way that many Lisp dialects provide support for dynamically scoped free variables; in particular, how such variables behave in a deep-bound, multi-threaded runtime like Interlisp-D. Extent-local variables improve on Lisp's free variables by adding type safety, immutability, encapsulation, and efficient access within and across threads.