6

PipeWire Under The Hood

 3 years ago
source link: https://venam.nixers.net/blog/unix/2021/06/23/pipewire-under-the-hood.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

PipeWire Logo PW but in black because I have a white background

Introduction šŸ”—

The PipeWire project is slowly getting popular as it matures. Its documentation is still relatively sparse but is gradually growing. However, itā€™s always a good idea to have people from outside the project try to grasp and explain it to others in their own words, reiterating ideas, seeing them from their own perspective.

In a previous posts I went over the generic audio stack on Unix and had a section mentioning PipeWire. Unfortunately, because at the time I didnā€™t find enough docs and couldnā€™t wrap my head around some concepts, I think I didnā€™t do justice to the project and might have even confused some parts.
In this post Iā€™ll try to explain PipeWire in the most simple way possible, to make it accessible to others that want to start following this cool new project but that donā€™t know where to start. Itā€™s especially important to do this to open the door for more people to join in and follow the current development, which is happening at a fast pace.

Disclaimer: Iā€™d like to preface this by saying that Iā€™m not a developer involved in the project, only an internet traveler that got interested. I might still have made mistakes and do not cover everything, so be sure to leave comments or emails so that I can correct or add info.
PS: If you like deep dives and similar discussions then check the nixers community, itā€™s full of people that love to do that.
PPS: Iā€™ve use a similar name for this article as the fabulous PulseAudio Under the Hood but I donā€™t think I go in as much details as the author did for PulseAudio.

Table of Content:

Whatā€™s PipeWire ā€” Quick Test Run šŸ”—

PipeWire is a media processing graph, this might not ring a bell so let me rephrase. PipeWire is a daemon that provides the equivalent of shell pipes but for media: audio and video.

What lives in this graph are nodes that can represent multiple things, from real devices such as headsets or webcams, to virtual ones such as audio filters.
These nodes have ports, and these ports can be linked together, the media flowing from the first nodeā€™s source in the direction of the next nodeā€™s sink. What happens in each node is up to them and the interfaces or functionalities they provide.

Practically, the nodes, links, ports, and others, are all different objects extending a basic type, and live in this graph. These objects donā€™t necessarily have to be media-related either, they can do a lot of other things. Weā€™ll see that they use a special type system, plugin system, and marshalling format/storage.

All to say, that PipeWire is a graph, ā€œitā€™s mechanism and not policyā€.
That means that to create and interact with that graph we need another piece of software. The standard way is to rely on what PipeWire calls a session manager or policy manager. The role of such software is to create and manage the entities in the graph depending on the environment, such as when a device is plugged in, or a restore volume policy is set, or permissions needs to be checked before allowing a client to access a device.

Currently, there are two implementations of such session manager: the default one called pipewire-media-session, and a work-in-progress, but extremely promising and interesting one, called WirePlumber.
Yet, you can choose to build your own session management workflow by relying on external tools to manage what is in the PipeWire graph.

Letā€™s cut this short, and have a look at how to run PipeWire.
Get it from your package manager, and do the following in a terminal.

$ pipewire

And in another one do:

$ pipewire-media-session

In some cases, as weā€™ll see from pipewire configuration, you might not need to execute the second command, because pipewire could be set to automatically start pipewire-media-session.
Try a ps beforehand just to be sure.

Still, doing the above wonā€™t get you anywhere if no application can speak with PipeWire ā€” only a handful do at the moment. Furthermore, PipeWire GUIs and toolings are still lacking, as weā€™ll see.
That is why, PipeWire offers three compatibility layers: with ALSA through a PCM device, with PulseAudio through pipewire-pulse server, and with Jack through the pw-jack command.

To make sure you got the ALSA compatibility, check /etc/alsa/conf.d/50-pipewire.conf which should define a pcm for pipewire, often created when installing the pipewire-alsa package. Then check if this has become the default pcm by dumping ALSA configuration through alsactl dump-cfg or aconfdump and verifying the pcm.default entry. However, it doesnā€™t have to if you still want to rely on the PulseAudio shim, in that case it would be type pulse.

For the PulseAudio layer, which allows using PulseAudio tools with PipeWire, install pipewire-pulse and start it. Your distribution package might even start it automatically along with the session manager through the init/service manager.

Finally, for jack, install the pipewire-jack package and issue pw-jack before any jack related command and they will automatically use PipeWire. For example:

pw-jack qjackctl

Additionally, for the video functionality you should install a desktop portal, which is a dbus service implementing the xdg-desktop-portal specifications and whose job is to check if the client requesting access to the video is allowed. There are many such software:

  • xdg-desktop-portal-gtk
  • xdg-desktop-portal-kde
  • xdg-desktop-portal-wlr

You should be familiar with these if you are running a Wayland compositor as this is the only way to access video on Wayland (webcam, screen sharing, screenshot).

Here you go, PipeWire is running!
Yet, thereā€™s a lot more to see than that, such as how to configure PipeWire and the session manager, how to write software that relies on PipeWire, the thinking that goes into it, how to manipulate the graph and rely on tools, and plenty of other examples.

Relation To Gstreamer šŸ”—

The best way to understand the ideas behind PipeWire is to take a look at GStreamer, which it inspires itself from and which is maintained by the same lead developer (a nice and welcoming person).
Even though the author wants to make the comparison with JACK instead. You could take a look at JACK and extract the concepts you want from it, but weā€™ll go for GStreamer in this article.

GStreamer plays around the concept of a pipeline created from objects added in a ā€œbinā€, its version of a graph. Like PipeWire, media flows from one end to another. In Gstreamerā€™s world, instead of nodes there are GstElements, instead of ports there are pads attached to GstElements, and links are connections between pads.

The resemblance goes deeper, GStreamer relies on GObject, which, if you are unaware, is a C framework/programming model to make development easier. It allows it to do multiple things such as registering for asynchronous events (called signals) that are triggered by GstElements present in the pipeline. GObject also has a main loop management, a type system, introspection mechanisms, and other goodies.

GStreamer offers different types of GstElements as plugins created through ā€œfactoriesā€. In fact, these are GObjects extending the common GstElement type to provide useful functionalities. Thereā€™s a list of them available in the GStreamer documentation. For example aasink.
Some are for videos, some for audio, some for logging, some for conversion between format and negotiation, etc..

These elements can also be introspected on the command line via the gst-inspect-1.0 tool. This makes it wonderfully easy to program with GStreamer.

Similarly, PipeWire has plugins that extend a basic node element type. However, it doesnā€™t rely on GObject/GstElement but on its own simpler plugin system, appropriately named SPA (Simple Plugin API). It also has factories, a loop management system, asynchronous events, and a message passing format called POD (Plain Old Data).
Think of PipeWire as a simpler Gstreamer running as a daemon, with a fully controllable loop, and that relies on a session manager to automatically create the graph/pipeline based on streams and devices that appear.

Yet, PipeWire is still missing documentation and introspection tools as excellent as GStreamer. Plugins are barely documented yet. I have to say, GStreamer is a really well-done project and I hope that PipeWire will soon be the same.

You can take a look at the following useful command line tools:

  • gst-discover-1.0
  • gst-inspect-1.0
  • gst-launch-1.0

An example usage of the creation of a pipeline:

gst-launch-1.0 videotestsrc pattern=1 ! optv ! videoconvert ! autovideosink

GStreamer is still relevant with PipeWire, as GStreamer can now integrate with it. ā€œGStreamer is intended to be a Swiss army knife of multimediaā€. New plugins for PipeWire are available under the names:

  • pipewiresrc: PipeWire source
  • pipewiresink: PipeWire sink
  • pipewiredeviceprovider: PipeWire Device Provider

Learning about GStreamer is a great way to better understand PipeWire, at least it was for me.

The Blocks: POD/SPA šŸ”—

The GObject, the GLib Object System, is well-known and battle-tested, but itā€™s not what PipeWire uses because of its heaviness. PipeWire relies on SPA and POD.
So what are SPA (Simple Plugin API) and POD (Plain Old Data)?

POD ā€” Plain Old Data šŸ”—

POD, the Plain Old Data (not to be confused with Perlā€™s Plain Old Documentation) is a generic data container format for marshalling/unmarshalling, serialization/unserialization, storage, and transfer. Itā€™s the usual flat passive data structure.

Think of it as yet another format, similar to XML, JSON, ASN.1, protobuf, and others. It inspires itself from formats such as D-bus Variant and LV2 Atom.

Practically, itā€™s an LTV (Length-Type-Value) format, thus using octet-counting framing, where the length and type are fixed 32bits values and the frames are always 8 bytes aligned. So padding is often added to values that donā€™t align on it. (NB: Framing refers to the concept of start and end of value, how to delimit.)
The type system, what the 32 bits T refers to, is called the SPA type system and has compound/container and basic/primitive types. These range from containers such as a array, struct, object, sequence, pointer, file descriptor, choice, to primitives such as bool, int, string, etc..
The advantage of such format is that itā€™s an ā€œas-isā€ format, it can directly be transferred on the network, read from memory, stored on the stack or on disk without extra marshalling.

NB: If you want to know more about protocol/format design in general refer to RFC 3117.

The POD library wasnā€™t designed to be specifically used for PipeWire, it can be used in any other project, though the question remains of why use this format instead of another.
The library is a small header-only C library without dependencies, which makes it a breeze to test with.
Iā€™ve published an example of its usage based on the official tutorial but letā€™s still go over the general aspect of it.

The best documentation for POD is to consult the headers themselves, as a lot of helpers arenā€™t documented anywhere else. You can usually find them in /usr/include/spa/pod/ or /usr/include/spa-<version>/pod/ (just be sure itā€™s the latest version). It comes bundled in the same directory as SPA as it relies on the type ID system in /usr/include/spa/utils/type.h and defs.h.

The pod structure struct spa_pod is defined in /spa/pod/pod.h along with the primitive and container values. The builder to construct them are found in /spa/pod/builder.h, while the parser is in /spa/pod/parser.h, and the manipulation can be done with helpers in /spa/pod/iter.h, /spa/pod/filter.h, and others.

Practically, to create a pod we initialize any part of memory, be it in the heap or stack, and use a builder helper to initialize it. Then we have to rely on a frame to set the start of a container object and its end, basically setting the final size of the object (LTV as we said) when weā€™re done. The frame acts as a sort of push and pop of value on the memory segment we chose.

Hereā€™s a taste of what it looks like, you can consult my example, the official docs, or the headers directly to know more.
Compilation should be as simple as cc pod-test.c -o pod-test.

We define any sort of storage, here 256B on the stack and tell the pod builder weā€™ll use it to store pods.

uint8_t buffer[256];
struct spa_pod_builder b = SPA_POD_BUILDER_INIT(buffer, sizeof(buffer));

We can then define a frame for a container object, here a simple struct, but it could be objects (key/value, called properties which also have their own IDs) or other complex types.

struct spa_pod_frame f;
spa_pod_builder_push_struct(&b, &f);

Once the frame starts we can add values to the struct.

spa_pod_builder_int(&b, 5);
spa_pod_builder_float(&b, 3.1415f);

Finally, we close the frame to say we are done with the struct, and it returns the struct spa_pod basic type which can later be casted to its appropriate type.

struct spa_pod *pod = spa_pod_builder_pop(&b, &f);
struct spa_pod_struct *example_struct = (struct spa_pod_struct*)pod;

This will look like this in memory (little-endian formatted):

length = 00000020 = 32
type   = 0000000e = 14 = SPA_TYPE_Struct
value  =
	length   = 00000004 = 4
	type     = 00000004 = SPA_TYPE_Int
	value    = 00000005 = 5
	_padding = 00000000 (always align to 8 bytes)

	length   = 00000004 = 4
	type     = 00000006 = SPA_TYPE_Float
	value    = 40490e56 = 3.1415
	_padding = 00000000 (always align to 8 bytes)

The types can be found in /spa/utils/type.h and others files. We can also notice the padding for alignment, which was automatically added by the builder.

Once we have a pod of a specific type created, we can handle it with related helpers. In the iter.h header we can find ways to loop over struct spa_pod_struct or ways to verify that a pod is indeed a struct.

struct spa_pod *entry;
SPA_POD_STRUCT_FOREACH(example_struct, entry) {
	printf("field type:%d\n", entry->type);
	// two ways to get the value, casting or using spa_pod_get_..
	if (spa_pod_is_int(entry)) {
		int32_t ival;
		spa_pod_get_int(entry, &ival);
		printf("found int, pod_int: %d\n", ival);
	}
	if (spa_pod_is_float(entry)) {
		struct spa_pod_float *pod_float = (struct spa_pod_float*)entry;
		printf("found float, pod_float: %f\n", pod_float->value);
	}
}

As you noticed, struct spa_pod are generic and only contain the first two 32bits value of a pod, namely size and type. Thus, we have to interrogate its type to find out what it actually is, and then cast it manually or using helpers such as spa_pod_get_* found in iter.h.

Instead of doing that, we can instead rely on an spa_pod_parser and other helpers to do the validation and inspection of raw data (see in iter.h spa_pod_from_data for example).

Yet, it would be annoying to manually check values when you only want to glance at its format. Thatā€™s why debugging functions are present in /spa/debug/pod.h. The spa_debug_pod_value is particularly useful to print whatā€™s in a pod. We could also use /spa/utils/json.h and /spa/utils/ansi.h to print structures as json, like is shown here, but thereā€™s no simple function to do that yet.

Thereā€™s a lot more to POD than this, itā€™s a whole object format, but letā€™s stop and move to SPA.

SPA - Simple Plugin API šŸ”—

While POD is just about data representation, SPA is about functionality. SPA, the Simple Plugin API, is a header-only and no dependencies framework that gives the ability to load libraries that have a specific format, enumerating factories, creating them, and the interfaces they provide that can be introspected and used, all at runtime.

Like POD, it is not necessarily tied to PipeWire, but can be used anywhere, though PipeWire got its own plugins using SPA.

The objects that are created by the factories in the SPA have a specific format, often internally relying on POD, and the SPA factory interfaces let us know the functionalities associated with them.

Practically, plugins using SPA take the form of dynamically loadable libraries (.so), usually found under /usr/lib/spa/, or the env SPA_PLUGIN_PATH, and opened on the fly using dlopen(3).
The SPA libs have at least one public symbol spa_handle_factory_enum, defined in /spa/support/plugin.h, which is loaded as follows.

#define SPA_PLUGIN_PATH "/usr/lib/spa-0.2/"
void *hnd = dlopen(
	SPA_PLUGIN_PATH"/support/libspa-support.so",
	RTLD_NOW);
spa_handle_factory_enum_func_t enum_func = dlsym(
	hnd,
	SPA_HANDLE_FACTORY_ENUM_FUNC_NAME);

Iā€™ve also written a simple example to show SPA usage, but letā€™s review the mechanism quickly as this is important to grasp PipeWire configuration.
Compiling should also be as simple as cc test-spa.c -ldl -o test-spa.

A plugin is composed of a list of factories, and in each factory we have a list of interfaces available. The idea is that factories are a set of methods used around a struct of a specific type, from its creation, to different interactions with it. Interfaces are about anything that can be bundled together.

Factories have well-known names and the interfaces in them have names too, they are defined in /spa/utils/names.h and the interfaces names are stored in the plugin header themselves in the form SPA_TYPE_INTERFACE_*.
Factories also have additional information such as version, author, description, and more.

This is what we need to know to use the library after loading it. Yet, we have to consult the header files and the library location /usr/lib/spa-<version> to know which factories are currently available and how to use them. In that library directory, folders are present displaying the categories of plugins, most of them related to PipeWire.
If we take a look at support/libspa-support.so, we see that it relates to the support headers we got. We can grep on SPA_TYPE_INTERFACE_ to find the headers that have plugins in them to be sure. Letā€™s see support/log.h as an example.

The factory name is SPA_NAME_SUPPORT_LOG and the only interface defined is SPA_TYPE_INTERFACE_Log. Internally, we see that all plugins using SPA define two things: a struct containing a struct spa_interface and another struct containing the methods related to that interface along with the version, additionally it can define events and callbacks. If youā€™re curious struct spa_interface is defined in /spa/utils/hook.h. However, the inner workings are not important, whatā€™s important is that we can take a look at which functions are in an interface which dictates what we can do with the plugin.

A series of command line tools starting with spa- are present to interact with the plugins. One in particular called spa-inspect can be used to dump the factories and interfaces present in a .so file. Yet, itā€™s not very solid and doesnā€™t confer that much information on what we can do with the plugins.

Example output:

factory version:     1
factory name:     'support.log'
factory info:
  none
factory interfaces:
 interface: 'Spa:Pointer:Interface:Log'
factory instance:
 interface: 'Spa:Pointer:Interface:Log'

Thus, after loading the .so and getting the enum_func function, we can loop over the factories, find the name we are interested in, then loop over its interfaces to see if it has the one we want, and get an instance of the factory to use it.

uint32_t i;
const struct spa_handle_factory *factory = NULL;
for (i = 0;;) {
	if (enum_func(&factory, &i) <= 0)
		break;
	printf("factory name: %s, version: %d\n",
			factory->name,
			factory->version);
	if (strcmp(factory->name, SPA_NAME_SUPPORT_LOG) == 0) {
		const struct spa_interface_info *info = NULL;
		uint32_t index = 0;
		// get interface at position 0
		int interface_available = 
			spa_handle_factory_enum_interface_info(
			factory,
			&info,
			&index);
		if (strcmp(info->type, SPA_TYPE_INTERFACE_Log) == 0) {
			// allocate a handle (struct) pointing
			// to this factory's interfaces
			size_t size = spa_handle_factory_get_size(
				factory, NULL);
			struct spa_handle *handle = calloc(1, size);
			spa_handle_factory_init(factory, handle,
				NULL, // info
				NULL, // support
				0     // n_support
			);
			// fetch the interface by name from the factory handle
			void *iface;
			int interface_exists =
				spa_handle_get_interface(
					handle,
					SPA_TYPE_INTERFACE_Log,
					&iface);
			// finally get something useful by casting it
			struct spa_log *log = iface;
			// use the methods in the interface of the factory
			spa_log_warn(log, "Hello World!");
			spa_log_info(log, "version: %i", log->iface.version);
			// clear the handle to the factory
			spa_handle_clear(handle);
		}
	}
}

This long example has all the parts we discussed and something more:

  • Loop over the factories present in the library and match by name
  • Get the name of the first interface (though we can loop over spa_handle_factory_enum_interface_info until it returns 0) and match by name.
  • Allocate a struct spa_handle * which is a pointer to all interfaces in the factory, a way to construct it. (We have to fetch its size to do that, Iā€™m sure there could be a better helper to make this easier in the future). The factory creation could possibly take parameters.
  • Get our interface from the created factory using spa_handle_get_interface, then cast it to the struct we talked about earlier struct spa_log.
  • Finally, use the methods associated with that struct.

There are a lot of plugins and different ways to use them. Some interfaces are asynchronous and use callbacks through registered event handlers and hooks, some are synchronous.

This is a general idea of what SPA is. It might seem cumbersome and kind of underwhelming: fixed factory names defined in header files, along with fixed interfaces and a common way of fetching them manually from dynamically loadable libraries, to then use the methods in it based on the info.
Now letā€™s see how POD and SPA are actually used within PipeWire.

The PipeWire Lib, An Inspiration From Wayland šŸ”—

PipeWire uses POD for encoding messages (methods and events) and some properties of the objects that live on the server. Meanwhile the SPA libraries are used to create objects in the serverā€™s graph.

Objects that live in the graph are plugins created through the SPA factories: objects that implement specific interfaces. Some of these objects are always present ā€” singletons such as the core and the registry ā€” while others appear dynamically ā€” be it because they get created by modules or by the session manager.
PipeWire, like PulseAudio, has a plugin architecture: the core is small and everything else is a module, in this case SPA plugins. That makes it very extensible and dynamic.

These objects created through SPA factories all have IDs. One object, the core singleton, has a fixed ID of 0 so that clients can always find it.
The client can then ā€œbindā€ ā€” thatā€™s what we call getting a proxy to a remote object ā€” to the registry singleton object to be able to list all objects living server side (to then also bind to them if needed).
Binding is needed to be able to call methods on objects or register to events they emit.

Some objects are related to the media processing, such as nodes, links, and ports ā€” yes, these are all separate entities in the graph. Others are modules to extend the core, or factories to create other objects (factories can live in the graph too), or management related objects for the session manager.
PipeWire objects also have permissions attached to them (read, write, execute, metadata) so that when clients attach to PipeWire they can only access methods and states that are allowed (a module called libpipewire-module-access is responsible of that).

Think of the factories present in the PipeWire graph as the equivalent of GstElement factories in GStreamer.
Some examples of interfaces that are implemented by objects created by these factories:

  • PipeWire:Interface:Core (struct pw_core)
  • PipeWire:Interface:Registry (struct pw_registry)
  • PipeWire:Interface:Module (struct pw_module)
  • PipeWire:Interface:Factory (struct pw_factory)
  • PipeWire:Interface:Client (struct pw_client)
  • PipeWire:Interface:Metadata (struct pw_metadata)
  • PipeWire:Interface:Device (struct pw_device)
  • PipeWire:Interface:Node (struct pw_node)
  • PipeWire:Interface:Port (struct pw_port)
  • PipeWire:Interface:Link (struct pw_link)

The PipeWire daemon offers an event loop where events are processed sequentially, as they arrive. Clients can interact with the remote objects living in the graph, calling methods and registering to receive events, the ones implementing the interfaces listed above, through proxies (bind) that allow calling the methods related to the interface that they follow.

If you know something about Wayland, this way of doing IPC would ring a bell. Youā€™d be right because PipeWire event loop, proxies, registry mechanisms have all been inspired by Wayland asynchronous IPC design.
However, unlike Wayland, the interfaces are still simply in header files and not defined in XML files, so you have to consult them like we did in the previous SPA example.

So far this is the explanation we have:

  • A daemon, what we call PipeWire service/daemon, implements a global graph (thereā€™s also the idea that part of the graph can run in clients but I havenā€™t seen it in the wild yet, the concept is interesting)
  • Clients operate on this graph
  • Media processing elements, and others, live in the graph
  • The elements are created by SPA plugins, have specific IDs, and permissions
  • Clients bind (create a proxy) to these elements to be able to call methods and register for events (hooks) that the elements implement
  • Message sent from client to server is a ā€œmethodā€
  • Message from server to client is an ā€œeventā€
  • The method data, event data, and some of the elements properties are encoded using POD.

As a client, this concretely takes the form of an initial connection to the PipeWire server, which gives back a proxy to the core object (ID=0), and then a continual event loop to poll for new activities.
Iā€™ve written a simple example here but letā€™s go over how the usual flow is done in five steps:

  1. Create a struct pw_loop, there are multiple kinds of loop depending on the criteria. The gist is that itā€™s an abstraction over poll(2). Each have specific ways to run and stop, control points. (somewhat similar to PulseAudio loops concept)
    The available ones are:
    • struct pw_main_loop, defined in /pipewire/main-loop.h
    • struct pw_thread_loop, defined in /pipewire/thread-loop.h
    • struct pw_data_loop, defined in /pipewire/data-loop.h
  2. After getting a loop we pass it to pw_context_new (casting it to struct pw_loop) to create a context object, struct pw_context, which manages the environment related resources.

  3. Then we connect to the PipeWire server using pw_context_connect which will return a proxy to the singleton core object, a struct pw_core.

  4. After getting our initial connection to the core we can start running our loop using the method specific to the type of loop we got. If we are using a struct pw_main_loop then we can call pw_main_loop_run.
    This particular loop only stops when pw_main_loop_quit is called, so itā€™s a good idea to register for some interesting events before starting this loop or to call methods on the core proxy. The usual method to call first is pw_core_get_registry to fetch a proxy to the singleton registry object.

  5. Finally, when the loop ends, we have to cleanup object, destroy the proxies weā€™ve created, disconnect from the core, destroy the context, and destroy the loop.

As you can notice, step 4 is the one that will have the actual logic into it. This is when we setup things before our loop and will dynamically handle events.
Like we said, normally we bind to get a struct pw_registry proxy and register for the ā€œglobalā€ event, which will emit an event for each object that exists in the graph. In the event callback hook to the ā€œglobalā€ event, we can choose to bind to any of the object weā€™re notified about, to do more things: inspect elements properties, call methods on them, or register for events that they can emit.

Additionally, apart from the registry object, the core object itself got interesting events to get notified of new clients binding, object creation, errors, etc.. A useful event is the ā€œdoneā€ event that is emitted after the core handles the ā€œsyncā€ method (along with the same sequence number set). It might sound benign, but because the core processes everything sequentially, that means that anything sent before the ā€œsyncā€ will be finished. Itā€™s a great way to know asynchronously that what was previously sent has reached the server and is indeed ā€œdoneā€.

Thatā€™s about it for the PipeWire library. The useful things that we can do on objects living in the graph depend on which object we are talking about and which interface it implements.
Some of the properties on these objects only make sense to the session manager, others to the node itself, and others to the core. For example, the node object has properties and params, all having different meanings and fetched at different times through different events that we can register to. The volume(s) is part of the params of a struct pw_node.

Again, there are a couple of examples in the official docs, and one I left here.

Configuration šŸ”—

Not everyone will write clients, but a lot of people are interested in configuring the PipeWire server and the session manager.
There are three or four blocks in the PipeWire equation that can be configured:

  • The PipeWire daemon running the core and hosting the processing graph
  • The clients which get their features automatically set according to the conf
  • The session manager to add nodes to the graph, discover, and set them up appropriately
  • The PipeWire PulseAudio server and JACK backward compatibility layer and their configurations

The PipeWire daemon configuration, the clients configuration, and the pipewire-media-session, pipewire-pulse, and WirePlumber configuration all have the same format. While the manpage pipewire.conf(5) explains it briefly, it can still be confusing so letā€™s try to make sense of it.
The format is a series of assignment in the form of name = value. The value can be either another simple assignment, a dictionary { key1=val1 key2=val2 }, an array [ val1 val2 ], or a composite such as an array of dictionaries. As you might have noticed, thereā€™s no comma in this format.

The initial list of names present and their meaning depends on what we are configuring. However, as I found out, most of them got some of the following:

  • context.properties
    A dictionary containing generic properties.
  • context.spa-libs
    A dictionary that tells the program where to find the .so when matching a SPA factories names.
  • context.modules
    An array of dictionaries containing modules to load on startup.
  • context.objects
    An array of dictionaries containing objects that will be created automatically using an SPA factory.
  • context.exec
    An array of dictionaries (not available for client configuration) with additional commands to execute sequentially after launch.

PipeWire Server Configuration šŸ”—

The PipeWire daemon configuration is available in multiple places, either globally in /etc/pipewire/pipewire.conf and /usr/share/pipewire/pipewire.conf, or locally in the $XDG_CONFIG_HOME, usually .config/pipewire/pipewire.conf.
So be sure to copy it to the local user directory before doing modifications, as the global configuration might change rapidly with the current development speed.

Before that, itā€™s good to know PipeWire respects a couple of environment variables including PIPEWIRE_DEBUG which takes a level of verbosity for debugging between 1 and 5 (5 is the most verbose) and could also have an optional category next to it to filter what is being logged, PIPEWIRE_LOG and PIPEWIRE_LOG_SYSTEMD to log in a specific file and to disable or enable systemd logs respectively, PIPEWIRE_LATENCY to configure the default latency (buffer-size/sample-rate or samples in buffer/samplerate, see the previous article to know what that means).

pipewire.conf contains all the configuration names weā€™ve mentioned before. In particular, its context.properties has information about the default aspect of the processing graph.
Letā€™s go over each section and the particularities of each one.

context.properties contains generic aspects of the core: logging level, scheduling settings, default global sample rate, default quantum min and max value (buffer), and more. The data in the pipeline will default to this sample rate and each node will negotiate automatically their own latency accordingly (buffer size within the quantum set in the config), and when finally reaching a device the signal will be converted to its sample rate.
Nodes can set a desired buffer size themselves by setting the property node.latency.

context.spa-libs contains, as we said before, a dictionary mapping factory names as regex to the library location on disk (.so files). Remember how we did the dlopen(3) in the previous section and looked for the available factories. Here itā€™s useful to quickly find how to create elements using the factory in the right library, either directly from the configuration in context.objects, through the command line tools, or others.
For example, we notice support.* = support/libspa-support, which says any call to use a factory that starts with support. will use the library in /usr/lib/spa-0.2/support/libspa-support.so.

context.modules contains an array of modules loaded sequentially as a series of dictionaries that has at least a name and optionally args and flags. The name is used to point which library will be loaded from the system, usually in /usr/lib/pipewire-<version>/. The args is a dictionary containing specific per-module settings, and the flags currently is an array that can have two values in it: ifexists, to only load the module if the library is on disk, and nofail, to not stop loading other modules or crash if there was an error.

There are quite a lot of modules available, each doing different things, some extending the core functionality, others providing ways to create filters, some changing the scheduling, some adding profiling, some doing access control, some providing the adapter around nodes for resampling, some providing factories, and much more.
You can usually find the description and usage of the already loaded modules, though very sparse, by doing pw-cli dump Module and checking the module.description and module.usage properties. Iā€™m currently not aware of a way to dump the possible arguments of a module without first loading it, similar to what spa-inspect does. You can always consult the source and find the PW_KEY_MODULE_USAGE, like here for module-filter-chain.

NB: Technically, modules are dynamic clients that have an exported function with the signature: SPA_EXPORT int pipewire__module_init(struct pw_impl_module *module, const char *args).

This interesting plugin, the libpipewire-module-filter-chain, described here allows to create a sub-graph of LADSPA and built-in plugins (SPA), then later exposing the sources and sinks of that sub-graph to the global PipeWire graph (Is this an instance of part of the graph running in a client, Iā€™m not sure). This makes inserting filtering easier than to rely on separate software but you need to be familiar with how to configure LADSPA library controls (which Iā€™m not).

context.objects, contains a list of objects that will be automatically created when PipeWire starts. It takes at least the factory which is the SPA factory name, and optionally the args passed to it as a dictionary, and flags which can be set to nofail to ignore errors when creating the object.

This section of the configuration is useful to autocreate virtual nodes in the graph or to manual set devices.
For example:

{
	factory = spa-node-factory
	args = {
		factory.name = videotestsrc
		node.name = videotestsrc
		Spa:Pod:Object:Param:Props:patternType = 1
	}
}

NB: Be sure to uncomment the SPA factory videotestsrc for this example.

This creates a sample video with a fixed pattern, similar to GStreamer videotestsrc. Thereā€™s a page in the Wiki that goes over the concept of virtual devices, it explains some of the usual properties that can be set on playback and capture.
NB: Even though it should be the same as the gst-launch-1.0 example from earlier, using gst-launch-1.0 pipewiresrc client-name=hello ! videoconvert ! autovideosink and connecting the videotestsrc to hello just displays an empty screen (yet connecting the webcam directly does indeed display the screen properly). It does work with audiotestsrc though, using gst-launch-1.0 pipewiresrc client-name=hello3 ! audioconvert ! autoaudiosink.

Lastly, context.exec contains an array of programs that will be launched by PipeWire after startup. It takes at least a path with the program along with optional args passed to it. Iā€™m not actually sure why this section is present when everywhere I look it seems like itā€™s recommended to rely on the service manager instead to start other software. I donā€™t think it should be the role of the PipeWire daemon to manage other services. Yet, itā€™s there, so good to know about.

Letā€™s note that pipewire-pulse is a copy of the pipewire binary with the only difference that it has a different name. PipeWire loads its configuration based on its name, so it will load pipewire-pulse.conf, which activates the libpipewire-module-protocol-pulse.

PipeWire Client Configuration šŸ”—

As with the daemon configuration, the client configuration is found in the same directory under the names client.conf and client-rt.conf.
The difference between these two is that one loads the libpipewire-module-rtkit module and the other doesnā€™t.

PipeWire clients either load a specific config file or load client.conf by default. They do this configuration step before joining the graph and being connected to other nodes by the policies of the session manager.

These files contain the usual sections weā€™ve seen before but also has two other sections fitler.properties and stream.properties which configure how filters and streams should be handled internally. The clients are the ones doing most of the sample conversion, mixing, and resampling, so these sections are about how it will be done.
From buffer size, to sample quality, to channel mixing, etc..

A quick explanation of some of the properties is found in the official docs here.

PipeWire Session Manager Configuration šŸ”—

The session manager is the piece of software that is responsible for the policy: to find and configure devices, attach them appropriately to the graph, set and restore their properties if needed, route streams to the right device, set their volume, and more.
It can create itā€™s own objects in the PipeWire graph related to session management such as endpoints and links between them, a sort of abstraction on top of PipeWire nodes.

There are currently two implementations of the session manager: pipewire-media session and WirePlumber. Each have a different policy format, pipewire-media-session having static matching rules while WirePlumber providing dynamic lua scripts for policy.
Yet the global idea is the same: Finding and setting up devices depending on where they come from (ALSA, JACK, Bluetooth), set their profiles/ports/name/volume, add them to the graph, connect devices and streams as they appear according to defined rules or restoration info, also do the same for properties of streams.

Think of the session manager configuration as the rulebook for what happens in the graph.

pipewire-media-session Configuration šŸ”—

pipewire-media-session ships alongside the pipewire daemon, at least for now with most package managers. Its configurations are found in the same place as the client conf and the daemon conf but under a directory named media-session.d. In this directory we find the entry-point config called media-session.conf along with three other configurations for the different device drivers that the software manages: alsa-monitor.conf, bluez-monitor.conf, v4l2-monitor.conf.

The configuration format in the media-session.conf should look familiar by now with all the usual section. The additional one here is the session.modules which is a dictionary of module categories (bundles) that are enabled when specific files with the same name as the key of the dictionary entry exists in the media-session.d directory (yes, itā€™s an interesting way to enable features). There is a default key which is always enabled, as the name implies.

Each module activates a specific functionality, hereā€™s a glimpse of the ones listed when issuing pipewire-media-session --help:

  • flatpak : manage flatpak access
  • portal : manage portal permissions
  • metadata : export metadata API
  • default-nodes : restore default nodes
  • default-profile: restore default profiles
  • default-routes : restore default route
  • restore-stream : restore stream settings
  • streams-follow-: move streams when default changes
  • alsa-seq : alsa seq midi support
  • alsa-monitor : alsa card udev detection
  • v4l2 : video for linux udev detection
  • libcamera : libcamera udev detection
  • bluez5 : bluetooth support
  • suspend-node : suspend inactive nodes
  • policy-node : configure and link nodes
  • pulse-bridge : accept pulseaudio clients
  • logind : systemd-logind seat support

There isnā€™t anything else in the config file, however, some of the modules give us the ability to have matching rules to set additional properties on devices and clients, apart from the usual client.conf. These rules are stored and read in separate files such as: alsa-monitor.conf, bluez-monitor.conf, and v4l2-monitor.conf, which should be consulted when a module with the same name is set, for example alsa-monitor module. (Though the last two are not mentioned in the list above, but somehow hinted at here for Bluetooth)

These files have two sections: properties which are properties that are always set on such device, and a rules section. For example, in the alsa-monitor.conf the properties has values related to alsa device reservation, more on that here.

The rules configuration is an array of dictionaries, the dictionaries containing two parts: matches, a list of regex and matching rules (AND/OR conditioning too), and action, which currently only has an update-props entry to set properties or remove them (when setting to null) on the device.

The properties that can be set are either generic node properties or dependent on the type of device we are handling. For example, ALSA got some properties listed in this doc. Iā€™m currently not aware if thereā€™s a way to list all of the properties that could possibly be set for a specific device driver, looking at the code they seemed to be defined and read in different places. This is something you should remember, that properties on nodes are read and interpreted differently by multiple software/modules/libraries handling them.

Another set of modules from the pipewire-media-session that are interesting are the ones related to restoration rules. These are similar to PulseAudio restoration mechanism, and actually reuses its code, however it stores the rules in a JSON format in the home directory (usually under ~/.config/pipewire/media-session.d/). This saves you from the binary format of PulseAudio, which I had to create a custom db editor to be able to edit them.
The files are the following:

  • default-nodes: Stores the default sink and source.
  • default-profile: Stores the default profiles for devices.
  • default-routes: List profiles and their settings/volumes for each device.
  • restore-stream: List of output and input stream names along with the last volume set on them. Additionally, it could have information such as the target-node which should, in theory, re-attach the stream to the right node when it appears (though I had issues with this not being respected).

Apart from these we also have the streams-follow-default configuration which will keep moving streams to whatever the default device is, even when it changes. This can be extremely annoying though, but my guess is that itā€™s somewhat a replacement for PulseAudio concept of module-always-sink and module-always-source which create a virtual device that streams are moved to when all other devices are disconnected. Regardless, this breaks the usual restoration rules flow for devices and streams, and is unintuitive.
You can always consult these PulseAudio restoration flowcharts when in doubts:

WirePlumber Configuration šŸ”—

WirePlumber is a more advanced session manager that is based on Glib/GObject to wrap PipeWire types/functionalities and relies on lua plugins for most of its extra logic.
That means, on one side that you can debug WirePlumber using the usual Glib stuff, such as the environment variables such as G_MESSAGES_DEBUG=all, though it is deprecated in favor of WIREPLUMBER_DEBUG, and on another side that you can extend its functionality to suit your need by adding lua plugins.

The project is still considered in early stage and changes at a rapid pace. The documentation is slowly taking form here.
Like pipewire-media-session, it also has mechanisms to set policies, discover devices, and plugins to extend functionalities, however it does it in its own way by creating wrappers over the native PipeWire library.

The configuration files are found either in the local user directory ~/.config/wireplumber or globally in /etc/wireplumber. The main configuration file wireplumber.conf follows the same format as the ones weā€™ve previously seen and has as an additional property the wireplumber.components which, as far as I understood, is currently used to bootstrap the lua scripting engine through libwireplumber-module-lua-scripting (found in /usr/lib/wireplumber-<version>).

Then, sub-directories with lua scripts will be loaded in alphabetical order (bluetooth.lua.d/, policy.lua.d/, main.lua.d/). The last script in the list usually enables and activate whatever was set in the previous ones.
For example, policy.lua.d will pass through the following files, in this order:

  • 00-functions.lua
  • 10-default-policy.lua
  • 50-endpoints-config.lua
  • 90-enable-all.lua

The last file 90-enable-all.lua calls a function defined in 10-default-policy.lua, default_policy.enable() which loads modules and scripts from /usr/share/wireplumber/scripts/ to set the policies in place. This is a bit confusing, partly because itā€™s not well documented, and partly because the flow is not clear. Yet, the gist is that itā€™s loading the policy set in these lua files similar to what pipewire-media-session was doing.
A better example would be to look at /etc/wireplumber/main.lua.d/50-alsa-config.lua which is the equivalent of alsa-monitor.conf. It sets properties and rules in the alsa_monitor object that is created in the file 30-alsa-monitor.lua, and then calls a function defined in that file alsa_monitor.enable() in the final file 90-enable-all.lua to load the monitor script and make it all fall into placeā€¦ Again, not so obvious but still makes sense when doing the comparison with pipewire-media-session.

While lua is used for configuration, it can also be used for scripting additional functionalities or side programs that rely on WirePlumber, all running in their own sandboxed environment.
As we said, WirePlumber extends PipeWire objects, mapping them to GObjects which it documents here in its C API. It also implements the Endpoint session related abstraction over objects in the graph. As with all GObjects, you can call methods and register for signals on them. Some are globally accessible when using the library and provide a gateway to the WirePlumber daemon.

The Lua API is a map of the C API unto LUA, along with helpers found in /usr/share/wireplumber/scripts/. Methods are called with the object:method notation and the signals are registered using object:connect("signal_name", function()...). Letā€™s show a couple of examples of these global GObjects available through Lua.

The Core and ObjectManager are currently the most interesting to take a look at, along with debugging log functions.

The Core provides a wrapper around WirePlumber core, which, as I could see, is very useful to load modules via the Core.require_api and call methods on them. There is currently no documentation for the available plugins and which calls and signals they provide, but you can always check the source for *_api_class_init functions, like here for the mixer api
An example of this can be found here, which I reproduce under:

#!/usr/bin/wpexec
--
-- WirePlumber
--
-- Copyright Ā© 2021 Collabora Ltd.
--    @author George Kiagiadakis <[email protected]>
--
-- SPDX-License-Identifier: MIT
--

-- Load the necessary wireplumber api modules
Core.require_api("default-nodes", "mixer", function(...)
  local default_nodes, mixer = ...

  -- configure volumes to be printed in the cubic scale
  -- this is also what the pulseaudio API shows
  mixer.scale = "cubic"

  local id = default_nodes:call("get-default-node", "Audio/Sink")
  local volume = mixer:call("get-volume", id)

  -- dump everything
  Debug.dump_table(volume)

  -- or maybe just the volume...
  -- print(volume.volume)

  Core.quit()
end)

As you can notice, these scripts are run by passing it to the wpexec command line tool.
It can also be used to create nodes on the fly:

local props = {
  ["media.class"] = "Audio/Sink",
  ["factory.name"] = "support.null-audio-sink",
  ["node.name"] = "ExampleNode",
  ["node.description"] = "ExampleNode",
  ["audio.position"] = "FL,FR",
}
node = Node("adapter", props)
node:activate(Features.ALL)

The ObjectManager is somewhat the equivalent of the PipeWire registry but made easy, letting you listen and filter nodes that are of interest to interact with them, possibly modifying their properties.
The ObjectManger takes a list of Interest, which are filters.

obj_mgr = ObjectManager {
  Interest {
    type = "node",
    Constraint { "media.class", "matches", "*/Sink" }
  }
}

Then we can add a listener for the ā€œinstalledā€ signal.

obj_mgr:connect("installed", function (om)
  for obj in om:iterate() do
    local id = obj["bound-id"]
    local global_props = obj["global-properties"]
    print("\n")
    print("Obj ID: ".. id)
    Debug.dump_table(global_props)
    print("\n")
  end
end)

Finally, we get the ObjectManager running so that it fires the signal:

obj_mgr:activate()

This script should dump all the properties of the sinks in the PipeWire graph.

More examples can be found here.
With that letā€™s move to other tools and debugging.

Tools & Debugging šŸ”—

Native PipeWire Tools šŸ”—

PipeWire comes with a series of tools that can be used to do common tasks, interact with the server, and debug or profiling. If you are familiar with the set of tools that come with PulseAudio then youā€™ll find a similarity.

Hereā€™s a list of some interesting ones:

  • pw-cat: used to play audio
  • pw-play, pw-record, pw-midirecord, pw-midiplay: symlink to pw-cat
  • pw-record: used to record audio
  • pw-loopback: create a dummy loopback node
  • pw-link: A port and link manager, used to list ports, monitor them, and create links
  • pw-dump: used to dump nodes in the graph or the whole graph
  • pw-dot: similar to pw-dump but dumps it in a graphviz format
  • pw-top, pw-profiler: used to monitor traffic efficiency between objects in the graph
  • pw-mon: used to monitor any events happening in the graph
  • pw-metadata: used to modify metadata in the graph, this is currently used for storing default nodes information.
  • pw-cli: generic command line to interface with PipeWire daemon, allowing dump, loading and unloading modules, listing objects, creating links and nodes, setting params, and more.

These tools, and in particular pw-cli, can be used to create objects and manage the graph on the fly or inspect things happening in the graph.

Hereā€™s an example of create a link between two front-left ports of two nodes:

pw-cli create-link "TestSink" 'monitor_FL' \
"alsa_output.usb-C-Media_Electronics_Inc.device.iec958-stereo" \
'playback_FL'

Or dumping all available factories:

pw-cli dump Factory

Or creating a virtual node:

pw-cli create-node adapter { \
factory.name=support.null-audio-sink node.name=my-mic \
media.class=Audio/Duplex object.linger=1 \
audio.position=FL,FR }

Apart from the command line tools, there currently exists a single native Rust-Gtk-based GUI under the name of helvum. It is still a WIP project, offering the basic functionality of connecting node by clicking on ports of both sides.
Other than this, there is a lack of frontend for PipeWire, especially those that would allow manipulating/representing properly a graph of any type of media, both audio and video.

Hereā€™s what helvum looks like:

helvum GUI

Iā€™ve attempted myself to build something using Rust, however the rust binding, especially related to POD and SPA, is still a work-in-progress. It is hard to bind to rust as the PipeWire library mostly only has static inline functions.

Yet, there exists fun tools such as PulseEffects that allow creating filter nodes to add audio effects on the fly.

On the other hand, when it comes to the session manager, the pipewire-media-session currently doesnā€™t have clients while WirePlumber does offer a bundle of command line tools.

Weā€™ve already seen wpexec to execute lua scripts. Thereā€™s also wpctl which is another introspection tool to interrogate WirePlumber about available devices, nodes, their properties, and their volume. It can be used to interface with the notion of ā€œendpointsā€ which is the abstract representation of where media start and ends for the session manager.

wpctl status can be used to list known endpoints, showing default devices with a star character prepended. This default can be changed using the set-default option, passing the node ID. The other options such as set-volume, set-mute, set-profile are straight forward to understand.

Tools Relying on PulseAudio šŸ”—

pipewire-pulse, which as we said is a wrapper to load the module libpipewire-module-protocol-pulse, allows to use most PulseAudio features and software on top of PipeWire compatibility layer. The module itself can be configured in pipewire-pulse.conf with certain options, as shown here. It even covers the modules for network protocol support.

It thus let us use any user interface that we got accustomed to use with PulseAudio such as pavucontrol, pulsemixer, pamixer, and more. This also includes command line tools such as pactl which then gives us yet another way to interface with PipeWire instead of pw-cli.

$ pactl info

Server String: /run/user/1000/pulse/native
Library Protocol Version: 34
Server Protocol Version: 35
Is Local: yes
Client Index: 106
Tile Size: 65472
User Name: vnm
Host Name: identity
Server Name: PulseAudio (on PipeWire 0.3.30)
Server Version: 14.0.0
Default Sample Specification: float32le 2ch 48000Hz
Default Channel Map: front-left,front-right
Default Sink: alsa_output.pci-0000_00_14.2.analog-stereo
Default Source: alsa_input.pci-0000_00_14.2.analog-stereo
Cookie: daad:fd8f

While pactl is supposed to be used with PulseAudio, the functionality is somewhat mapped to PipeWire. This includes the loading of modules, which can be used to create nodes in the graph.

pactl load-module module-null-sink object.linger=1 \
media.class=Audio/Sink \
sink_name=my-sink \
channel_map=surround-51

As you can notice, the syntax of the module-null-sink isnā€™t anything like the syntax youā€™d use for PulseAudio. This same module can also be used to create duplex and source nodes. More on the creation of virtual devices here.

The mapping even works with listing modules through pactl list modules, but lists PipeWire modules instead. Obviously, we can do listing of objects, or specific ones such as sinks with pactl list sinks.

Thereā€™s this useful guide that shows features mapped between PulseAudio and PipeWire, including the pactl we showed and certain configuration specific to PulseAudio.

Hereā€™s a screenshot of pulsemixer running on top of PipeWire:

pulsemixer UI

Tools Relying on Jack šŸ”—

As with PulseAudio, PipeWire offers a compatibility layer with JACK. It is initiated by prepending JACK commands with pw-jack, for example:

pw-jack jack-plumbing
pw-jack qjackctl

NB: you can find jack-plumbing here, a very nice tool to create jack rules connection rules.

pw-jack is a shell script that sets some environment variables and modifies the LD_LIBRARY_PATH to point to PipeWireā€™s jack lib before the global one. You can notice this by doing the following:

pw-jack ldd /usr/bin/qjackctl| grep -i libjack

libjack.so.0 => /usr/lib/pipewire-0.3/jack/libjack.so.0 (0x00007f65c9fa0000)

Similarly to PulseAudio shim, this allows us to use most JACK based GUI tools such as qjackctl, carla, catia, and all the beautiful visualizers. The advantage over PulseAudio tools is that JACK tools already display a graph by default which maps well to PipeWire concepts, however it only maps audio devices and not video.
PipeWire also allows modifying the graph on the fly, the effect being instantaneous, unlike JACK, which is nifty.

The documentation also lists specific configurations related to JACK, usually found in jack.conf.

Hereā€™s a screenshot of qjackctl running on top of PipeWire:

qjackctl UI

Tools Relying on GStreamer šŸ”—

Lastly, as weā€™ve mentioned before, we can rely on any program that uses gstreamer and the gstreamer command line tools to play with the media.
Again, through the plugins:

  • pipewiresrc: PipeWire source
  • pipewiresink: PipeWire sink
  • pipewiredeviceprovider: PipeWire Device Provider

We can create a PipeWire sink that will provide audio from the internet:

gst-launch-1.0 \
uridecodebin 'uri=http://podcast.nixers.net/feed/download.php?filename=nixers-podcast-2020-07-221.mp3' ! \
pipewiresink mode=provide \
stream-properties="props,media.class=Audio/Source,node.description=podcast"

And then link the ports to an audio device, either via the command line tools or GUI such as helvum.

pw-link gst-launch-1.0:capture_1 \
alsa_output.usb-C-Media_Electronics_Inc._Microsoft_LifeChat_LX-3000-00.iec958-stereo:playback_FL
pw-link gst-launch-1.0:capture_2 \
alsa_output.usb-C-Media_Electronics_Inc._Microsoft_LifeChat_LX-3000-00.iec958-stereo:playback_FR

In theory, that should allow us to manipulate videos too, however, as Iā€™ve mentioned above I got some issues with it.
Iā€™ve tested the following and tried to open it with cheese webcam app but the video only showed for 2s and stopped:

gst-launch-1.0 \
uridecodebin uri=https://venam.nixers.net/workflow-compil-venam-2020.webm ! \
pipewiresink mode=provide \
stream-properties="props,media.class=Video/Source,node.description=compilation"

Still the possibility is fascinating: to easily be able to modify and add filters to videos (including cameras) on the go.

Conclusion šŸ”—

This review should help kick-start people on the PipeWire journey, give the key to unlock more knowledge around it ā€” to get started.

We reviewed the basic ideas behind PipeWire, the concept of the graph, ā€œmechanism not policyā€, we also saw the relation with Gstreamer.
We had a look at the building blocks such as the POD format and the SPA to build factories of objects following an interface. Weā€™ve also glimpsed at the PipeWire library loop and itā€™s inspiration from Wayland with its singleton objects such as the registry.
Afterwards, we explained the configuration format, what can be configured, for the client, PipeWire server, and the session manager, be it the default pipewire-media-session or WirePlumber with its flexible lua scripting.
Then weā€™ve had a quick overview of the tooling around PipeWire, from native, to the ones relying on compatibility layers with PulseAudio and JACK.

Yet, this is only the tip of the iceberg, thereā€™s a lot of things that werenā€™t covered such as how to compile PipeWire from scratch, how to create audio streams/filters and manage their buffers, diving into the metadata and how its used by different pieces, how the session manager can use the concept of Endpoints, the policy mechanism and the link with desktop portals, etc..

This post is descriptive but the real story happens as we speak, the development is quick and the discussion is lively on IRC (currently on irc.oftc.net in the #pipewire channel). The developers are nice fellows that are very active in the discussion.

This post will probably age badly because the tech is relatively new, but thereā€™s a need for such blog post to get acquainted with the project through new eyes.
Iā€™ve only touched a small part of PipeWire but that should get anyone started, at least I wouldā€™ve personally loved to have such content when I first heard of PipeWire.

Let me know if it helped!


References šŸ”—

https://pipewire.org/
https://gstreamer.freedesktop.org/
https://gstreamer.freedesktop.org/data/events/gstreamer-conference/2016/Wim%20Taymans%20-%20Simple%20Plugin%20API%20(SPA).pdf
https://wiki.gentoo.org/wiki/PipeWire
https://lwn.net/Articles/847412/
https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Config-ALSA
https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/SPA
https://www.youtube.com/watch?v=1w6yVqU0lkU
https://www.youtube.com/watch?v=yjh5Eg3Efjg
https://gitlab.freedesktop.org/ryuukyu/helvum
https://wiki.archlinux.org/title/PipeWire
https://docs.pipewire.org/page_pipewire.html
https://en.wikipedia.org/wiki/Passive_data_structure
https://gitlab.freedesktop.org/pipewire/pipewire/-/blob/master/doc/spa-pod.dox
https://pipewire.github.io/pipewire/page_remote_api.html
https://www.collabora.com/news-and-blog/blog/2020/03/05/pipewire-the-media-service-transforming-the-linux-multimedia-landscape/
https://wiki.debian.org/PipeWire
https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/home
https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Access-control
https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/FAQ
https://docs.pipewire.org/page_spa.html
https://docs.pipewire.org/modules.html
https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Config-PipeWire
https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Filter-Chain
https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Virtual-Devices
https://www.guyrutenberg.com/2021/03/11/replacing-pulseaudio-with-pipewire/
https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Config-pipewire-media-session
http://old-docs.automotivelinux.org/docs/en/halibut/apis_services/reference/audio/audio/pipewire.html
https://pipewire.pages.freedesktop.org/wireplumber/daemon-configuration.html
https://docs.pipewire.org/page_registry.html
https://github.com/PipeWire/pipewire/blob/master/doc/tutorial2.md
https://github.com/PipeWire/pipewire/blob/master/src/tools/pw-dump.c
https://pipewire.pages.freedesktop.org/pipewire-rs/pipewire/index.html
https://gitlab.freedesktop.org/pipewire/pipewire-rs/-/blob/main/pipewire/examples/
https://pipewire.pages.freedesktop.org/pipewire-rs/
https://www.antixforum.com/forums/topic/pipewire-to-manage-audio-in-antix-21/
https://blog.linuxplumbersconf.org/2009/slides/Paul-Davis-lpc2009.pdf


If you want to have a more in depth discussion I'm always available by email or irc. We can discuss and argue about what you like and dislike, about new ideas to consider, opinions, etc..
If you don't feel like "having a discussion" or are intimidated by emails then you can simply say something small in the comment sections below and/or share it with your friends.

One Comment

Type Comment Here (at least 3 chars)
Michael Lerroā€¢18 hours ago

Mate this is fantastic. Well done


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK