Descriptors

A good way to think about a buffer or an image descriptor is to imagine it as a very fat pointer. This is, in fact, not too far removed from reality, as we shall see.

Taking a peek at radv, we find the descriptor behind UNIFORM_BUFFER and STORAGE_BUFFER to be a 4-word tuple, where the first two words make up the address, followed by length in bytes for bounds checking and an extra word, which holds format information and bounds checking behavior 1.

Dynamic buffer descriptors are similar, with a small difference in that they conceptually live in the command buffer (in case of radv, they compete for space with push constants 2) as opposed to descriptor pool.

Similarly, the descriptor behind SAMPLED_IMAGE is a 16-word tuple containing an address, a format, extent, number of samples, mip levels, layers, and other bits found in the VkImageView 3.

A sampler descriptor is an odd one in that most sampler descriptors are pure fat, but some keep an index into a stash of samplers' extra bits. That is, in radv, a sampler descriptor is a 4-word tuple, which holds all of the sampler bits, unless custom border color is used, in which case the last word also maintains an index into an array of custom border colors 4. Anv and turnip are similar 56.

Combining these bits of knowledge, it is easy to guess that a combined image-sampler descriptor is, in fact, a sampled image and a sampler descriptors glued together.

Descriptor Sets

Descriptors are grouped into descriptor sets, not unlike variables are composed into structures in C, with descriptor set layouts being akin to type definitions and vkUpdateDescriptorSets akin to member writes and copies. Let's conceive an arbitrary descriptor set

// A list of VkDescriptorSetLayoutBindings making up an "everything"
// descriptor set. For simplicity, all stages can access all bindings.

	{0, VK_DESCRIPTOR_TYPE_INLINE_UNIFORM_BLOCK_EXT, 128, VK_SHADER_STAGE_ALL, NULL}, // camera
	{1, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_ALL, NULL},             // transforms
	{2, VK_DESCRIPTOR_TYPE_SAMPLER, 2, VK_SHADER_STAGE_ALL, NULL},                    // samplers
	{3, VK_DESCRIPTOR_TYPE_SAMPLED_IMAGE, 10000, VK_SHADER_STAGE_ALL, NULL},          // manyimages

In our C analogy, such a descriptor set would be written as follows

struct DescriptorSetOfEverything {
	char camera[128];
	StorageBuffer transforms;
	Sampler samplers[2];
	SampledImage manyimages[10000];
};

with a slight caveat that the offsets are unspecified and are hidden inside VkDescriptorSetLayout 7. Nevertheless, let's put on shoes of radv and calculate the descriptor offsets and the size of the descriptor set. First, we shall familiarize ourselves with size and alignment of each descriptor

Then let there be a sequence nᵢ, n₀ = 0, nᵢ₊₁ = roundup(nᵢ, aᵢ) + kᵢmᵢ, where aᵢ, mᵢ are, respectively, the alignment and size of i-th binding's descriptor, kᵢ is i-th binding's descriptor count and roundup(x, y) = min {yn | yn ≧ x, n ∈ ℤ}. For each binding i, roundup(nᵢ, aᵢ) is the offset of the binding's first descriptor and given the number of bindings p, nₚ is the descriptor set's size. Writing out this sequence, we get 0, 128, 144, 176, 640192. The offsets of each binding's first descriptor are thus 0, 128, 144, 192 and the size of the descriptor set is 640192 bytes.

Memory

There's no malloc for descriptor sets and, in fact, no good analogy that a reader would be familiar with appears to exist. Descriptor pools can be confusing. Read the following closely, lest you will find your program only works on your computer.

A good starting point for reasoning about descriptor pools is to pretend that a VkDescriptorPool is a VkDeviceMemory for descriptor sets. The list of descriptor pool sizes taken by vkCreateDescriptorPool specifies the size of the underlying VkDeviceMemory as a sum of each descriptor size times descriptor count 8. For a concrete example, let's consider the following list of pool sizes

	{VK_DESCRIPTOR_TYPE_SAMPLER, 1},
	{VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER, 100},

In radv, sampler and combined image-sampler descriptors take up 32 and 96 bytes respectively, thus the VkDeviceMemory inside such descriptor pool will be 32⋅1 + 96⋅100 = 9632 bytes. This is plenty to allocate a descriptor set of 200 UNIFORM_BUFFER descriptors and such a vkAllocateDescriptorSets call will indeed succeed on radv, where a buffer descriptor takes 32 bytes. This behavior is to be exploited, but not to be relied upon.

A simple method to deal with the allocation is to use a very large capacity descriptor pool and allocate descriptor sets until VK_ERROR_OUT_OF_POOL_MEMORY is returned. In the out of pool memory case, the pool becomes a zombie and when the descriptor sets it backs are not needed any more, the pool can be freed.

VkDescriptorPoolSize poolSizes[] = {
	{VK_DESCRIPTOR_TYPE_SAMPLER, 1000},
	{VK_DESCRIPTOR_TYPE_SAMPLED_IMAGE, 1000},
	{VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, 1000},
	{VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER, 1000},
	{VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1000},
};
// ...

	if ((r = vkCreateDescriptorPool(device, &(VkDescriptorPoolCreateInfo) {
		.sType         = VK_STRUCTURE_TYPE_DESCRIPTOR_POOL_CREATE_INFO,
		.maxSets       = 10000,
		.poolSizeCount = nelem(poolSizes),
		.pPoolSizes    = poolSizes,
	}, NULL, &descriptorPool)) != VK_SUCCESS) {
		// Handle error.
	}

While this method is simple, it can waste significant amounts of memory for some applications, unless some tuning is done.

This inefficiency may be remedied by creating a descriptor pool per descriptor set layout, which will accomodate some number of descriptor sets of this layout.

If a lot of descriptor sets have the same lifetime such as in cases, for example, when the application allocates all descriptor sets during initialization, it's possible to compute optimal pool size and use a single pool, side-stepping descriptor pool cycling headaches entirely.

Dirty Details

It's important to note that on some implementations, some descriptors differ significantly from the model presented. Updates of descriptors of such types may modify some structures not encapsulated in VkDescriptorSet.

A possible example of such are SAMPLED_IMAGE and SAMPLER descriptor types on NVIDIA hardware. Starting with NV50 (GeForce 8 series), all textures and samplers that are expected to be accessed are stored in two big arrays, referred to as texture image control (TIC) and texture sampler control (TSC) blocks, respectively 910. Switching TxCBs 1112 has to happen when shaders aren't running, so it is desirable to have as few TxCBs as possible to minimize switches and associated waiting. Descriptors SAMPLED_IMAGE and SAMPLER would then be indices into respective TxCB.

Future Directions

As of writing, descriptors are treated as special entities, separate from data. This is unfortunate for structures that wish to refer to the objects descriptors point to. A correspondence between some data (for example, an integer) and a descriptor has to be introduced, which has an impression of being unnecessary.

For some types of descriptors, this indirection can be avoided. For example, buffer device address replaces storage buffer descriptors with familiar pointers. The experimental VK_NVX_image_view_handle extension in turn addresses images and combined image-samplers. Although the interface assumes specific hardware, the assumption is easily relaxed by modifying vkGetImageViewHandleNVX to return multiple uint32_t values required to describe an object, allowing for straightforward implementations on other hardware. It is obvious how this could be extended to other types of descriptors that don't require special treatment. Let's hope this materializes at one point.

Handling Window Resize

A rather common mistake in handling resize requests is initiating resize in response to vkAcquireNextImageKHR or vkQueuePresentKHR returning VK_ERROR_OUT_OF_DATE_KHR and querying swapchain extent with either vkGetPhysicalDeviceSurfaceCapabilitiesKHR or through windowing system-specific means.

On Windows, resizing windows of programs that interleave DispatchMessage with redrawing will cause DispatchMessage to block. The window will appear to have its contents "frozen" for duration of the resize and "unfreeze" when the user is done. Programs that have message loop and redrawing concurrent will race vkCreateSwapchainKHR against resize and break VUID-VkSwapchainCreateInfoKHR-imageExtent-01274.

On Wayland, VK_ERROR_OUT_OF_DATE_KHR is never returned and there's no window size to query: VkSurfaceCapabilitiesKHR returned by vkGetPhysicalDeviceSurfaceCapabilitiesKHR will have currentExtent set to 0xFFFFFFFF×0xFFFFFFFF, minImageExtent to 1×1 and maxImageExtent to some large value. Instead, the window size is specified by the program using imageExtent when creating the swapchain. When the user resizes the window, a resize message will be delivered, specifying the desired size, but it is up to the program to resize the swapchain (and thus the window).

Another common mistake is assuming that the width and height are always positive. This assumption is easily broken by shrinking the window to zero.

The correct approach to handling resizes is thus to listen to the windowing system's resize message and be defensive about the window size. After a swapchain is created, at least a single frame should be redrawn so that the program is not stuck handling resize events, without ever having something to present to the user.

The example program is structured into redraw and resize functions. To avoid the mistake covered in the first paragraph of this section, redraw will simply return instead of initiating resize (creating new swapchain).

VkDevice device;

VkSurfaceKHR surface;

VkSwapchainKHR swapchain;
bool swapchain_ok;

void
redraw(void)
{
	VkResult r;

	// Calling vkAcquireNextImageKHR or vkQueuePresentKHR on swapchain
	// for which a prior call returned VK_ERROR_OUT_OF_DATE_KHR is an
	// error, so it is our responsibility to make it sticky.
	if (swapchain_ok) {
		r = vkAcquireNextImageKHR(/* ... */);
		if (r == VK_ERROR_OUT_OF_DATE_KHR) {
			swapchain_ok = false;
		} else if (r != VK_SUCCESS && r != VK_SUBOPTIMAL_KHR) {
			// Handle the error.
		}
	}
	if (!swapchain_ok)
		return;

	// Record commands and submit.

	r = vkQueuePresentKHR(/* ... */);
	if (r == VK_ERROR_OUT_OF_DATE_KHR) {
		swapchain_ok = false;
	} else if (r != VK_SUCCESS && r != VK_SUBOPTIMAL_KHR) {
		// Handle the error.
	}
}

void
resize(uint32_t width, uint32_t height)
{
	VkResult r;

	assert(width > 0 && height > 0);

	vkDeviceWaitIdle(device);

	VkSwapchainKHR oldSwapchain = swapchain;

	if ((r = vkCreateSwapchainKHR(device, &(VkSwapchainCreateInfoKHR) {
		.sType = VK_STRUCTURE_TYPE_SWAPCHAIN_CREATE_INFO_KHR,
		.surface = surface,
		.imageExtent = (VkExtent2D) {width, height},
		/* ... */
	})) != VK_SUCCESS) {
		// Handle the error.
	}

	// If it is possible for an image acquired from oldSwapchain to still
	// not be presented at this point, it should be made a zombie instead.
	vkDestroySwapchainKHR(device, oldSwapchain, NULL);

	// Create swapchain-dependent resources.

	swapchain_ok = true;

	redraw();
}

Now, to wire up with the windowing system

static void resize_callback(GLFWwindow *w, int width, int height)
{
	if (width > 0 && height > 0)
		resize(width, height);
}

int main(int argc, char **argv)
{
	// ...

	glfwSetWindowSizeCallback(w, resize_callback);

	while (!glfwWindowShouldClose(window))
	{
		if (swapchain_ok)
		{
			glfwPollEvents();
		}
		else
		{
			glfwWaitEvents(); // don't burn cpu
			continue;
		}

		redraw();
	}

	// ...
}

int main(int argc, char **argv) {
	// ...

	for (;;) {
		SDL_Event e;
		while ((swapchain_ok ? SDL_PollEvent(&e) : SDL_WaitEvent(&e)) != 0) {
			switch (e.type) {
			case SDL_WINDOWEVENT:
				if (e.window.event == SDL_WINDOWEVENT_SIZE_CHANGED && e.window.data1 > 0 && e.window.data2 > 0)
					resize(e.window.data1, e.window.data2);
				break;

			// Handle the remaining cases.
			}
		}

		redraw();
	}

	// ...
}

If Windows API is used, it is important to be careful when using CreateWindow. Misusing CreateWindow will lead to the first WM_SIZE being lost.

static LRESULT CALLBACK
WndProc(HWND hwnd, UINT msg, WPARAM wparam, LPARAM lparam)
{
	switch (msg) {
	case WM_CREATE:
		// Create VkSurfaceKHR here
		break;

	case WM_SIZE: {
		uint32_t width = lparam & 0xffff;
		uint32_t height = (lparam >> 16) & 0xffff;

		if (width > 0 && height > 0)
			resize(width, height);
		break;
	}

	// Handle the remaining cases.
	}
}

On X11, XCB_CONFIGURE_NOTIFY is not sent in response to window being created. The program should create the swapchain with size of the window it created. Note that on X11, vkCreateSwapchainKHR always races against resize. This will inevitably break VUID-VkSwapchainCreateInfoKHR-imageExtent-01274, which is expected and should be muted in the validation layer.

int
main(int argc, char **argv)
{
	// ...

	resize(/* width and height the window was created with */);

	for (;;) {
		void *e;
		while ((e = xcb_poll_for_event(X)) != NULL) {
			xcb_generic_event_t *generic = e;

			switch (generice->response_type&~0x80) {
			case XCB_CONFIGURE_NOTIFY: {
				xcb_configure_notify_event_t *configure_notify = e;

				// Note that this message is also sent when
				// the window is being moved and not resized.
				// It might be desirable to ignore configure
				// notifications that do not change width and
				// height.

				if (configure_notify->width > 0 && configure_notify->height > 0)
					resize(configure_notify->width, configure_notify->height);
				break;
			}

			// Handle the remaining cases.
			}
			free(e);
		}

		redraw();
	}

On Wayland, the application never receives VK_ERROR_OUT_OF_DATE_KHR. It is also much easier to handle the case when the window is shrunk to zero.

bool swapchain_ok = true; // can be made a constant on Wayland :-)

bool closed;

static void xdg_surface_handle_configure(void *data,
	struct xdg_surface *xdg_surface, uint32_t serial)
{
	xdg_surface_ack_configure(xdg_surface, serial);
}

static const struct xdg_surface_listener xdg_surface_listener = {
	.configure = xdg_surface_handle_configure,
};

static void
xdg_toplevel_configure(void *data,
	struct xdg_toplevel *xdg_toplevel, int32_t width, int32_t height,
	struct wl_array *states)
{
	if (width <= 1)
		width = 1;
	if (height <= 1)
		height = 1;
	resize(width, height);
}

static void
xdg_toplevel_close(void *data, struct xdg_toplevel *toplevel)
{
	closed = true;
}

static const struct xdg_toplevel_listener xdg_toplevel_listener = {
	.configure = xdg_toplevel_configure,
	.close = xdg_toplevel_close,
};

int
main(int argc, char **argv)
{
	// ...

	resize(/* any width and height desired */);

	while (!closed) {
		wl_display_dispatch_pending(display);

		redraw();
	}

	// ...
}

Inevitably, there will be bugs. Worse, drivers sometimes have hard to reproduce bugs of their own, lurking in the swapchain. Should an issue arise, a vkDeviceWaitIdle at the beginning of resize is often a sufficient workaround.

Concurrency

Some programs may wish to process input at a rate independent from that of redrawing. This requires that the windowing system message handling and redrawing are made independent of each other, and are allowed to execute concurrently.

typedef /* intentionally left blank */ Mutex;

void threadcreate(void (*f)(void*), void *a); // spawns a thread executing f(a)
void lock(Mutex *m);                          // locks m
void unlock(Mutex *m);                        // unlocks m

Mutex mu; // protects state accessed by resize and redraw

 void
 resize(uint32_t width, uint32_t height)
 {
 	VkResult r;

 	assert(width > 0 && height > 0);
+
+	lock(&mu);

 	vkDeviceWaitIdle(device);

 	VkSwapchainKHR oldSwapchain = swapchain;

 	if ((r = vkCreateSwapchainKHR(device, &(VkSwapchainCreateInfoKHR) {
 		.sType = VK_STRUCTURE_TYPE_SWAPCHAIN_CREATE_INFO_KHR,
 		.surface = surface,
 		.imageExtent = (VkExtent2D) {width, height},
 		/* ... */
 	})) != VK_SUCCESS) {
 		// Handle the error.
 	}

 	// If some images acquired from oldSwapchain have not yet been
 	// presented, it should be made a zombie instead.
 	vkDestroySwapchainKHR(device, oldSwapchain, NULL);

 	// Create swapchain-dependent resources.

 	swapchain_ok = true;

 	redraw();
+
+	unlock(&mu);
 }

void
redrawLoop(void *a)
{
	while (/* redraw stopping condition */) {
		lock(&mu);
		redraw();
		unlock(&mu);
	}
}

int
main(int argc, char **argv)
{
	// Setup.

	threadcreate(redrawLoop, NULL);

	while (/* message loop stopping condition */) {
		// Exchange messages with the windowing system.
	}

        // Done. Some programs may want to join thread execing redrawLoop at
	// this point.
}

Careful readers will note the busy-waiting that occurs when the window is shrunk to zero. It is desirable that the waiting is done by communicating what the program is waiting for to the environment, so that CPU is not being burned in vain. redraw needs to be modified so as to communicate to the caller when to begin waiting. The caller will then wait on a condition variable, which will be signaled after a swapchain is created.

typedef /* intentionally left blank */ Cond;
void wait(Cond *c, Mutex *m); // unlocks m, waits on c, locks m
void signal(Cond *c);         // wakes up waiters on c

Cond cond;

-void
+bool
 redraw(void)
 {
 	VkResult r;

 	// Calling vkAcquireNextImageKHR or vkQueuePresentKHR on swapchain
 	// for which a prior call returned VK_ERROR_OUT_OF_DATE_KHR is an
 	// error, so it is our responsibility to make it sticky.
 	if (swapchain_ok) {
 		r = vkAcquireNextImageKHR(/* ... */);
 		if (r == VK_ERROR_OUT_OF_DATE_KHR) {
 			swapchain_ok = false;
 		} else if (r != VK_SUCCESS && r != VK_SUBOPTIMAL_KHR) {
 			// Handle the error.
 		}
 	}
 	if (!swapchain_ok)
-		return;
+		return false;

 	// Record commands and submit.

 	r = vkQueuePresentKHR(/* ... */);
 	if (r == VK_ERROR_OUT_OF_DATE_KHR) {
 		swapchain_ok = false;
 	} else if (r != VK_SUCCESS && r != VK_SUBOPTIMAL_KHR) {
 		// Handle the error.
 	}
+
+	return true;
 }

 void
 resize(uint32_t width, uint32_t height)
 {
 	VkResult r;

 	assert(width > 0 && height > 0);

	lock(&mu);

 	vkDeviceWaitIdle(device);

 	VkSwapchainKHR oldSwapchain = swapchain;

 	if ((r = vkCreateSwapchainKHR(device, &(VkSwapchainCreateInfoKHR) {
 		.sType = VK_STRUCTURE_TYPE_SWAPCHAIN_CREATE_INFO_KHR,
 		.surface = surface,
 		.imageExtent = (VkExtent2D) {width, height},
 		/* ... */
 	})) != VK_SUCCESS) {
 		// Handle the error.
 	}

 	// If some images acquired from oldSwapchain have not yet been
 	// presented, it should be made a zombie instead.
 	vkDestroySwapchainKHR(device, oldSwapchain, NULL);

 	// Create swapchain-dependent resources.

 	swapchain_ok = true;

+	// Don't care if redraw fails, if it does, the next resize event will
+	// be handled shortly.
 	redraw();

	unlock(&mu);
+
+	// Wake up waiters on cond. There is only ever at most a single waiter.
+	// Doesn't matter if signal happens before dropping mu or after.
+	signal(&cond);
 }

 void
 redrawLoop(void *a)
 {
 	while (/* redraw stopping condition */) {
 		lock(&mu);
-		redraw();
+		if (!redraw()) {
+			// Unlock mu and begin waiting on cond. Spurious wake
+			// ups are okay, because redraw will just fail and end
+			// up waiting again (or resize happens in the
+			// meanwhile).
+			wait(&cond, &mu);
+		}
 		unlock(&mu);
 	}
 }

Always remember to sanitize your threads once in a while!

Redraw on Request