Recently at work, I needed to trace several syscalls to understand what SQL Server was doing. My usual tool for this purpose on Windows was API Monitor, but, unfortunately, it hasn’t been updated for a few years already and became unstable for me. Thus, I decided to switch back to WinDbg. In the past, my biggest problem with tracing the system API in WinDbg was the missing symbols for the internal NT objects. Moreover, I discovered some messy ways to work around it. Fortunately, with synthetic types in WinDbg Preview it’s no longer a problem. In this post, I will show you how to create a breakpoint that nicely prints the arguments to a sample NtOpenFile syscall.

Creating a custom header file

There are various places where you can look for the syscall signature, but I usually check Microsoft docs and Process Hacker headers. For our example, Process Hacker signature looks as follows:

We also need to find definitions of POBJECT_ATTRIBUTES and PIO_STATUS_BLOCK. They are also in Process Hacker headers:

Let’s save these definitions to a file, for example, c:\temp\nt.h.

NOTE: You may be wondering why not use windows.h or phnt.h files and have all the types parsed. Unfortunately, the Synthetic Types extension (we will use it in the next paragraph) is quite limited and can’t parse such complex files : (

Load the custom header file into WinDbg

To load and parse the header file into WinDbg, we will use Andy Luhrs’ Synthetic Types extension. Checkout the repo and load the script as stated in the README:

Next, let’s read our header file:

Place the breakpoint

It’s time to set a breakpoint on the NtOpenFile and decode its parameters:

As I’m debugging a 64-bit version of SQL Server, AMD64 calling convention applies:

As we are interested in decoding ObjectAttributes, let’s use our custom header type:

Eek! We hit an error. After some experimenting, I learned that this error signifies that the extension was not able to calculate the size of the object instance. The most probable reason is that WinDbg did not correctly decode some of the types from our custom header.

When things do not work

When you hit an error like the one above, you need to perform a mundane task of finding the faulting type. To do that, read the header file once again and click on the DML Types link. That’s also a way to know what is the name of your type in WinDbg, in case you have many typedefs. You should see something similar to the output below:

Next, click [0x0] and then Description:

All simple types are good, so click on the one complex field available, which is [0x2]:

Here we go. PUNICODE_STRING is an unknown type to WinDbg, and we need to define it. There are few more structures we need to add, so below is the final working header for NtOpenFile:

NOTE: Once you change the definition, you need to restart the debugging session. Reloading the changed header file did not take any effect in my version of WinDbg (1.0.1906.12001 / 10.0.18914.1001).

Producing nice output

With the final header file read, it is time to create the final breakpoint:

You may improve the output with .printf or other meta-functions, but it looks nice already, doesn’t it?