.NET debugging in a single picture

Have you ever wonder what happens when you create and use breakpoints in .NET? Here’s a little picture that answers that question (if you don’t like the font, you have a different version at the bottom).

We have the main actors here as follows:

• .NET Application – our regular .NET application that we want to debug. Methods, as provided by the compiler in the Intermediate Language form (IL) are Just-in-Time compiled to a native code when called. So, imagine our “e8 50 ff ff” represents an example binary code of a line we want to debug (no matter what it does now)
• Debugger Runtime Control Thread (hereinafter referred to as Debugger RC Thread) – it is a special thread inside every .NET process for debugging purposes and serves as a bridge between the CLR and an external debugger. It consists of a so-called “debugger loop”, listening on events coming from the Debug Port (supported by the OS). Please note that in case of native debugging, such a special thread is typically injected into the debuggee process. But we don’t need to do that here, as .NET runtime provides it. And moreover, this thread understand the CLR data structures, so it is able to cooperate with JIT and so.
• external Debugger – it is our external process that we cooperate with. Imagine it as a tooling part of Visual Studio or other IDE you use. It is using a set of COM objects that are able to communicate via Inter-process communication (IPC) mechanism with Debugger Runtime Control Thread.

Having described the actors, let’s move on to the control flow description. Please be warned that while it described pretty low-level stuff, some even more detailed implementation details may be a little oversimplified. But still, it is just enough knowledge to understand the whole process. So, let’s look what is happenning when you use “breakpoint” debugging .NET application:

1. You set a breakpoint on a specific C# line code, which with the help of compiler/symbol magic is turned into a specific IL offset inside a given method. And knowing IL offset, the Debugger is using ICorDebugCode::CreateBreakpoint function which via IPC asks Debugger RC Thread in a target process to set such a breakpoint.
2. Debugger RC Thread cooperates with the JIT to get a native (JITted) address of a given IL offset (or sets a “trap” waiting for it being JITted in future)
3. Debugger RC Thread overwrites first byte of the target address by CC opcode (and keeps the original one on the side). It is an opcode for INT3 assembly instruction, a one-byte shorthand for INT X instruction. This instruction, when executed, generates software interrupt number 3.
4. We resume the program execution and some time passes. One of the threads will start executing our method and will execute the int 3 instruction (and the thread will remain suspended).
5. This will, as previously said, emits software interrupt number 3, which is specially handled by the operating system.
6. In the end, via Debug Port, our Debugger RC Thread will receive a notification with value EXCEPTION_BREAKPOINT. So, it is now notified that the given exception has been hit.
7. Debugger RC Thread restores the original byte of our application, to make it possible to be executed normally.
8. Via IPC external Debugger is notified about hitting the breakpoint. This is a point when IDE will show you all the fancy stuff you usually see. So you can look around, inspect variables etc.
9. Assuming we resume program execution in the IDE, the Debugger via IPC issues a command to the Debugger RC Thread to do so.
10. We want to execute our code, but know we are in the situation where the original code is untouched, without the CC opcode, so we would not hit the same breakpoint again if we just let the code to continue. That’s why a trick is used, with the same mechanism that is used while using “step by step” debugging – so-called Trap flag (TF) is set in the special CPU regisster called FLAGS (EFLAGS/RFLAGS in 32/64-bit case). This flag is responsible for setting CPU in “single-step mode“. It means, from now on the CPU will execute only single instruction and stop (additioanlly emitting software interrupt 1 each time it happens).
11. Now, we can resume executing the code, but…
12. …because of TF is set, it will just execute our single, “breakpoint” instruction and stop.
13. This will emit software interrupt 1.
14. Which, in the end via Debug Port, notifies Debugger RC Thread with value EXCEPTION_SINGLE_STEP.
15. Now, we can restore the breakpoint by overwriting our target instruction with CC again.
16. And we need to unset TF flag to let run the CPU normally.

And… that’s all! Only ~16 step of this simplified workflow is responsible for handling such simple breakpoint scenario. On top of that, hit counts and conditional breakpoints are being built – so the condition is checked around step 8 to check whether indeed we should notify the IDE.

PS. Because there were some oppinions that the original font is unreadable (although I like it as it gives nice “sketchy” style), here’s another one with different font: