Monday, May 31, 2010

Remote Debugging with WinDbg and NTSD

What is Remote Debugging?

Remote Debugging is the activity during which we debug a process \ service \ etc… executing on different system. When remote debugging, we start debugger on local system and connect to remote system via network.  Debugging experience is basically the same as ordinary debugging. The only difference is that in this case we have debugger and debuggee (debugged process) running on two different systems.

Remote Debugging with WinDbg and NTSD

In the following scenario WinDbg represents a client debugger. It is located on the local machine and plays the role of debugger UI: it receives debugging commands from user and  shows him debugging information (such as process state). NTSD plays the role of server debugger. It executes on remote system and controls the process according to commands received from client debugger. Also, it sends information back to client debugger which then presents these information to user through its UI.

Let’s take a look at  how we start remote debugging session and what actions need to be done on remote and local systems.

Remote System

We have to make sure that debuggee starts under NTSD. One solution is to start it manually with the following command:

ntsd -server tcp:port=9001 -g -G notepad.exe

Obviously, we chose to debug notepad.exe. Option server creates debugging session that other people can connect to via TCP port 9001. Additionally, we specify two other options: g (ignores initial breakpoint in debuggee) and G (ignores final breakpoint at debuggee termination).

However, in most cases, process we want to debug is not started directly by us. For example it can be started by other user or on some external event. In these cases we have to setup the remote system so that debugged process always starts under NTSD. This can be done by going to registry key:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options

and adding a sub-key with the name of debugged executable (in this case notepad.exe). After that, we need to go to previously created registry key and add the following string value:

(Name=Debugger, Data=C:\WinDDK\7600.16385.1\Debuggers\ntsd.exe -server tcp:port=9001:9100 -g -G)

Note that above value contains the address of NTSD on my machine. You need to modify it if you have NTSD placed at some other location. Additionally, in this case we specified TCP port range 9001:9100. This means that every time new notepad.exe instance is started, it will run under new NTSD instance with debugging session available at first free port from this range. Therefore, for the first notepad.exe instance NTSD will choose port 9001, for the second notepad.exe instance NTSD will choose port 9002,…

Local System

Now that we set up remote system, the only thing left is to start WinDbg on local system and connect to server debugger. Let’s say that we have two notepad.exe instances running on remote system. We know that their debugging sessions are available via TCP ports 9001 and 9002. If we want to start debugging session from command line we will use the following command:

windbg -remote tcp:server=myremotehost,port=9001 (for the first instance)

windbg -remote tcp:server=myremotehost,port=9002 (for the second instance)

Please replace myremotehost with the name of your remote system (machine). If you want to start debugging session from the WinDbg UI, just start the WinDbg and then go to File > Connect to Remote Session or press Ctrl + R. In the dialog that appears, enter the following connection string:

tcp:server=myremotehost,port=9001 (for the first instance)

tcp:server=myremotehost,port=9002 (for the second instance)

After that, you can use WinDbg the same way you would use it when debugging a process on your local machine. The difference is, as we previously noted, that in this case NTSD plays the role of debugging engine. WinDbg plays the role of debugging UI, and therefore we use it to enter debugging commands and get debugging information.

Additional Notes

We can use NTSD, CDB or KD as client debugger in the same way (by using same command line arguments). In this particular case we used WinDbg simply because it has UI, which makes it the most comfortable debugger from this group.

Remote debugging can be useful even in some cases when debugger and debuggee are located on the same machine. Let’s consider the following situation. Debugged process starts without our control and under non-UI account, for example (NETWORK SERVICE). If we specified WinDbg in Image File Execution Options, then WinDbg would also start under NETWORK SERVICE and we would not be able to access its UI. Remote Debugging solves this problem: in Image File Execution Options we specify NTSD (which doesn’t need UI, so it can run as NETWORK SERVICE) and we connect to it with WinDbg running as current user.

No comments: