Creating Adplus configuration file

As command line parameters do not allow us to use many Adplus advanced features I generally prefer to store Adplus run settings in config files with names explaining their usage. In this post we will work on some sections of the configuration file but I strongly encourage you to read the Adplus documentation and find detailed information about the configuration file structure. We will start from the following log-exceptions.adplus.config file (all samples can be downloaded from the blog samples website):

<adplus version="2">
  <settings>
    <runmode>crash</runmode>
    <lastscriptcommand>qd</lastscriptcommand>
  </settings>
  <exceptions>
    <allexceptions>
      <actions1>LOG</actions1>
      <returnaction1>GN</returnaction1>
      <actions2>LOG</actions2>
      <returnaction2>GN</returnaction2>
    </allexceptions>
    <allevents>
      <actions1>LOG</actions1>
      <returnaction1>GN</returnaction1>
      <actions2>LOG</actions2>
      <returnaction2>GN</returnaction2>
    </allevents>
  </exceptions>
</adplus>

If we use the above configuration file with the Thrower.exe application (can be downloaded from here):

> adplus -c log-exceptions.adplus.config -o c:\dumps -sc Thrower.exe

Adplus will geneate three files in the output directory:

• ADPlust_report.txt – contains the settings with which the Adplus was run
• DebuggerScript.txt – the script generated by the Adplus and executed in the debugger at the beginning of the debugging process
• Adplus_log – all the ouput produced by the debugger

If we look at the Adplus_log we can see that not much information was provided about the exceptions that occurred:

...
(12d8.1a14): CLR exception - code e0434352 (first chance)
FirstChance_clr_NET_CLR
ModLoad: 000007fe`e5e30000 000007fe`e5f1a000   C:\Windows\Microsoft.NET\Framework64\v4.0.30319\diasymreader.dll
FirstChance_ld_DLL_Load
(12d8.1a14): CLR exception - code e0434352 (first chance)
FirstChance_clr_NET_CLR
(12d8.1a14): CLR exception - code e0434352 (first chance)
FirstChance_clr_NET_CLR
ModLoad: 000007fe`fca50000 000007fe`fca5c000   C:\Windows\system32\VERSION.dll
FirstChance_ld_DLL_Load
(12d8.1a14): CLR exception - code e0434352 (!!! second chance !!!)
SecondChance_clr_NET_CLR
...

<adplus version="2">
  <keywords>
    <!-- 
    If you are attaching to the process just use: .loadby sos clr.
    
    If it's .net 2.0 you need to use: sxe -c ".loadby sos mscorwks" ld:mscorjit
    or only .loadby sos mscorwks if you are attaching to the process
     -->
    <keyword name="LoadSOS">sxe -c ".loadby sos clr;gn" ld:clrjit</keyword>
  </keywords>
  <settings>
    <runmode>crash</runmode>
    <lastscriptcommand>qd</lastscriptcommand>
  </settings>
  <postcommands>
    <debugactions>LoadSOS</debugactions>
  </postcommands>
  <exceptions>
    <allexceptions>
      <actions1>LOG</actions1>
      <returnaction1>GN</returnaction1>
      <actions2>LOG</actions2>
      <returnaction2>GN</returnaction2>
    </allexceptions>
    <allevents>
      <actions1>LOG</actions1>
      <returnaction1>GN</returnaction1>
      <actions2>LOG</actions2>
      <returnaction2>GN</returnaction2>
    </allevents>
  </exceptions>
</adplus>

Logging the managed exception type and stack trace

At this point we should have a SOS extension loaded into the debugger and thus be able to use commands that it provides. Let’s define two new keywords (help on the SOS commands can be found on MSDN):

...
<keyword name="PrintManagedException">!PrintException</keyword>
<keyword name="ManagedStack">!CLRStack -a</keyword>
...

Next step is to define new exception type handling in the exceptions section of the configuration file. As I’ve already pointed the cdb debugger is not aware of .NET exception types. Fortunately all framework exceptions have the same code and so it’s easy to write a general handler for them. Our final log-managed-exceptions.adplus.config will be as follows (notice that we are concerned only about 1st chance exception):

<adplus version="2">
  <keywords>
    <!-- 
    If you are attaching to the process just use: .loadby sos clr.
    
    If it's .net 2.0 you need to use: sxe -c ".loadby sos mscorwks" ld:mscorjit
    or only .loadby sos mscorwks if you are attaching to the process
     -->
    <keyword name="LoadSOS">sxe -c ".loadby sos clr;gn" ld:clrjit</keyword>
    <keyword name="PrintManagedException">!PrintException</keyword>
    <keyword name="ManagedStack">!CLRStack -a</keyword>
  </keywords>
  <settings>
    <runmode>crash</runmode>
    <lastscriptcommand>qd</lastscriptcommand>
  </settings>
  <postcommands>
    <debugactions>LoadSOS</debugactions>
  </postcommands>
  <exceptions>
    <allexceptions>
      <actions1>LOG</actions1>
      <returnaction1>GN</returnaction1>
      <actions2>LOG</actions2>
      <returnaction2>GN</returnaction2>
    </allexceptions>
    <allevents>
      <actions1>LOG</actions1>
      <returnaction1>GN</returnaction1>
      <actions2>LOG</actions2>
      <returnaction2>GN</returnaction2>
    </allevents>
    <exception code="clr">
      <actions1>PrintManagedException;ManagedStack</actions1>
      <returnaction1>GN</returnaction1>
      <actions2>Log</actions2>
      <returnaction2>GN</returnaction2>
    </exception>
  </exceptions>
</adplus>

After issuing:

> adplus -c log-managed-exceptions.adplus.config -o c:\dumps -sc Thrower.exe

(ef8.145c): CLR exception - code e0434352 (first chance)
Exception object: 00000000024e1778
Exception type:   System.InvalidOperationException
Message:          ex1
InnerException:   <none>
StackTrace (generated):
<none>
StackTraceString: <none>
HResult: 80131509
OS Thread Id: 0x145c (0)
Child SP         IP               Call Site
000000000013ee38 000007fefda8cacd [HelperMethodFrame: 000000000013ee38] 
000000000013ef80 000007ff0016019f Thrower.Main(System.String[])
    PARAMETERS:
        args (0x000000000013f020) = 0x00000000024e16f0
    LOCALS:
        0x000000000013efa8 = 0x0000000000000000
        0x000000000013efb0 = 0x0000000000000000

Creating a dump when a managed exception occurs

Very often the above Adplus_log is not enough to successfully diagnose the exception’s root cause. At these moments you might try to add just few more debugger commands to keywords section (for locks, processes or threads), but definitely the best solution would be to have a memory dump (snapshot) taken at the time of the failure. With a full memory dump you will be able to examine the stacks of all the threads, read garbage collector heap(s) or find lock acquisitions. Native debuggers (such as cdb or windbg) allow you to control what should be written to the dump file. Read help of a .dump command to see all the options available. However, in managed debugging scenarios, most of the times, you will need a full memory dump (created with the \ma switch). There is already an Adplus built-in keyword for creating this kind of dumps and its name is (I bet you guessed:)) FullDump. Let’s try it in our CLR first chance exceptions:

...
  <exceptions>
    <exception code="clr">
      <actions1>PrintManagedException;ManagedStack;FullDump</actions1>
      <returnaction1>GN</returnaction1>
      <actions2>Log;FullDump</actions2>
      <returnaction2>GN</returnaction2>
    </exception>
  </exceptions>
...

After running the Thrower application under Adplus you should have noticed that four dump files were generated in the output directory. Unfortunately the time needed to write dumps to the disk impacts the application performance and responsiveness, especially when the number of thrown exceptions is high (though this often signifies a buggy application) or when the application consumes big chunks of memory. Sometimes your process runs in a strict environment and if it does not respond in a predefined amount of time it will be killed (IIS application pool might be a good example here). For reasons listed above and all others that might come to your head you should generate memory dumps carefully and only when it’s really necessary. It might be a good general practice to first run application using log-managed-exceptions.adplus.config file (eventually generating full dumps only for the failures – second chance exceptions). After analyzing the generated logs you should find exceptions that raise your suspicion and generate full dumps only for them. This is the subject of the next paragraph.

Filtering exceptions

So you’ve found exception types for which you would like Adplus to generate full memory dumps. There is a special SOS command !StopOnException that will help us here. When issued in the debugger it adds a new handling mechanism for the clr exception:

0:000> !soe -create System.ArgumentNullException 1
Breakpoint set
0:000> sx
...
 clr - CLR exception - break - not handled
       Command: "!soe  System.ArgumentNullException 1;.if(@$t1==0) {g} .else {.echo 'System.ArgumentNullException hit'}"
...

Moving back to Adplus let’s define two new keywords in the keywords section:

<keywords>
...
    <keyword name="FullDumpOnSystemArgumentException">!soe  System.ArgumentNullException 1;.if(@$t1==1) {.dump /ma /u ${AdpDumpDir}\FULLDUMP_SystemArgumentException_${AdpProcName}_.dmp}</keyword>
    <keyword name="FullDumpOnSystemInvalidOperationException">!soe  System.InvalidOperationException 1;.if(@$t1==1) {.dump /ma /u ${AdpDumpDir}\FULLDUMP_SystemInvalidOperationException_${AdpProcName}_.dmp}</keyword>
</keywords>
...

Now, we can modify the exception handling logic and use our newly defined keywords (file log-and-dump-specific-managed-exceptions.adplus.config from samples):

...
  <exceptions>
    <exception code="clr">
      <actions1>
         PrintManagedException;
         ManagedStack;
         FullDumpOnSystemArgumentException;
         FullDumpOnSystemInvalidOperationException
      </actions1>
      <returnaction1>GN</returnaction1>
      <actions2>Log;FullDump</actions2>
      <returnaction2>GN</returnaction2>
    </exception>
  </exceptions>
...

After running Adplus with the new configuration file, it should generate two full dumps for the first chance System.ArgumentNullException and System.InvalidOperationException and one full dump for the second chance System.NotImplementedException.

I hope that you find this article helpful – if you would like to play with Adplus all configuration files as well as the Thrower application can be downloaded from http://lowleveldesign.codeplex.com/releases/view/80582. Happy debugging!

Downloading a PDB file for a given PE file

Some time ago Mike Stall blogged about different Symbol APIs available for developers. As the PDB file format is a Microsoft’s secret all the tools that I’m presenting are just wrappers over those APIs. To work with PDB files we first need to get them. Let’s list the tools that will help us in it.

symchk.exe

Symbol Checker (Symchk.exe) is an application that compares executable files to symbol files to verify that the matching symbols are available. Symchk may also be used to populate your symbol cache. It can read symbol information from PE files (exe, dll), dump files and processes. It also supports recursive directory search and batch files.

We will start from loading symbols for the kernel32.dll library:

c:\Windows\System32>echo %_NT_SYMBOL_PATH%
SRV*C:\Symbols\MSS*http://referencesource.microsoft.com/symbols;SRV*C:\Symbols\MSS*http://msdl.microsoft.com/download/symbols

c:\Windows\System32>symchk /v /os kernel32.dll
[SYMCHK] Searching for symbols to c:\Windows\System32\kernel32.dll in path SRV*C:\Symbols\MSS*http://referencesource.microsoft.com/symbols;SRV*C:\Symbols\MSS*http://msdl.microsoft.com/download/symbols
DBGHELP: Symbol Search Path: SRV*C:\Symbols\MSS*http://referencesource.microsoft.com/symbols;SRV*C:\Symbols\MSS*http://msdl.microsoft.com/download/symbols
[SYMCHK] Using search path "SRV*C:\Symbols\MSS*http://referencesource.microsoft.com/symbols;SRV*C:\Symbols\MSS*http://msdl.microsoft.com/download/symbols"
DBGHELP: No header for c:\Windows\System32\kernel32.dll.  Searching for image on disk
DBGHELP: c:\Windows\System32\kernel32.dll - OK
SYMSRV:  C:\Symbols\MSS\kernel32.pdb\9B30FD7CD6B44975BF34B43B6EF668212\kernel32.pdb not found
SYMSRV:  http://referencesource.microsoft.com/symbols/kernel32.pdb/9B30FD7CD6B44975BF34B43B6EF668212/kernel32.pdb not found
SYMSRV:  kernel32.pdb from http://msdl.microsoft.com/download/symbols: 704453 bytes - copied
DBGHELP: kernel32 - public symbols
         C:\Symbols\MSS\kernel32.pdb\9B30FD7CD6B44975BF34B43B6EF668212\kernel32.pdb
[SYMCHK] MODULE64 Info ----------------------
[SYMCHK] Struct size: 1680 bytes
[SYMCHK] Base: 0x0000000078D20000
[SYMCHK] Image size: 1175552 bytes
[SYMCHK] Date: 0x4e21213b
[SYMCHK] Checksum: 0x0012386d
[SYMCHK] NumSyms: 0
[SYMCHK] SymType: SymPDB
[SYMCHK] ModName: kernel32
[SYMCHK] ImageName: c:\Windows\System32\kernel32.dll
[SYMCHK] LoadedImage: c:\Windows\System32\kernel32.dll
[SYMCHK] PDB: "C:\Symbols\MSS\kernel32.pdb\9B30FD7CD6B44975BF34B43B6EF668212\kernel32.pdb"
[SYMCHK] CV: RSDS
[SYMCHK] CV DWORD: 0x53445352
[SYMCHK] CV Data:  kernel32.pdb
[SYMCHK] PDB Sig:  0
[SYMCHK] PDB7 Sig: {9B30FD7C-D6B4-4975-BF34-B43B6EF66821}
[SYMCHK] Age: 2
[SYMCHK] PDB Matched:  TRUE
[SYMCHK] DBG Matched:  TRUE
[SYMCHK] Line nubmers: FALSE
[SYMCHK] Global syms:  FALSE
[SYMCHK] Type Info:    FALSE
[SYMCHK] ------------------------------------
SymbolCheckVersion  0x00000002
Result              0x00030001
DbgFilename
DbgTimeDateStamp    0x4e21213b
DbgSizeOfImage      0x0011f000
DbgChecksum         0x0012386d
PdbFilename         C:\Symbols\MSS\kernel32.pdb\9B30FD7CD6B44975BF34B43B6EF668212\kernel32.pdb
PdbSignature        {9B30FD7C-D6B4-4975-BF34-B43B6EF66821}
PdbDbiAge           0x00000002
[SYMCHK] [ 0x00000000 - 0x00030001 ] Checked "c:\Windows\System32\kernel32.dll"

SYMCHK: FAILED files = 0
SYMCHK: PASSED + IGNORED files = 1

Let’s try to load symbols for the notepad.exe process:

c:\temp\symtest>tasklist /FI "IMAGENAME eq notepad.exe"

Image Name                     PID Session Name        Session#    Mem Usage
========================= ======== ================ =========== ============
notepad.exe                   2264 Console                    1      6 036 K

c:\temp\symtest>symchk /ip 2264 /s SRV*.*http://msdl.microsoft.com/download/symbols

SYMCHK: FAILED files = 0
SYMCHK: PASSED + IGNORED files = 26

c:\temp\symtest>tree .
Folder PATH listing
Volume serial number is 00000002 C622:C13F
C:\TEMP\SYMTEST
├───advapi32.pdb
│   └───6AEFDCFF7F2A429B8532CD2BFDDF85D12
├───CLBCatQ.pdb
│   └───60B9D310C472440BA13F66BFF0FC39E32
├───comctl32.pdb
│   └───943BA638A2CD4D88A1C7E7418EAF796C1
├───comdlg32.pdb
│   └───631B57376F8549FDB2E7A8AB3D2D1FDF2
├───cryptbase.pdb
│   └───F03E074BB9E74C9F9BBFB0E42EF3A0AB2
├───dwmapi.pdb
│   └───8683ED0C3DBE4053883EC22FD9B4F2102
├───gdi32.pdb
│   └───FB9403C3B1304DA192C4D0E3485E25ED2
├───imm32.pdb
│   └───98F27BA5AEE541ECBEE00CD03AD50FEE2
├───kernel32.pdb
│   └───9B30FD7CD6B44975BF34B43B6EF668212
├───kernelbase.pdb
│   └───61044362232B410AA600843CEBFD11612
...

Another great functionality of symchk.exe is something called manifest files. Manifest files contain information about all symbols that must be downloaded. You may then run symchk with /om switch which will produce the manifest file without downloading any symbols. Then you can copy the manifest file to any computer that has the Internet connection and download the symbol files using /im switch. The snippet below shows an usage example:

c:\temp\symtest>symchk /om notepad-symbols.man /ip 2264

SYMCHK: FAILED files = 0
SYMCHK: PASSED + IGNORED files = 26

c:\temp\symtest>type notepad-symbols.man
notepad.pdb,36CFD5F9888C4483B522B9DB242D84782,1
notepad.exe,4a5bc9b335000,1
ntdll.pdb,6192BFDB9F04442995FFCB0BE95172E12,1
ntdll.dll,4ce7c8f91a9000,1
kernel32.pdb,9B30FD7CD6B44975BF34B43B6EF668212,1
kernel32.dll,4e21213b11f000,1
kernelbase.pdb,61044362232B410AA600843CEBFD11612,1
KernelBase.dll,4e21213c6c000,1
...
c:\temp\symtest>symchk /im notepad-symbols.man /s SRV*.*http://msdl.microsoft.com/download/symbols

SYMCHK: FAILED files = 0
SYMCHK: PASSED + IGNORED files = 52

dbh.exe

This tool is a wrapper over the DbgHelp.dll library and uncovers almost all of its functionality. We will just look at the one usage example so if you would like to go deeper have a look at the Debugging Tools for Windows help. When you run dbh.exe with a module name as an argument it will automatically download the symbol files. So by simply calling dbh c:\windows\system32\kernel32.dll info you will download the symbol file (_NT_SYMBOL_PATH environment variable is used) and print information about it and its PE file (kernel32.dll):

c:\temp>dbh c:\windows\system32\kernel32.dll info

    SizeOfStruct : 0x690
     BaseOfImage : 0x1677721664x
       ImageSize : 0x1000000
   TimeDateStamp : 0x4e21213b
        CheckSum : 0x12386d
         NumSyms : 0x0
         SymType : SymPdb
      ModuleName : kernel32
       ImageName : c:\windows\system32\kernel32.dll
 LoadedImageName : c:\windows\system32\kernel32.dll
   LoadedPdbName : C:\Symbols\MSS\kernel32.pdb\9B30FD7CD6B44975BF34B43B6EF668212\kernel32.pdb
           CVSig : 0x53445352
          CVData : kernel32.pdb
          PdbSig : 0x0
        PdbSig70 : 0x9b30fd7c, 0xd6b4, 0x4975, 0xbf, 0x34, 0xb4, 0x3b, 0x6e, 0xf6, 0x68, 0x21
          PdbAge : 0x2
    PdbUnmatched : false
    DbgUnmatched : false
     LineNumbers : false
   GlobalSymbols : false
        TypeInfo : false
   SourceIndexed : false
   PublicSymbols : true
     MachineType : X64

dumpbin.exe

You might be surprised that this tool appears here, but have you ever noticed its /PDBPATH[:VERBOSE] switch? Issuing the dumpbin /pdbpath:verbose on our kernel32.dll library will result in downloading the PDB file from the public symbol store:

c:\temp\symtest>dumpbin /pdbpath:verbose c:\windows\system32\kernel32.dll
Microsoft (R) COFF/PE Dumper Version 10.00.40219.01
Copyright (C) Microsoft Corporation.  All rights reserved.

Dump of file c:\windows\system32\kernel32.dll

File Type: DLL
  PDB file 'c:\windows\system32\kernel32.pdb' checked.  (File not found)
  PDB file 'c:\temp\symtest\kernel32.pdb' checked.  (File not found)
  PDB file found at 'C:\Symbols\MSS\kernel32.pdb\9B30FD7CD6B44975BF34B43B6EF668212\kernel32.pdb'

  Summary

        2000 .data
        A000 .pdata
       6E000 .rdata
        8000 .reloc
        1000 .rsrc
       9B000 .text

Downloading source code from the source server

srctool.exe

This tool is quite interesting as it allows you to check which source files are indexed in the PDB file and eventually extract them. With -r switch you may check which source code paths were hardcoded in the PDB file, eg.

c:\temp>srctool -r ConsoleApplication1.pdb
D:\lab\symbols-lab\symbols\ConsoleApplication1\Program.cs
D:\lab\symbols-lab\symbols\ConsoleApplication1\AdvertQuickView.cs

c:\temp>srctool ConsoleApplication1.pdb
[D:\lab\symbols-lab\symbols\ConsoleApplication1\Program.cs] cmd: cmd /c svn.exe cat "svn://localhost/test2/Program.cs@1" --non-interactive --username admin --password admin > "
[D:\lab\symbols-lab\symbols\ConsoleApplication1\AdvertQuickView.cs] cmd: cmd /c svn.exe cat "svn://localhost/test2/AdvertQuickView.cs@1" --non-interactive --username admin --password admin > "

ConsoleApplication1.pdb: 2 source files are indexed

pdbstr

If you would like to have a better control over the source server stream you may check the pdbstr command. With its aid you can read and update the source server information in the PDB file. The source server stream is actually a text block with predefined sections (more info can be found here). You can dump its content by issuing:

c:\temp>pdbstr -r -p:ConsoleApplication1.pdb -s:srcsrv > stream.txt

SRCSRV: ini ------------------------------------------------
VERSION=1
INDEXVERSION=2
VERCTRL=Subversion
DATETIME=Thu Nov 17 13:31:46 2011
SRCSRV: variables ------------------------------------------
SVNUSER=admin
SVNPASS=admin
SVN_EXTRACT_TARGET=%targ%\%fnbksl%(%var3%)\%var4%\%fnfile%(%var1%)
SVN_EXTRACT_CMD=cmd /c svn.exe cat "%var2%%var3%@%var4%" --non-interactive --username %svnuser% --password %svnpass% > "
%svn_extract_target%"
SRCSRVTRG=%SVN_extract_target%
SRCSRVCMD=%SVN_extract_cmd%
SRCSRV: source files ---------------------------------------
D:\lab\symbols-lab\symbols\ConsoleApplication1\Program.cs*svn://localhost/*test2/Program.cs*1
D:\lab\symbols-lab\symbols\ConsoleApplication1\AdvertQuickView.cs*svn://localhost/*test2/AdvertQuickView.cs*1
SRCSRV: end ------------------------------------------------

c:\temp>pdbstr -w -p:ConsoleApplication1.pdb -s:srcsrv -i:stream.txt

That will end our list of PDB tools – I hope that you will find it useful. Please remember to leave a comment if you know any other tool that should be enlisted here.

Prepare the content

Let’s start from a table of contents for a hypothetical 3-slides presentation:

Welcome

1. Slide1
1.1 Slide1.1
2. Slide2

Thank you

Save this content as a toc.txt file – this will be our source file. Now let’s prepare slides. You may create a sub-directory slides or even create a separate directories for each group of slides – as you will shortly see there is no restriction here. In our case let’s stay with the one-directory-for-all approach. Let’s now assume that we want some contents to be presented for slides entitled: Welcome, 1.1 Slide1.1 ,2. Slide2 and Thank you (we omit 1. Slide1 by purpose). As mentioned previously the slides will be presented on the console so we may safely store their contents in text files named adequately: welcome.txt, slide1.1.txt and slide2.txt. Example content for a welcome.txt might be as follows:

                     SAMPLE PRESENTATION
                     ===================

                      Sebastian Solnica

              http://lowleveldesign.wordpress.com

Now we need to prepare a mapping file that will map the slide title with its corresponding content. Create a new text file and fill it with the following text:

Welcome
 slides\welcome.txt
1.1 Slide1.1
 slides\slide1.1.txt
2. Slide2
 slides\slide2.txt
Thank you
 thank-you.txt

Each indented line stores a path to the content of the slide with the title above it. The titles must be the same as in the toc.txt file and the file paths must be valid (however as you can see not all titles need to be mapped).

Create the presentation compiler

Having the presentation content ready we now need to create an exe file that will be run by the debugger. I suppose that the easiest approach is to use classes from the System.Reflection.Emit namespace. The pseudo code for the compiler (or rather emitter) will be as follows:

var asm = new DynamicAssembly()
MakeDebuggable(asm)

var mappingsContent = ReadFile(toc_slides.txt)
var mappings = StoreMappingsInDictionary(mappingsContent) // slide title -> slide file

for each slideTitle in GetContent(toc.txt) {
  if (mappings.Contains(slideTitle)) {
    EmitMsilCodeThatWillReadAndPrintTheSlide(mappings[slideTitle]);
    MarkSequencePoint(currentLineNum); // binds a line of the source code with the emitted MSIL
  }
}

SaveAssembly(asm)

I will now explain few methods that occur in the code above that might have confused you (the C# source code that implements the above logic can be downloaded from here). MakeDebuggable is responsible for adding a DebuggableAttribute to the assembly. This attribute informs the JIT compiler that it should produce assembly code that will be debuggable (more here). To make the debugging process possible we also need to create a PDB file that will store the debug information for our module. ModuleBuilder contains a special method called DefineDocument that will do this for us. Here’s a C# code that implements all the logic described:

Type daType = typeof(DebuggableAttribute);
ConstructorInfo daCtor = daType.GetConstructor(new Type[] { typeof(DebuggableAttribute.DebuggingModes) });
CustomAttributeBuilder daBuilder = new CustomAttributeBuilder(daCtor, new object[] { 
    DebuggableAttribute.DebuggingModes.DisableOptimizations | 
    DebuggableAttribute.DebuggingModes.Default });
assemblyBuilder.SetCustomAttribute(daBuilder);

ModuleBuilder module = assemblyBuilder.DefineDynamicModule(outputFileName, true); // <-- pass 'true' to track debug info.

// Tell Emit about the source file that we want to associate this with. 
ISymbolDocumentWriter doc = module.DefineDocument(tableOfContentsPath, Guid.Empty, Guid.Empty, Guid.Empty);

EmitMsilCodeThatWillReadAndPrintTheSlide emits MSIL that will open the slide file, read it at once and print the content to the console. Ildasm the code below and you will get it:) (or check emitReadAndPrintSlideMethod in Program.cs from the slide compiler source code):

using (StreamReader reader = new StreamReader(new FileStream(fileName, FileMode.Open, FileAccess.Read, FileShare.Read)))
{
   Console.Clear();
   Console.Write(reader.ReadToEnd());
}

To make the assembly smaller you may consider putting this logic in a separate method and emit just calls to this method with adequate parameters (this is what I do in the sample code). For each slide printed we need to inform the debugger which line of the source code (or table of contents in our case:)) should be highlighted. For this purpose we use MarkSequencePoint method of the ILGenerator instance:

ilGenerator.MarkSequencePoint(doc, currentLineNum, 1, currentLineNum, 100);

To make the debugging even more feasible I emit System.Diagnostics.Debugger.Break() call after the first sequence point (so even when starting with F5 the debugger should stop on the first slide).

Compile and show:)

By now we should have the presentation compiler and the presentation content (table of contents, slide files and slide title <-> slide file mapping file) ready. So it’s time to compile it and debug it:) If you are using my compiler run the following in the command window:

SlideCompiler.exe -files toc_slides.txt -o toc.exe toc.txt

This should produce a toc.exe file (and toc.pdb) that you can now open in Visual Studio (File->Open->Project/Solution…). After pressing F5 you should see something like:

By pressing F11 or F10 you may advance the slides. To move back just select the line before the slide you want to show, press Ctrl+Shift+F10 and then F10. You may also place breakpoints on your slides:)

The code (and the sample presentation as well as my symbol presentation from WG.NET) can be found on my blog samples site.

Have fun and surprise you listeners on your next presentation!

Break into the session-open event

The best place to break into the session creation process is the SessionImpl constructor. You just need to choose New Breakpoint from the Debug menu and then Break at function… (or Ctrl + B) and in the Function box type NHibernate.Impl.SessionImpl.SessionImpl. The debugger will probably complain that it can’t bind the breakpoint at this time – just ignore this message and confirm that you want to set your breakpoint. It’s confusing that CLR uses .ctor as a constructor name when VS prefers the class name (although sometimes after binding the breakpoint it shows locations such as NHibernate.Impl.SessionImpl..ctor so don’t worry – everything is fine;)). One more caveat – in Visual Studio when you create a function breakpoint it will be set on all functions which names match the pattern that you provided. In our case we will end up with three child breakpoints as there are three constructs of the SessionImpl object (have a look at the first screenshot). There is no rule on when to create the breakpoint, but if you want to trace all the sessions you probably want to set it before even starting the debugging process.

Now, let’s start the debugger and try to do some actions in the application that will require a new session. While starting have a look at the breakpoints dialog (Debug -> Windows -> Breakpoints) and see how magically our breakpoints become available.

If you correctly prepared your symbols and sources you should now become quite happy seeing:

After the breakpoint was hit let’s examine the thread call stack:

I marked with a red rectangle two frames that actually tell us everything. The session was opened on ApplicationBeginRequest_ event by the default ISessionFactory implementation (SessionFactoryImpl). The great thing is that you can add this to the watch window and examine all the properties of the session. Also, you may set a breakpoint in the callstack and come back to ISessionFactory instance and check its properties too (like currentSessionContext that defines how long session will last and where it will be stored).

Break into the session-close event

In this paragraph we are going to step through the closing session process in order to find places where things may go off the rails. The closing process may be divided into two parts: flushing and actual closing. While flushing, NHibernate synchronizes the state of the session with the database, preparing an ordered collection of queries that must be then executed. We will start our analysis from placing an unresolved breakpoint on the NHibernate.Impl.SessionImpl.Flush. After firing the debugger (F5) our breakpoint should become alive and stop the execution at the following code:

The upper part of the picture contains the current call stack and the bottom part – the code. Normally you should first check if the Flush is called from where you planned it (in my case it was called while committing the transaction in response to the Application_EndRequest event – so no surprise here). We will be interested in the flushEventListener[i].OnFlush(new FlushEvent(this)) line as this is where the hard work is done. If you didn’t provide any own IFlushListener implementations to NHibernate the default one (DefaultFlushListener) will be used:

The marked lines are the most interesting. FlushEverythingToExecution checks the session objects states and prepares the queries, when PerformExecutions (as its name suggests) executes the queries against the database. At this moment it might be interesting to check what objects are stored in the session entity dictionary and what are their properties. To accomplish that expand the @event.Session object and choose PersistenceContext property of the SessionImpl object or just add ((NHibernate.Impl.SessionImpl)(@event.Session)).PersistenceContext expression to your watch window. Browse through the EntitiesByKey and CollectionsByKey properties.

The comment over FlushEntities explains exactly what this method does:

    
    // 1. detect any dirty entities
    // 2. schedule any entity updates
    // 3. search out any reachable collections
    private void FlushEntities(FlushEvent @event)
    {
    // other code goes here ...

In the code below you can see that it iterates through the EntityEntries collection firing OnFlushEntity event for each entity with the correct status:

The selected line above might be an interesting place for a conditional breakpoint filtering entries that we are interested in, like entry.Id.Equals(1) or entry.EntityName.Equals("NHibernate.Example.Web.Models.Item"). Our next place to visit will be DefaultFlushEntityEventListener and its OnFlushEntity handler:

The first red rectangle selects the code that checks the entity status – whether its properties had changed from loading or it was deleted or it’s a new entity. You may debug this code if you find that your object is not being updated, but before you get your hands dirty you may first check if the SessionImpl.ActionQueues is being updated by this method. ActionQueues as its name suggests contains actions that must be executed in order to synchronize the session with the database. They are grouped into queues based on their category: insertions, updated, deletions, collectionCreations, collectionUpdates, collectionRemovals. At this point you may wish to check if those collections contain a valid number of elements (or even the correct number of dirty properties). After filling the action queues there comes a time to translate them to SQL queries. This is done in the PerformExecutions method of the DefaultFlushingEventListner (or rather AbstractFlushingEventListener):

Internally ExecuteActions calls NHibernate.Engine.ActionQueue.ExecuteActions method which iterates through all the actions in all queues calling their Execute method, which internally prepares SQL queries (Persister‘s Update method) and (if configured) puts them into batches or executes immediately.

This would be the end of the description of the session flushtprocess. There is one more thing that needs some explanation – session’s closing. Fortunately for you, probably already bored readers;), this method is fairly simple and it’s main role is to close all opened connections and dispose internal NHibernate objects (as batchers, transaction wrappers etc.).

I hope that this post revealed some of the mysteries behind NHibernate session management and will help you in your future struggles with NHibernate.

Now, we are ready to index the NHibernate symbol (pdb) files with the zipped sources. Open powershell console and type:

.\sourcepack.ps1 -sourcesRoot "d:\CSharp\NH\NH\nhibernate\" -sources C:\src\NHibernate-3.1.0.GA-src.zip -symbols C:\temp\sample-app\bin -archiverCommandPath C:\bin\7za\7za.exe -verbose

where:

• d:\CSharp\NH\NH\nhibernate is a path to the source root on a computer where the NHibernate was built (you may check this path using srctool.exe or dbh.exe from the Debugging Tools for Windows – please have a look at my blog for usage examples)
• C:\src\NHibernate-3.1.0.GA-src.zip is a place where you put the downloaded source codes
• C:\temp\sample-app\bin is a place where you have your application binaries (dlls + pdbs)
• C:\bin\7za\7za.exe sourcepack requires a path to the 7za command that will be used by the debugger to extract the necessary source files (sourcepack release already contains the 7za.exe application so just copy it someplace)

After running the command you should see some yellow (verbose) messages and NHibernate pdb files are now linked with the source code zip file. It’s time to test if the debugger will be able to successfully extract them. For this purpose you should again use srctool.exe (it’s located in the srcsrv folder of the Debugging Tools for Windows installation):

srctool -l:*AbstractSessionImpl.cs NHibernate.pdb -x -d:c:\temp\nh-test

The ouput of the upper command should be:

c:\temp\nh-test\src\NHibernate\Impl\AbstractSessionImpl.cs

NHibernate.pdb: 1 source files were extracted

If you saw something different you probably made some mistake in the indexing and you will need to rerun it (I know it’s tedious and I have plans to make sourcepack usage a little bit more pleasant). If the previous sentence does not apply to your situation you are ready to fire Visual Studio and start debugging NHibernate sources with your brand new symbol files .

I also added a new page to the blog entitled My Projects and I hope that its content will grow as the time flies by:) Enough said, if you are interested in sourcepack please visit sourcepack.codeplex.com and read some documentation.

<%@ Application Language="C#" %>
<%@ Import Namespace="System.Diagnostics" %>

<script runat="server">
    public Guid AppId = Guid.NewGuid();

    public String TestMessage;

    public String GetAppDescription(String eventName)
    {
        StringBuilder builder = new StringBuilder();

        builder.AppendFormat("-------------------------------------------{0}", Environment.NewLine);
        builder.AppendFormat("Event: {0}{1}", eventName, Environment.NewLine);
        builder.AppendFormat("Guid: {0}{1}", AppId, Environment.NewLine);
        builder.AppendFormat("Thread Id: {0}{1}", System.Threading.Thread.CurrentThread.ManagedThreadId, Environment.NewLine);
        builder.AppendFormat("Appdomain: {0}{1}", AppDomain.CurrentDomain.FriendlyName, Environment.NewLine);
        builder.AppendFormat("TestMessage: {0}{1}", TestMessage, Environment.NewLine);
        builder.Append((System.Threading.Thread.CurrentThread.IsThreadPoolThread ? "Pool Thread" : "No Thread") + Environment.NewLine);

        return builder.ToString();
    }

    void Application_Start(object sender, EventArgs e) 
    {
        TestMessage = "not null";

        // Code that runs on application startup
        Trace.WriteLine(GetAppDescription("Application_Start"));
    }

    void Application_End(object sender, EventArgs e) 
    {
        //  Code that runs on application shutdown
        Trace.WriteLine(GetAppDescription("Application_End"));
    }

    void Application_Error(object sender, EventArgs e) 
    { 
        // Code that runs when an unhandled error occurs
        Trace.WriteLine(GetAppDescription("Application_Error"));
    }

    void Application_BeginRequest(object sender, EventArgs e)
    {
        Trace.WriteLine(GetAppDescription("Application_BeginRequest"));
    }

    void Application_EndRequest(object sender, EventArgs e)
    {
        Trace.WriteLine(GetAppDescription("Application_EndRequest"));
    } 
</script>

namespace ASP
{
	[CompilerGlobalScope]
	public class global_asax : HttpApplication
	{
		private static bool __initialized;
		public Guid AppId = Guid.NewGuid();
		public String TestMessage;

		protected DefaultProfile Profile
		{
			get
			{
				return (DefaultProfile)base.Context.Profile;
			}
		}
		// events go here
		[DebuggerNonUserCode]
		public global_asax()
		{
			if (!global_asax.__initialized)
			{
				global_asax.__initialized = true;
			}
		}
	}
}

To prove what I described let’s use the above Global.asax file in a very simple web application. We will use the System.Diagnostics.Trace so some changes to the web.config file are required:

<?xml version="1.0"?>
<configuration>
  <system.diagnostics>
    <trace autoflush="true">
      <listeners>
        <add name="LogFile" type="System.Diagnostics.TextWriterTraceListener" initializeData="webapp.log"/>
      </listeners>
    </trace>
  </system.diagnostics>
  <system.codedom>
    <compilers>
      <compiler language="c#" extension=".cs" compilerOptions="/d:TRACE" type="Microsoft.CSharp.CSharpCodeProvider, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" warningLevel="1"/>
    </compilers>
  </system.codedom>
  <system.web>
    <compilation debug="true" targetFramework="4.0"/>
  </system.web>
</configuration>

-------------------------------------------
Event: Application_Start
Guid: d8ab181d-b1f0-4733-83c1-04e6af5e6038
Thread Id: 1
Appdomain: /LM/W3SVC/3/ROOT-1-129556455331183204
TestMessage: not null
No Thread

-------------------------------------------
Event: Application_BeginRequest
Guid: 22089cce-f79a-4123-b455-e7f9e1091dd5
Thread Id: 5
Appdomain: /LM/W3SVC/3/ROOT-1-129556455331183204
TestMessage: 
Pool Thread

-------------------------------------------
Event: Page_Load
Guid: 22089cce-f79a-4123-b455-e7f9e1091dd5
Thread Id: 5
Appdomain: /LM/W3SVC/3/ROOT-1-129556455331183204
TestMessage: 
Pool Thread

-------------------------------------------
Event: Application_EndRequest
Guid: 22089cce-f79a-4123-b455-e7f9e1091dd5
Thread Id: 5
Appdomain: /LM/W3SVC/3/ROOT-1-129556455331183204
TestMessage: 
Pool Thread

So what’s the correct way of using global.asax? Firstly, remember:

• It’s not a singleton.
• Application_Start/End events are called only once.
• There is one instance of global_asax per request and additionally there are instances for special events

Secondly, try to apply those guidelines to your Global.asax files:

• All private fields should be initialized in the constructor.
• Static fields may be initialized either in the Application_Start or static constructor.
• Don’t use lock blocks – your request possesses this instance of the glboal_asax.

You can stop reading here if what I wrote is satisfactory for you. If you still have some doubts and would like to dig deeper keep on reading:) – I will show you how to look at the ASP.NET internals with the SOS debugger extension. Basically you start with attaching to the w3wp.exe process that your application is running in. Then load sos (replace clr with mscorwks if it’s .NET 2.0):

.loadby sos clr

0:024> !DumpHeap -type System.Web.HttpApplicationFactory
------------------------------
Heap 0
         Address               MT     Size
total 0 objects
------------------------------
Heap 1
         Address               MT     Size
000000013ff40e78 000007fee65c1900      136     
total 0 objects
------------------------------
total 0 objects
Statistics:
              MT    Count    TotalSize Class Name
000007fee65c1900        1          136 System.Web.HttpApplicationFactory
Total 1 objects

0:024> !do 000000013ff40e78 
Name:        System.Web.HttpApplicationFactory
MethodTable: 000007fee65c1900
EEClass:     000007fee63147a8
Size:        136(0x88) bytes
File:        C:\Windows\Microsoft.Net\assembly\GAC_64\System.Web\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Web.dll
Fields:
              MT    Field   Offset                 Type VT     Attr            Value Name
000007fee8b1d6e8  4000bf3       78       System.Boolean  1 instance                1 _inited
000007fee8b169d0  4000bf4        8        System.String  0 instance 000000013ff41070 _appFilename
000007fee8b1f8a0  4000bf5       10 ...tions.ICollection  0 instance 0000000000000000 _fileDependencies
000007fee8b1d6e8  4000bf6       79       System.Boolean  1 instance                1 _appOnStartCalled
000007fee8b1d6e8  4000bf7       7a       System.Boolean  1 instance                0 _appOnEndCalled
000007fee65c1988  4000bf8       18 ...pApplicationState  0 instance 000000013ff410c8 _state
000007fee8b18358  4000bf9       20          System.Type  0 instance 000000013ff40e48 _theApplicationType
000007fee8b2d360  4000bfa       28 ...Collections.Stack  0 instance 000000013ff40f00 _freeList
000007fee8b1c8b8  4000bfb       60         System.Int32  1 instance                1 _numFreeAppInstances
000007fee8b1c8b8  4000bfc       64         System.Int32  1 instance                1 _minFreeAppInstances
000007fee8b2d360  4000bfd       30 ...Collections.Stack  0 instance 000000013ff40f98 _specialFreeList
000007fee8b1c8b8  4000bfe       68         System.Int32  1 instance                1 _numFreeSpecialAppInstances
... continued ...

0:024> !do 000000013ff40f00 
Name:        System.Collections.Stack
MethodTable: 000007fee8b2d360
EEClass:     000007fee87561c0
Size:        40(0x28) bytes
File:        C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
Fields:
              MT    Field   Offset                 Type VT     Attr            Value Name
000007fee8b1aed8  4000b9f        8      System.Object[]  0 instance 000000013ff40f28 _array
000007fee8b1c8b8  4000ba0       18         System.Int32  1 instance                1 _size
000007fee8b1c8b8  4000ba1       1c         System.Int32  1 instance                1 _version
000007fee8b15b28  4000ba2       10        System.Object  0 instance 0000000000000000 _syncRoot
0:024> !da 000000013ff40f28 
Name:        System.Object[]
MethodTable: 000007fee8b1aed8
EEClass:     000007fee869fdb8
Size:        112(0x70) bytes
Array:       Rank 1, Number of elements 10, Type CLASS
Element Methodtable: 000007fee8b15b28
[0] 000000013ff76bc8
[1] null
[2] null
[3] null
[4] null
[5] null
[6] null
[7] null
[8] null
[9] null

0:024> !do 000000013ff76bc8
Name:        ASP.global_asax
MethodTable: 000007ff00147a80
EEClass:     000007ff001b25f0
Size:        240(0xf0) bytes
File:        C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\root\8fbb8387\1608fccf\assembly\dl3\7d155608\c7473357_d746cc01\App_global.asax.dll
Fields:
              MT    Field   Offset                 Type VT     Attr            Value Name
000007fee65c1988  4000b84        8 ...pApplicationState  0 instance 000000013ff410c8 _state
... some fields here ...
000007fee8b38738  4000002       d8          System.Guid  1 instance 000000013ff76ca0 AppId
000007fee8b169d0  4000003       d0        System.String  0 instance 0000000000000000 TestMessage
000007fee8b1d6e8  4000001       40       System.Boolean  1   static                1 __initialized
0:024> dd 000000013ff76ca0 
00000001`3ff76ca0  22089cce 4123f79a f9e755b4 d51d09e1 // which is 22089cce-f79a-4123-b455-e7f9e1091dd5
00000001`3ff76cb0  00000000 00000000 e8b21c98 000007fe
00000001`3ff76cc0  3ff76d10 00000001 00000000 00000000
00000001`3ff76cd0  00000000 00000000 00000000 00000000
00000001`3ff76ce0  00000000 00000000 00000000 00000000
00000001`3ff76cf0  00000001 00000000 00000002 3f3851ec
00000001`3ff76d00  00000001 00000000 00000000 00000000
00000001`3ff76d10  e8b21f18 000007fe 00000003 00000000

I hope that this post helped you better understand the Global.asax file and its role in the ASP.NET applications. The simple web project is available for download here.

<Reference Include="NHibernate">
  <HintPath>packages\NHibernate.3.1.0.4000\lib\Net35\NHibernate.dll</HintPath>
</Reference>

We are also using code repository and our .csproj file is checked-in as well as the whole packages directory. One day we noticed that one repository (DAO) function is performing really badly. While debugging we found out that the problem lies in Session.Get method. Without knowledge of what’s happening inside we are not able to tell what causes this performance loss. But we also know that NHibernate library is open-source so let’s get the source code, debug it and have a grasp at what NHibernate gurus put in there. We download the source code (NHibernate-3.1.0.GA-src.zip), save them to c:\NHibernate-src and run the compilation script (NHibernate is using nant script, we want pdb files that’s why project.config=debug, I skipped tests just to speed up the build process):

nant -t:net-3.5 -D:skip.tests=true -D:skip.manual=true -D:project.config=debug

You probably will need to download ILMerge and place it in the c:\NHibernate-src\Tools\ILMerge. I also needed to remove c:\NHibernate-src\src\NHibernate.Test.build – somehow I was getting compilation errors and didn’t have time to check why (we won’t need the NHibernate.Test.dll anyhow).

The question now is how to bind the newly compiled assemblies (from c:\NHibernate-src\build\NHibernate-3.1.0.GA-debug\bin\net-3.5\) with our projects. There are two approaches: we can either replace NHibernate references in all projects and copy our debug-enabled binaries to packages\NHibernate.3.1.0.4000\lib\Net35 folder or use ReferencePath properties. First approach has many disadvantages – we are messing project files used by all developers (.csproj), source code is on our machine so other users won’t be able to debug NHibernate, on deployment we will need to restore all release-compiled binaries. When using ReferencePath we don’t have such problems. ReferencePath is your local project property and is stored in .csproj.user file which you usually don’t upload to the repository (and if you do, you shouldn’t ). Visual Studio considers ReferencePath in the first place (before HintPath) while looking for assemblies and so when you compile your project locally it will be bound with the debug-enabled assemblies. Just to finish the subject: you set the ReferencePath by right clicking your project in Solution Explorer, choosing Properties and then ReferencePaths tab:

Click “Add Folder” and a new section should appear in your .csproj.user file:

<PropertyGroup>
  <ReferencePath>c:\NHibernate-src\build\NHibernate-3.1.0.GA-debug\bin\net-3.5\</ReferencePath>
</PropertyGroup>

Now, while debugging press F11 on Session.Get and you should be able to step NHibernate source code. Have fun!

• host – is a machine on which Visual Studio is installed and on which you control the debugging process
• remote computer – is a machine on which the debuggee is ran

Since the debugging process is wholly controlled from the host there must be a listener on the remote computer that executes commands sent from Visual Studio. This listener is called Remote Debugging Monitor (msvsmon.exe). There are two possible ways to install it – you can either download and run the setup package from microsoft or copy all the necessary files from C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\Remote Debugger (path on my 64-bit Win7). A caveat here: if your host is x86 machine than you have no choice but x86 package for your remote computer – there is no way you can debug a 64-bit process in a 32-bit debugger. Though you can debug 32-bit process in a 64-bit debugger.

Next step (the most boring and error prone in my opinion) is a network setup. Ideally Visual Studio and Remote Debugging Monitor should take care of the Windows Firewall settings after you run them with the elevated privileges but life is life and you might need to get your hands dirty. Fortunately the manual configuration process is pretty well explained on MSDN sites for WinXP, Vista and Win7. Next important thing is security. You can either perform debugging in “NoAuthentication” mode (which is of course less secure) or in Windows Authentication mode. Unfortunately Windows Authentication mode requires the same account to be setup on the host and the remote computer – both usernames and passwords must be the same. If it’s a problem for you then you are left with the “NoAuthentication” mode in which you take a risk of allowing all people to access your debugging monitor.

Having finished the configuration part we may finally start the debugging process. Begin with starting msvsmon.exe (Remote Debugging Monitor) on the remote computer – it will communicate with Visual Studio running on the host. Let’s assume that our application is named RemoteControl.exe and is written in C#. First we need to deploy it on the remote computer, for example into the C:\Users\consept\Dektop\RemoteControl folder (remember to copy also all the pdb files and the assemblies required by the main application). We can now either start the application under the debugger or attach the debugger to the already running process. Let’s start with the first scenario. If you have access to the application solution folder and to all the project files then you can start the remote debugging by modifying the start project properties page:

If you don’t have access to the application solution then you need to use the application exe file. From File menu choose Open->Project/Solution… and find the exe file for your application. Select the newly opened project and choose properties. Fill the page with following values:

Attaching to the already started process is fairly simple: from the Debug menu choose Attach to process…. If you are debugging in the Windows Authentication mode set the Transport box to Default. In Qualifier field type name of your Remote Debugging Monitor session. Usually it is your user name + @ + the remote computer name, for example Sebastian@REMOTE-PC (the session name is also displayed in the listbox after you start the msvsmon.exe).

Finally few words about the debugging process. Unfortunately the debugging symbol configuration differs depending on the chosen debugging engine (native, managed, mixed). When using managed debugging engine symbols must be accessible on the remote computer (so you need to deploy your application with all pdb files and set _NT_SYMBOL_PATH variable on the remote computer for .NET Framework debugging symbols). When using native debugging engine symbols are loaded from the host machine (so you must include the directory with all pdb files for your application into the Visual Studio symbols search path – Tools->Options->Debugging->Symbols). Interestingly source files are always loaded from the host machine (in case that you don’t use any kind of source server). To summarize: remember to always choose the managed engine for your .net applications that don’t have any interop parts. The native engine requires much more information (like symbols for all native libraries) and provides you with capabilities that are usually not necessary in debugging pure .NET application.

I haven’t touched the subject of the web applications remote debugging and running Remote Debugging Monitor as a service. Leave a comment if you wish such a post to appear in the future:)

// Brilliantly written taken from mdbg source code.
// returns a type token from name
// when the function fails, we return token TokenNotFound value.
public int GetTypeTokenFromName(string name)
{
    IMetadataImport importer = GetMetadataInterface<IMetadataImport>();

    int token = TokenNotFound;
    if (name.Length == 0)
        // this is special global type (we'll return token 0)
        token = TokenGlobalNamespace;
    else
    {
        try
        {
            importer.FindTypeDefByName(name, 0, out token);
        }
        catch (COMException e)
        {
            token = TokenNotFound;
            if ((HResult)e.ErrorCode == HResult.CLDB_E_RECORD_NOTFOUND)
            {
                int i = name.LastIndexOf('.');
                if (i > 0)
                {
                    int parentToken = GetTypeTokenFromName(name.Substring(0, i));
                    if (parentToken != TokenNotFound)
                    {
                        try
                        {
                            importer.FindTypeDefByName(name.Substring(i + 1), parentToken, out token);
                        }
                        catch (COMException e2)
                        {
                            token = TokenNotFound;
                            if ((HResult)e2.ErrorCode != HResult.CLDB_E_RECORD_NOTFOUND)
                                throw;
                        }
                    }
                }
            }
            else
                throw;
        }
    }
    return token;
}

internal sealed class MetadataType : Type
{
    private readonly IMetadataImport p_importer;
    private readonly Int32 p_typeToken;

    internal MetadataType(IMetadataImport importer, Int32 typeToken)
    {
        this.p_importer = importer;
        this.p_typeToken = typeToken;
    }

    ...
    
    public override System.Reflection.MethodInfo[] GetMethods(System.Reflection.BindingFlags bindingAttr)
    {
        IntPtr hEnum = new IntPtr();
        ArrayList methods = new ArrayList();

        Int32 methodToken;
        try
        {
            while (true)
            {
                Int32 size;
                p_importer.EnumMethods(ref hEnum, p_typeToken, out methodToken, 1, out size);
                if (size == 0)
                    break;
                methods.Add(new MetadataMethodInfo(p_importer, methodToken));
            }
        }
        finally
        {
            p_importer.CloseEnum(hEnum);
        }
        return (MethodInfo[])methods.ToArray(typeof(MethodInfo));
    }
    
    ...
}

As you could see in the listing we also used MetadataMethodInfo. The listing below presents body of this class:

internal sealed class MetadataMethodInfo : MethodInfo
{
    private readonly Int32 p_methodToken;
    private readonly Int32 p_classToken;
    private readonly IMetadataImport p_importer;
    private readonly String p_name;


    internal MetadataMethodInfo(IMetadataImport importer, Int32 methodToken)
    {
        p_importer = importer;
        p_methodToken = methodToken;

        int size;
        uint pdwAttr;
        IntPtr ppvSigBlob;
        uint pulCodeRVA, pdwImplFlags;
        uint pcbSigBlob;

        p_importer.GetMethodProps((uint)methodToken,
                                  out p_classToken,
                                  null,
                                  0,
                                  out size,
                                  out pdwAttr,
                                  out ppvSigBlob,
                                  out pcbSigBlob,
                                  out pulCodeRVA,
                                  out pdwImplFlags);

        StringBuilder szMethodName = new StringBuilder(size);
        p_importer.GetMethodProps((uint)methodToken,
                                out p_classToken,
                                szMethodName,
                                szMethodName.Capacity,
                                out size,
                                out pdwAttr,
                                out ppvSigBlob,
                                out pcbSigBlob,
                                out pulCodeRVA,
                                out pdwImplFlags);

        p_name = szMethodName.ToString();
        //m_methodAttributes = (MethodAttributes)pdwAttr;
    }

    ...
    
    public override string Name
    {
        get { return p_name; }
    }
    
    public override int MetadataToken
    {
        get { return this.p_methodToken; }
    }
}

Finally we are ready to implement the method that will return CorFunction instance:

public CorFunction ResolveFunctionName(CorModule module, String className, String functionName)
{
    Int32 typeToken = module.GetTypeTokenFromName(className);
    if (typeToken == CorModule.TokenNotFound)
        return null;

    Type t = new MetadataType(module.GetMetadataInterface<IMetadataImport>(), typeToken);
    CorFunction func = null;
    foreach (MethodInfo mi in t.GetMethods())
    {
        if (String.Equals(mi.Name, functionName, StringComparison.Ordinal))
        {
            func = module.GetFunctionFromToken(mi.MetadataToken);
            break;
        }
    }
    return func;
}

We will now concentrate on the second type of breakpoints: the code breakpoints which are set at a specific line of the source code file. Example of usage whould be “set-break mcmdtest.cs:23″. So how to set this type of breakpoint? First we need to find a module that was built from the given source file. We will iterate through all loaded modules and if a given module has symbols loaded (SymReader property != null) then we will check its documents URLS and compare them with the requested file name (snippet based on mdbg source code):

if(managedModule.SymReader==null)
  // no symbols for current module, skip it.
  return false;

foreach(ISymbolDocument doc in managedModule.SymReader.GetDocuments())
{
  if(String.Compare(doc.URL,m_file,true,CultureInfo.InvariantCulture)==0 ||
    String.Compare(System.IO.Path.GetFileName(doc.URL),m_file,true,CultureInfo.InvariantCulture)==0)
  {
     // we will fill for body later
  }
}

Having found the module we need to find a class method that the given source line belongs to. So first let’s locate the line that is the nearest sequence point to the given line by calling ISymbolDocument.FindClosestLine method. Next, with the help of the module’s ISymbolReader we will find the ISymbolMethod instance that represents our wanted function. The last step is to get the CorFunction instance based on the method’s token:

// the upper "for" body
Int32 line = 0;
try
{
  line = symdoc.FindClosestLine(lineNumber);
}
catch (System.Runtime.InteropServices.COMException ex)
{
  if (ex.ErrorCode == (Int32)HResult.E_FAIL)
      continue; // it's not this document
}
ISymbolMethod symmethod = symreader.GetMethodFromDocumentPosition(symdoc, line, 0);
CorFunction func = module.GetFunctionFromToken(symmethod.Token.GetToken());

Code breakpoints are created using ICorDebugCode.CreateBreakpoint method which takes as its parameter a code offset at which the breakpoint should be set. We will get an instance of the ICorDebugCode from the CorFunction instance (found in the last paragraph):

// from CorFunction.cs
public CorCode GetILCode()
{
    ICorDebugCode cocode = null;
    p_cofunc.GetILCode(out cocode);
    return new CorCode(cocode);
}

Then we will find the IL offset in the function IL code that corresponds to the given source file line number:

// from CorFunction.cs
internal int GetIPFromPosition(ISymbolDocument document, int lineNumber)
{
    SetupSymbolInformation();
    if (!p_hasSymbols)
        return -1;

    for (int i = 0; i < p_SPcount; i++)
    {
        if (document.URL.Equals(p_SPdocuments[i].URL) && lineNumber == p_SPstartLines[i])
            return p_SPoffsets[i];
    }
    return -1;
}

Finally we are ready to parse user’s input and set breakpoints accordingly. I used two simple regex expressions to check the breakpoint type and call process.ResolveFunctionName for function breakpoints and process.ResolveCodeLocation for code breakpoints:

static Regex methodBreakpointRegex = new Regex(@"^((?<module>[\.\w\d]*)!)?(?<class>[\w\d\.]+)\.(?<method>[\w\d]+)$");
static Regex codeBreakpointRegex = new Regex(@"^(?<filepath>[\\\.\S]+)\:(?<linenum>\d+)$");

...

// try module!type.method location (simple regex used)
Match match = methodBreakpointRegex.Match(command);
if (match.Groups["method"].Length > 0)
{
    Console.Write("Setting method breakpoint... ");

    CorFunction func = process.ResolveFunctionName(match.Groups["module"].Value, match.Groups["class"].Value,
                                                    match.Groups["method"].Value);
    func.CreateBreakpoint().Activate(true);
    
    Console.WriteLine("done.");
    continue;
}
// try file code:line location
match = codeBreakpointRegex.Match(command);
if (match.Groups["filepath"].Length > 0)
{
    Console.Write("Setting code breakpoint...");

    int offset;
    CorCode code = process.ResolveCodeLocation(match.Groups["filepath"].Value, 
                                               Int32.Parse(match.Groups["linenum"].Value), 
                                               out offset);
    code.CreateBreakpoint(offset).Activate(true);

    Console.WriteLine("done.");
    continue;
}

I also corrected the main debugger loop so it is starting to look as a normal debugger command line . As always the source code is available under http://mindbg.codeplex.com (revision 55832).