Writing a .net debugger (part 2) – handling events and creating wrappers

In this part I will describe which events the debugger has to deal with and how it should respond to them. Additionally we will create few COM wrappers for ICorDebug* interfaces. Let’s first examine the ICorDebugManagedCallback interface (imported from COM object – more in part 1). You may notice that each event handler has its own set of parameters, but the first parameter is always of type either ICorDebugAppDomain or ICorDebugProcess. Both ICorDebugAppDomain and ICorDebugProcess implement ICorDebugController which allows you to control the debuggee.

In part 1 we ended with an application that could start a new process or attach to the running one and then stop it. We will now find a way to make the process running and log all events coming from it. Let’s introduce a simple HandleEvent method which will be called from all other event handlers (except ICorDebugManagedCallback.ExitProcess):

void HandleEvent(ICorDebugController controller)
{
    Console.WriteLine("event received");
    controller.Continue(0);
}

All events handlers bodies (except ICorDebugManagedCallback.ExitProcess) will now look as follows:

{
    HandleEvent(pAppDomain); // or HandleEvent(pProcess) if first parameter is pProcess
}

If we now execute our application it will print few “event received” messages and then stop. Under the debugger we will see that the debugging API throws a COM exception:

System.Runtime.InteropServices.COMException crossed a native/managed boundary
  Message=Unrecoverable API error. (Exception from HRESULT: 0x80131300)
  Source=mindbg
  ErrorCode=-2146233600
  StackTrace:
       at MinDbg.NativeApi.ICorDebugController.Continue(Int32 fIsOutOfBand)
       at ...

It took me some time to figure out why this error occurred. It seems that in order to be able to receive all managed events the debugger must be attached to the debuggee appdomains. So I needed to modify the ICorDebugManagedCallback.CreateAppDomain handler by adding one new instruction:

void ICorDebugManagedCallback.CreateAppDomain(ICorDebugProcess pProcess, ICorDebugAppDomain pAppDomain)
{
    pAppDomain.Attach();
    HandleEvent(pProcess);
}	

Finally our debugged process executes normally without any interruption from the debugger side.

Let’s now focus on the second topic of the post: ICorDebug COM wrappers. As COM objects are not very comfortable in use and certainly we don’t want to pass them outside the bounds of the assembly we need to create a way to represent them in our application. For each used ICorDebugName interface we will create a CorName class that will implement methods and properties which will give access to the inner COM object API. Additionally these wrappers should be comparable so we could later check whether the object returned from a callback function is the same as the one that we already have. This is even better explained in the mdbg source code – look at the comment for the WrapperBase class. Our WrapperBase class would be actually almost the same as the one in mdbg sources:

/// <summary>
/// A base class for all COM wrappers. 
/// </summary>
public abstract class WrapperBase : MarshalByRefObject
{
    protected WrapperBase(Object value)
    {
        Debug.Assert(value != null);
        coobject = value;
    }

    /// <summary cref="System.Object.Equals">
    /// </summary>
    public override bool Equals(Object value)
    {
        if (!(value is WrapperBase))
            return false;
        return ((value as WrapperBase).coobject == this.coobject);
    }

    /// <summary cref="System.Object.GetHashCode">
    /// </summary>
    public override int GetHashCode()
    {
        return coobject.GetHashCode();
    }

    /// <summary>
    /// Override also equality operator so we compare 
    /// COM objects inside instead of wrapper references.
    /// </summary>
    /// <param name="operand">first operand</param>
    /// <param name="operand2">second operand</param>
    /// <returns>true if inner COM objects are the same, false otherwise</returns>
    public static bool operator ==(WrapperBase operand, WrapperBase operand2)
    {
        if (Object.ReferenceEquals(operand, operand2))
            return true;

        if (Object.ReferenceEquals(operand, null))               // this means that operand==null && operand2 is not null 
            return false;

        return operand.Equals(operand2);
    }

    /// <summary>
    /// Override also inequality operator so we compare 
    /// COM objects inside instead of wrapper references.
    /// </summary>
    /// <param name="operand">first operand</param>
    /// <param name="operand2">second operand</param>
    /// <returns>true if inner COM objects are different, true otherwise</returns>
    public static bool operator !=(WrapperBase operand, WrapperBase operand2)
    {
        return !(operand == operand2);
    }

    private Object coobject;
}

That’s all for today. The source code for the second part is available here. You may also connect to the TFS repository and download the 54704-th revision of the code.