# Thursday, 28 September 2006

I'm working on a new project these days, and we're going all out with the bleeding edge goodies, including .NET 3.0.  To set the stage, we've got a server that internally calls a WF workflow instance, which is inherently asynchronous, so the interface we expose to clients is a WCF "Duplex Contract" which basically means two sets of one-way messages that are loosely coupled.  Think MSMQ, or other one-way message passing applications. 

[ServiceContract(CallbackContract=typeof(ITestServiceCallback))]

    public interface ITestService

    {

        [OperationContract(IsOneWay=true)]

        void SignOn(SignOnRequest request);

    }

 

    [ServiceContract()]

    public interface ITestServiceCallback

    {

        [OperationContract(IsOneWay = true)]

        void SignOnResponse(SignOnResponse response);

    }

However, we need to call this duplex contract from an ASP.NET application, since that's how we write out clients 80% of the time.  Since HTTP is inherently synchronous, this poses a bit of a challenge.  How do you call a duplex contract, but make it look synchronous to the caller?

The solution I came up with was to use a generated wrapper class that assigns each outgoing message a correlation id (a GUID), than uses that id as a key into a dictionary where it stores a System.Threading.WaitHandle, in this case really an AutoResetEvent.  Then the client waits on the WaitHandle.

When the response comes back (most likely on a different thread) the wrapper matches up the correlation id, gets the wait handle, and signals it, passing the response from the server in another dictionary which uses the WaitHandle as a key.  The resulting wrapper code looks like this:

 

public class Wrapper : ITestServiceCallback

{

    #region private implementation

    private static Dictionary<Guid, WaitHandle> correlationIdToWaitHandle = new Dictionary<Guid,WaitHandle>();

    private static Dictionary<WaitHandle, Response> waitHandleToResponse = new Dictionary<WaitHandle, Response>();

    object correlationLock = new object();

 

    private static void Cleanup(Guid correlationId, WaitHandle h)

    {

        if(correlationIdToWaitHandle.ContainsKey(correlationId))

            correlationIdToWaitHandle.Remove(correlationId);

        if(waitHandleToResponse.ContainsKey(h))

            waitHandleToResponse.Remove(h);

    }

 

    ITestService client = null;

 

    #endregion

 

    public Wrapper()

    {

        client = new TestServiceClient(new InstanceContext(this));

    }

 

    public SignOnResponse SignOn(SignOnRequest request)

    {

        request.CorrelationId = Guid.NewGuid();

        WaitHandle h = new AutoResetEvent(false);

 

        try

        {

            lock (correlationLock)

            {

                correlationIdToWaitHandle.Add(request.CorrelationId, h);

            }

 

            client.SignOn(request);

 

            h.WaitOne();

 

            SignOnResponse response = (SignOnResponse)waitHandleToResponse[h];

 

            return response;

        }

        finally

        {

            Cleanup(request.CorrelationId, h);

        }

    }

 

    #region ITestServiceCallback Members

 

    void ITestServiceCallback.SignOnResponse(SignOnResponse response)

    {

        Guid correlationId = response.CorrelationId;

 

        WaitHandle h = correlationIdToWaitHandle[correlationId];

 

        waitHandleToResponse.Add(h, response);

 

        ((AutoResetEvent)h).Set();

 

        return;

    }

 

    #endregion

}

This worked just fine in my command line test app, but when I moved to an actual Web app, hosted either in IIS or in VisualStudio 2005, I started getting random but very frequent exceptions popping up in my client app, AFTER the synchronous method had completed.  The exception had a very long and pretty incomprehensible call stack

System.ServiceModel.Diagnostics.FatalException was unhandled
Message="Object reference not set to an instance of an object."
Source="System.ServiceModel"
StackTrace:
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage4(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage3(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage2(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage1(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.Dispatch(MessageRpc& rpc, Boolean isOperationContextSet)
at System.ServiceModel.Dispatcher.ChannelHandler.DispatchAndReleasePump(RequestContext request, Boolean cleanThread, OperationContext currentOperationContext)
at System.ServiceModel.Dispatcher.ChannelHandler.HandleRequest(RequestContext request, OperationContext currentOperationContext)
at System.ServiceModel.Dispatcher.ChannelHandler.AsyncMessagePump(IAsyncResult result)
at System.ServiceModel.Dispatcher.ChannelHandler.OnAsyncReceiveComplete(IAsyncResult result)
at System.ServiceModel.Diagnostics.Utility.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
at System.ServiceModel.AsyncResult.Complete(Boolean completedSynchronously)
at System.ServiceModel.AsyncResult.Complete(Boolean completedSynchronously, Exception exception)
at System.ServiceModel.Channels.InputQueue`1.AsyncQueueReader.Set(Item item)
at System.ServiceModel.Channels.InputQueue`1.Dispatch()
at System.ServiceModel.Channels.InputQueueChannel`1.Dispatch()
at System.ServiceModel.Channels.ReliableDuplexSessionChannel.ProcessDuplexMessage(WsrmMessageInfo info)
at System.ServiceModel.Channels.ClientReliableDuplexSessionChannel.ProcessMessage(WsrmMessageInfo info)
at System.ServiceModel.Channels.ReliableDuplexSessionChannel.HandleReceiveComplete(IAsyncResult result)
at System.ServiceModel.Channels.ReliableDuplexSessionChannel.OnReceiveCompletedStatic(IAsyncResult result)
at System.ServiceModel.Diagnostics.Utility.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
at System.ServiceModel.AsyncResult.Complete(Boolean completedSynchronously)
at System.ServiceModel.AsyncResult.Complete(Boolean completedSynchronously, Exception exception)
at System.ServiceModel.Channels.ReliableChannelBinder`1.InputAsyncResult`1.OnInputComplete(IAsyncResult result)
at System.ServiceModel.Channels.ReliableChannelBinder`1.InputAsyncResult`1.OnInputCompleteStatic(IAsyncResult result)
at System.ServiceModel.Diagnostics.Utility.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
at System.ServiceModel.AsyncResult.Complete(Boolean completedSynchronously)
at System.ServiceModel.AsyncResult.Complete(Boolean completedSynchronously, Exception exception)
at System.ServiceModel.Channels.InputQueue`1.AsyncQueueReader.Set(Item item)
at System.ServiceModel.Channels.InputQueue`1.EnqueueAndDispatch(Item item, Boolean canDispatchOnThisThread)
at System.ServiceModel.Channels.InputQueue`1.EnqueueAndDispatch(T item, ItemDequeuedCallback dequeuedCallback, Boolean canDispatchOnThisThread)
at System.ServiceModel.Channels.InputQueue`1.EnqueueAndDispatch(T item, ItemDequeuedCallback dequeuedCallback)
at System.ServiceModel.Channels.InputQueue`1.EnqueueAndDispatch(T item)
at System.ServiceModel.Security.SecuritySessionClientSettings`1.ClientSecurityDuplexSessionChannel.CompleteReceive(IAsyncResult result)
at System.ServiceModel.Security.SecuritySessionClientSettings`1.ClientSecurityDuplexSessionChannel.OnReceive(IAsyncResult result)
at System.ServiceModel.Diagnostics.Utility.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
at System.ServiceModel.AsyncResult.Complete(Boolean completedSynchronously)
at System.ServiceModel.AsyncResult.Complete(Boolean completedSynchronously, Exception exception)
at System.ServiceModel.Security.SecuritySessionClientSettings`1.ClientSecuritySessionChannel.ReceiveAsyncResult.OnReceive(IAsyncResult result)
at System.ServiceModel.Diagnostics.Utility.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
at System.ServiceModel.AsyncResult.Complete(Boolean completedSynchronously)
at System.ServiceModel.AsyncResult.Complete(Boolean completedSynchronously, Exception exception)
at System.ServiceModel.Channels.ReliableChannelBinder`1.InputAsyncResult`1.OnInputComplete(IAsyncResult result)
at System.ServiceModel.Channels.ReliableChannelBinder`1.InputAsyncResult`1.OnInputCompleteStatic(IAsyncResult result)
at System.ServiceModel.Diagnostics.Utility.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
at System.ServiceModel.AsyncResult.Complete(Boolean completedSynchronously)
at System.ServiceModel.AsyncResult.Complete(Boolean completedSynchronously, Exception exception)
at System.ServiceModel.Channels.InputQueue`1.AsyncQueueReader.Set(Item item)
at System.ServiceModel.Channels.InputQueue`1.Dispatch()
at System.ServiceModel.Channels.InputQueue`1.OnDispatchCallback(Object state)
at System.ServiceModel.Channels.IOThreadScheduler.WorkItem.Invoke()
at System.ServiceModel.Channels.IOThreadScheduler.ProcessCallbacks()
at System.ServiceModel.Channels.IOThreadScheduler.CompletionCallback(Object state)
at System.ServiceModel.Channels.IOThreadScheduler.ScheduledOverlapped.IOCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* nativeOverlapped)
at System.ServiceModel.Diagnostics.Utility.IOCompletionThunk.UnhandledExceptionFrame(UInt32 error, UInt32 bytesRead, NativeOverlapped* nativeOverlapped)
at System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)

Obvious? Not so much.

Interestingly, if I removed the lock around the hashtable insertion in the signon method

            lock (correlationLock)

            {

                correlationIdToWaitHandle.Add(request.CorrelationId, h);

            }

everything worked just fine.  Hmmmm. 

The problem, it turned out, was that ASP.NET uses (by default) a little thing called the SynchronizationContext.  As near as I can tell (I haven't researched this thoroughly, to be honest) one of it's jobs it to make sure that any callbacks get run on the UI thread, thereby obviating the need to call Control.Invoke like you do in WinForms.  In my case, that additional lock was giving something fits, and it was trying to clean stuff up on a thread that wasn't around any more, hence to NullReferenceException. 

The solution was provided by Steve Maine (thanks Steve!).  Unbeknownst to me (since I didn't know there was a SynchronizationContext) you can turn it off.  Just add a CallbackBehavior attribute to your client side code, with UseSynchronizationContext=false.  So now my wrapper class looks like this

[CallbackBehavior(UseSynchronizationContext=false)]

public class Wrapper : ITestServiceCallback

and everything works fine. 

I'm open to other ways of making this work, so I'd love to hear from people, but right now everything is at least functioning the way I expect.  Several people have wondered why we don't just make the server-side interface synchronous, but since we are calling WF internally, we'd essentially have to pull the same trick, only on the server side.  I'd rather burn the thread on the client then tie up one-per-client on the server.  Again, I'm open to suggestions.

CodeGen | Indigo | Work
Tuesday, 03 October 2006 11:27:38 (Pacific Daylight Time, UTC-07:00)
Are you using ASP.NET 2.0 AsyncPages here?
Tuesday, 03 October 2006 11:34:15 (Pacific Daylight Time, UTC-07:00)
I am not. Apparently there was a problem with that before that they fixed, but since I was mucking around with the thread synchronization myself that didn't help.
Tuesday, 03 October 2006 12:00:27 (Pacific Daylight Time, UTC-07:00)
Interesting.
If you find some time to post a stripped-down demo, that would be nice ;)
Comments are closed.