In this post I am going to talk about the async programming model in Indigo. Indigo supports the Asynchronous Programming pattern common in the .NET Framework on the client and the server. An overview of how to use the Asynchronous Programming pattern in the .NET Framework can be found here.
As a concrete example, suppose we have the following contract:
[ServiceContract]
interface ISleep
{
[OperationContract]
void Sleep(int milliseconds);
}
Indigo supports expressing this contract on the client or server as:
[ServiceContract]
interface ISleep
{
[OperationContract(AsyncPattern=true)]
IAsyncResult BeginSleep(int milliseconds, AsyncCallback callback, object state);
void EndSleep(IAsyncResult result);
}
On the client, this is great for introducing parallelism without getting too wacky with threads, though there are some interesting issues we'll discuss later.
On the server, this is non-trivial to implement, but once you do you can do long-running asynchronous I/O operations without eating a thread the whole time. This turns out to be useful when building scalable services.
Here is a sample that shows a client making async calls on a sync server. The output shows that the calls run concurrently on the server, even though the client uses just one thread. At the end, the client just calls End to wait for the results.
Client Async
Here is another sample that shows an asynchronous middle-tier service. Both the client and the backend server are implemented synchronously. The output demonstrates that a thread is only active on the middle tier just before and just after the backend server call.
Middle-tier Async
Sync vs. Async is a Local Thing
As you might expect, Indigo allows the same operation to be defined both synchronously and asynchronously on the same interface. However, it may surprise you that calling either from the client results in the same message sent to the server. This has interesting implications.
One implication is that an async client can talk to a sync server and vice versa, as shown in the samples above.
Another is that if the server defines sync and async variants of the same operation, we have a demux problem. Since both versions of the method have the same wire format, nothing about the message says which one to call. In this case, Indigo chooses the synchronous version because the synchronous version generally results in higher throughput.
Sometimes people expect the sync method on the client to invoke the sync method on the server, and the async method on the client to invoke the async method on the server. This is NOT how it works.
If you care about whether Indigo invokes you synchronously or asynchronously (and you probably should), you need to ensure that the interface you implement only has one variant of each method. Since async is local in Indigo, you can still have both variants on the client.
How does this work?
Let's dissect at a simple async call. To illustrate and keep the discussion simple, I am assuming an async request-reply scenario occuring on the client using TCP, where the caller passes a callback.
The pseudocode below tries to show a thread's callstack over time. Time progresses vertically, and entering/exiting a scope indicates calling/returning from a method.
Client
Calls to an async proxy method ultimately resolve down to an async I/O. If the I/O does not complete synchronously, call returns control to the caller, who goes back to doing whatever it wants.
...
proxy.BeginCall()
{
channel.BeginSend()
{
Start async I/O ---Request-->
}
channel.BeginReceive();
}
...
At some point (if all goes well), a reply comes back and the BeginReceive completes. This calls the callback that the proxy passed to BeginReceive, which can then call the user's callback:
I/O-CompletionPortCallback() <---Reply---
{
ReceiveCallback()
{
channel.EndReceive();
UserCallback()
{
Message reply = proxy.EndCall()
...
}
}
}
This leaves out a lot of details, but hopefully gives you some idea of what is going on here.
Stack Dive
Before we look at the server, I want to digress into an issue that can plague naive uses of the async pattern.
Thinking about the pseudocode above, imagine if one of the things in the "..." inside UserCallback() was calling BeginCall again. This is probably a bug. I/O may complete synchronously, which results in calling I/O-CompletionPortCallback inside of the channel.BeginSend call.
There are scenarios where this could end up recursing indefinitely. Recursing indefinitely is generally a Bad Idea.
This is what the CompletedSynchronously property on IAsyncResult is for. In your callback, you can continue working if IAsyncResult.CompletedSynchronously is false on your parameter. In your caller, you can continue working if IAsyncResult.CompletedSynchronously is true on the return value. This prevents unwanted recursion.
Of course, if you don't do anything interesting in your callback, you don't have to worry about this.
Here is an example (minus error handling) of how to do an infinite loop calling an async method repeatedly while avoiding stack dive:
void AsyncLoop()
{
for (;;)
{
IAsyncResult result = proxy.BeginCall(this.Callback, proxy);
if (!result.CompletedSynchronously)
break;
this.InvokeEnd(proxy, result);
}
}
void Callback(IAsyncResult result)
{
if (result.CompletedSynchronously)
return;
this.InvokeEnd((IMyProxyType)result.AsyncState, result);
this.AsyncLoop();
}
void InvokeEnd(IMyProxyType proxy, IAsyncResult result)
{
proxy.EndCall(result);
}
Server
On the server, when a message arrives for your service operation, Indigo calls your corresponding Begin method and passes a callback. At this point, you own the request--assuming you don't complete synchronously, you can return the thread to the system. You are responsible for making sure that something wakes up eventually that calls the callback.
If you are implementing an async server method, it is important to set CompletedSynchronously correctly before letting Indigo see the IAsyncResult. Setting CompletedSynchronously to true means you are calling the callback method on the same thread that called your Begin method. It is also important that the IAsyncResult returned from Begin is the same IAsyncResult passed to the callback. Errors in these cases typically result in a hang or an Exception.
Exceptions
How do Exceptions work with all of this? It turns out that Exceptions can sometimes come back from the Begin call. This means whatever Exception handling you are doing has to be done in two places instead of one. In case you were wondering why I broke out InvokeEnd() above as a separate method, this allows you to put your Exception handling for the EndCall in a single method. So Begin and End can both throw on client and server.
What about the callback method? If I invoke a callback method when an async method completes and it throws an Exception, what does that Exception mean? I'm supposed to be doing the Exception throwing, since I implemented the method. Normally I would throw an Exception back to my caller from End. However, the callback is my only mechanism for telling my caller to call End, and it just threw. At this point, neither of us is able to recover the system to a known Correct state.
Should I let it go? Nobody is below me on the stack to handle that Exception. (Or worse, someone is on the stack eating all Exceptions and the Exception silently disappears).
Should I try calling again? I am likely to end up with more (possibly unbounded) Exceptions.
Should I swallow all Exceptions? This is a possibility, but extremely dangerous in cases where the Exception is AccessViolation or something else that could result in data corruption.
Indigo Beta 1 rethrows the Exception on a clean thread, with the goal of triggering the default CLR behavior for unhandled Exceptions. By default this unloads the AppDomain or process. However, I believe there is a way to configure a standalone App to behave differently. In WebHost (or as a Windows Service), this works well, since ASP.NET just spins up a new AppDomain. In self-host this is not quite as nice (your process exits), but at least you didn't go on to corrupt your database or open a security hole in your application.
Indigo will not normally throw Exceptions at you from its callback on the server. We provide a custom error handling mechanism for the application to handle Exceptions, and we use that to avoid throwing Exceptions here. Of course, an error handler may throw, or you may get catastrophic Exceptions like OutOfMemory or ThreadAbort, but these should not be handled. Therefore, I believe it is reasonable for async server implementations to either let any Exceptions fall through, or throw the Exception on a clean thread.
Threading considerations
One of the features of Indigo I hope to talk more about later is its integration with the CLR SynchronizationContext. I want to just briefly mention that Indigo does capture and use the SynchronizationContext to ensure that the callback for an async operation does occur on the correct thread.
What this means for you is that if you are using Indigo from your Avalon or WinForms application, and you do an async call to Indigo, you don't have to do any crazy threading stuff like control.Invoke if you call UI components from the callback thread.
Hopefully this gives you some idea of how the .NET Framework Asynchronous Programming pattern fits into Indigo.
