Why I wish C# never got async/await

async/await is an abstraction aiming to solve the problem of asynchronous programming. Asynchronous programming is difficult so we certainly need some tools to help us reason about our asynchronous programs. While async/await is certainly a brave attempt to give us the tool we wanted I am not sure it’s the tool we need.

Concurrency

Asynchronous programming is about concurrent programming which is difficult. Concurrency is partly difficult to understand because it’s an overloaded noun, it means too much. Herb Sutter argues it makes sense to split concurrency into three subdomains.

  1. Scalability/Performance
  2. Responsiveness
  3. Consistency

Scalability deals with performance, responsiveness is about not letting a lengthy operation stop the program from responding and consistence has concepts to allow us writing correct concurrent programs.

We do concurrent programming primarily for two reasons.

  1. We want to scale (across multiple CPUs) for better performance
  2. We want to keep an application responsive when we are waiting for something non-CPU bound to complete like I/O.

Consistency is a cursed necessity because we do concurrent programming.

Responsiveness

It’s important not to block an application because it is poor usability if the user can’t interact with it. Also its poor usage of the CPU resource as demonstrated by @brendangregg (from “Systems Performance: Enterprise and the Cloud”).

Latencies as explained by @brendangregg

What this image says is that if we normalize a clock cycle (the smallest unit of time for a CPU) to 1 sec the CPU have to wait for 4 “years” for a web request to finish. It makes sense to let the CPU continue with other tasks while waiting for the request to finish, ie asynchronous programming ie responsiveness.

async/await

async/await is used to achieve responsiveness ie don’t block the thread while we are waiting for the web request to finish. We can use threads to achieve responsiveness but the advantage of async/await is that it can reuse the calling thread. On a web server this is a big deal as spinning up 1000 threads to handle multiple requests means the operating system will waste precious clock cycles on thread scheduling.

Of course, like with everything in computer science asynchronous programming isn’t a new idea. In Win32 it was called OVERLAPPED operations and has been around for ages. When programming against I/O hardware on Atari ST we implemented asynchronous programming using hardware interrupts.

For good intentions async/await also tries to protect us from having to deal with consistency.

async/await wants to solve responsiveness and consistency in one mean abstraction.

Great but this leads to a quite complex abstraction. Is complexity a problem? As long as it just works let the developers at Microsoft deal with the complexities and we can continue our lives blissfully ignorant.

The problem is that most abstractions are in some sense leaky and async/await is no exception. Leaky concurrency abstractions are the worst because if we use them without understanding their limitations we get race-conditions and dead-locks. Leaky abstractions such as binary floating point values (ie double) at least doesn’t dead-lock.

Consider this very simple usage of async/await.

        public static async Task<string> ReadSomeTextAsync(string fileName)
        {
            TraceThreadId ();

            try
            {
                using (var sr = new StreamReader(fileName))
                {
                    var result = await sr.ReadToEndAsync();
                    return result;
                }
            }
            finally
            {
                TraceThreadId ();
            }
        }

If we call the function like this does the code work?

            var readTask    = SomeClass.ReadSomeTextAsync ("SomeText.txt");
            var text        = readTask.Result;

The answer is; it depends.

If we run it from a console application it works, if we run it from a GUI or ASP.NET it dead-locks. This is expertly explained here but in short it depends on a global state called SynchronizationContext which is unknown to most .NET developers. Depending on which SynchronizationContext is used the program either works or dead-locks ie the abstraction leaks which SynchronizationContext is used.

By the way, SynchronizationContext is set per thread so the code above may work in one thread but not another.

But it will get worse; does this code work?

        // Let's track how many file we read concurrently
        int readingFiles;

        async Task<string> ReadTask ()
        {
            SomeClass.TraceThreadId ();

            ++readingFiles;

            var text = await SomeClass.ReadSomeTextAsync("SomeText.txt");

            --readingFiles;

            SomeClass.TraceThreadId ();

            return text;
        }

        // Some function that uses ReadTask
        var task = ReadTask ()

Once again the answer is; it depends.

If we run it from a console application we have a race-condition, if we run it from a GUI or ASP.NET it works. This also depends on the global SynchronizationContext.

By studying the output of the TraceThreadId method we see that in ASP.NET/GUI it’s the same thread that enters ReadTask and that exits ReadTask ie no problems. When we run it as a Console application we see that ReadTask is entered by one thread and exited by another ie readingFiles is accessed by two separate threads with no synchronization primitives which mean we have a race-condition.

Once again async/await leaks what SynchronizationContext is used.

In addition this means that one of the benefits of async/await over threads ie reusing the calling thread isn’t happening in a console application. It also means that the documentation for async/await is incorrect for console application. The documentation states: “The async and await keywords don’t cause additional threads to be created … you don’t have to guard against race conditions” but as we saw this depends on what SynchronizationContext is used and sometimes await may create additional threads.

The race-condition is much worse than the dead-lock because the dead-lock always occurs which means we fix it. The race-condition silently occurs and might get shipped.

In order to use async/await correctly we have to explicit knowledge how the runtime behavior of async/await is affected by the SynchronizationContext. If I am writing a library then I have to assume worst case:

  1. Continuations might execute on another thread so I have to protect shared resources using mutexes or similar
  2. Task.Wait on async/await task might dead-lock.

Remember that async/await wanted to solve responsiveness and consistency but to me it seems that it doesn’t quite reach this goal.

COM Apartments

COM Apartments

At this point async/await makes me remember COM Apartments. Contrary to popular belief they were actually invented to help developers writing concurrent software without having to care about things like race conditions and locks.

In short:

  1. COM objects belongs to a single apartment
  2. An apartment has specifics thread semantics
  3. The most common type of apartment was the STA which guaranteed that an object living in that apartment only get called by a specific thread. This solves the consistency problems
  4. A process may have many apartments, this allows us to design responsive applications
  5. All calls to an object goes through proxies which make sure the thread semantic of the housing apartment is obeyed, in the case of STA it means the proxy does a thread-switch to the correct thread

This sounds great! Why did developers not like COM apartments?

  1. COM apartments were leaky abstractions
  2. COM apartments were a complex abstractions making them hard to understand
  3. As COM apartments were leaky it meant that you could get race-conditions or dead-locks if you used the proxies wrongly
  4. As COM apartments were complex most developers never understood them which meant many late nights hunting for race-conditions
  5. COM apartments were opaque meaning when things failed it was very difficult to see why things went wrong

In school we learn about mutexes, threads and so on, COM Apartments tries to remove these concerns but since it’s a leaky abstraction we have to learn about message loops, proxies, agile objects, marshalling pointers and so on. Most developers seems to be thinking that that trade-off isn’t worth it.

Comparing COM and async/await

In my opinion async/await shares many traits with COM apartments

  1. async/await is a leaky abstraction
  2. async/await is a complex abstraction
  3. As async/await is leaky it means we can get race-conditions or dead-locks
  4. As async/await is complex it means the most devs don’t understand the abstraction
  5. async/await is an opaque abstraction by this I mean that under the hood async/await transforms your code into an invisible state machine. When we get dead-locks the code often blocks inside this state machine and the callstack has no trace of your code making it difficult to understand why we dead-locked.

In school we learn about mutexes, threads and so on, async/await tries to remove these concerns but since it’s a leaky abstraction we have to learn about SynchronizationContext, continuations, state machines and so on. Is it worth it?

Conclusion

The goal of COM apartments and async/await is to make life easier for programmers by attempting to be the abstraction we wanted but I am not sure it is the abstraction we need.

I think the abstraction we need ought to be:

  1. Simple
  2. Transparent

In order to be simple the abstraction need to focus on one task: Responsiveness

In order to be transparent the abstraction shouldn’t switch threads or start new threads implicitly. Switching threads or starting threads have significant impact on the semantics of a program and shouldn’t just happen.

My experience makes me believe that it’s not practically possible to make an abstraction that won’t leak locks or threads so I accept that the abstraction will be leaky but because it’s simple and transparent this will be manageable.

Where can we find such an abstraction?

Drinking the F# koolaid

IMO the F# abstraction async manages to be the abstraction we need.

async is implemented using F# computation expression. Computation expressions are not simple to understand but computation expressions can be independently understood and are well-defined.

For an F# developer that grasps computation expression the definition of async is simple (reverse-engineered)

type Async<'T> = ('T->unit)*(exn->unit)*(OperationCanceledException->unit)->unit

In english the this means that a async is function that takes three functions as input

  1. A continuation function to call if the operation succeeds (ie I managed to read the file)
  2. A continuation to call if the operation fails with an exception
  3. A continuation to call if we detect that the operation should be cancelled prematurely

Together with F# computations expressions this builds a very powerful and transparent abstraction.

Using this simple abstraction together with a computation expression builder we can define asynchronous workflows such as my game loop (for a very simple game)

            let gameLoop =
                // async is kind-of like async in C#
                async {
                    // Switching to new thread is explicit
                    do! Async.SwitchToNewThread ()
                    while true do
                        // let! is kind-of like await in C#
                        let! messages = fromVisual.AsyncDequeue 1000

                        for message in messages do
                          processMessage message
                }

Apart from the simplicity and transparency what makes me love async is the openness. For instance I needed to switch to a new thread that had STA semantics (yes COM apartments again) because the thread is opening a UI. I couldn’t find a built in function that did just that so I defined my own that let me specify the thread priority and apartment:

    let SwitchToThread2 (state : ApartmentState) (tp : ThreadPriority): Async<unit> =
        Async.FromContinuations <| fun (cont, econt, ccont) ->
                try
                    let thread = Thread(fun () ->
                                    try
                                        cont ()
                                    with
                                    | e -> econt e
                                    )
                    thread.IsBackground <- true
                    thread.SetApartmentState state
                    thread.Priority <- tp
                    thread.Start ()
                with
                | e -> econt e

    // Defines my gui workflow 
    let formProcessor ct toui fromui sharedResources = 
        async {
            // Switching to new STA thread and higher prio
            do! Async.SwitchToThread2 ApartmentState.STA ThreadPriority.AboveNormal

            // Opens my form in the new thread
            ShowForm title (float32 width) (float32 height) background ct toui fromui sharedResources

            return ()
        }

IMO async/await is an opaque, complex, limited and leaky abstraction. async/await does bring value but so did the discredited COM Apartments.

I would have preferred if C# had computations expressions like F# and async/await implemented using those. In addition computation expressions are generic which means they can be used for wide range of problems, see websharper formlets for an amazing example. Being generic it also means that if I for some reason don’t like async in F# I can implement my own async abstraction on top of computation expressions, with async/await it is take it or leave it.

That is why I wish C# never got async/await and instead would have gotten something like F# computation expressions.

23 comments

  1. tomas

    Hi, nice post. It is not so bad with C#, fortunately there exist Rx framework ;) And I would say it was quite wasted effort wirh async/await when there was already Rx. I can say it from my own experiences – no async/await, all by Rx ;)
    Cheers

  2. Anton (@PieCalculus)

    Could you please give a credit for that chart to @brendangregg? Not exactly my work, just a quote that went viral. ;) (from “Systems Performance: Enterprise and the Cloud” book). Thanks!

  3. vincpa

    At a language level, async/await does not create new threads. When the SynchronizationContext is unavailable (console apps), it will use the DefaultScheduler which will choose to run work on a different thread and then marshal that result back to the original context. But that is decided by which scheduler is used, not the async/await keywords.

  4. Pingback: Comparing async/await and async workflows in F# | Mårten Rånge - Meta Programmer
  5. Pingback: DIY asynchronous workflows in F# | Mårten Rånge - Meta Programmer
  6. Brian

    The documentation is correct if subtle.

    >> Async methods don’t require multithreading because an async method doesn’t run on its own thread. The method runs on the current synchronization context and uses time on the thread only when the method is active.

    An async method doesn’t run on its own thread, it runs on a thread owned by the synchronization context. The synchronization context may own multiple threads, and may run your code on different threads. However the synchronization context does as the name suggests, it manages synchronization such that race conditions aren’t a problem with async\await.

    More to the point F# and C# use the same machinery to run async code, so if you find issues with that plumbing in C# then be aware the same issues exist in F# as well.

  7. Pingback: Some words about the .NET “async hype” | Marcel Beck´s Blog
  8. fschwiet

    Can you expand on or provide a source showing how the readingFiles race condition happens? To me it seems the increment must complete before the decrement is done.

  9. Josip Bakić

    Interesting post! I’ve been “instinctively” avoiding async/await simply because I like my code to clearly state what it is doing, particularly when it comes to threading, so I fully agree with you – this abstraction is opaque and quite leaky, the worst combination.

    How do you feel about STM? I wrote an STM library for .NET, if you’re interested to give it a look: https://github.com/jbakic/Shielded . It doesn’t do any thread creation, leaving that as a choice to the user of the lib, whatever method they prefer. (It can block, of course, but it cannot deadlock.) Wrapping its transactions in a custom Task should be easy. Even the more complex things like the conditionals will all execute on whichever thread triggers them, forcing the user of the lib to spin a thread/task manually if that is desired.

    • mrange

      I haven’t looked into STM that much but from my trivial viewpoint they are intended for a different problem ie the “consistency” probolem. async/await is intended to solve the “responsiveness” problem. Then there’s the “scalability” problem ie Parallel.For. I should look into to how STM works, it’s a missing piece in my knowledge. Anyway, I do think async/await is useful but it requires understanding how it interacts with the SynchronizationContext to use safely. MS does it somewhat a disservice of selling it as “It just works” when it doesn’t thus scaring devs. It would be better IMHO if SynchronizationContext wouldn’t be used by async/await but now as it is what’s needed is clear instructions on how to use it safely.

  10. Jonathan

    It seems to me that the only difference between F#’s Async and Node.js’s callbacks is the syntax transformation, which I thought was really the point behind async/await in the first place. Is the issue the additional complexity introduced by SynchronizationContext? How does F# get around this complexity?

    • mrange

      I am the opinion that the big problem is that async/await uses SynchronizationContext “invisible” to run the continuations. This makes the code behavior not well-defined unless you know the SynchhronizationContext (which is “invisible”). If async/await always ran the continuation with a well-defined semantic lot of problems would disappear.

      • mrange

        F# async AFAIK doesn’t rely on SynchronizationContext making F# async more predictable. However it’s not completely without problems as often you await on tasks and then async implicitly uses SynchronizationContext and you get the same problem. That’s why I in a later post demonstrated how to build your own async in F# that has a more defined continuation semantic.,

  11. nightwatch77

    OS thread scheduler is quite efficient because it’s a core piece of the OS, has been optimized for many years and has hardware support in the CPU. And the async framework in .Net is more or less another thread scheduler that runs in user space inside the CLR, confined to single process only. I can’t really find any advantage of .Net soft scheduler vs the OS – did anyone do a comparison?

    • mrange

      There are benefits with event driven programming (ie async/await) over threads. If you have to listen to 10,000 connections one of the problems is that each OS thread allocates alot of task space (around 1Meg). There are other issues. A general guideline I heard is that you roughly like sqrt (connections) threads servicing 10K connection thus roughly 100 threads handling 100 connections each. http://www.webcitation.org/6ICibHuyd

  12. Pingback: Professional Development – 2014 – Week 45
  13. Sarfaraz Jamal

    Hi! I finally read this post – a colleague had forwarded this to me a while back. Thanks I may need to revisit it again and I look forward to seeing more things about F# – I have only heard of it through posts such as this and it sounds really mysterious [in a black box, good kinda way]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s