56 comments

[ 4.5 ms ] story [ 135 ms ] thread
Genuine question...

We have way over a million lines of c#, in asp.net, mvc and windows forms. We get 80,000,000 http requests a day.

We have no async (delegates or 4.5 stuff), no threading other than what WCF, AppFabric and ASP.Net give us.

About 25% is generic CRUD code but the rest is complicated matching, integration and math code. We also touch most fundamental computer science domains.

This begs the question: you really need async in the language?

It's about concurrency, not performance. What's the 95th percentile response time (backend only)?

Take out all that complicated code, and replace it with a call to an external 3rd party API. They get a hiccup, your response time goes up by 500%, your cpu load drops, and you start timing out connections.

C10M http://www.youtube.com/watch?v=D09jdbS6oSI

We use appfabric and workflow to handle all 3rd party integration. Nothing blocks threads. But we don't use async language features to do that. Workflow could be considered async but it's a tenuous link.

95% is 41ms.

async doesn't really give you anything new you couldn't do before, just nicer syntax for composing it all. As you say you've already found other ways to compose your asynchronous flows for you.
"WCF, AppFabric and ASP.Net" happen to give you an awful lot of that sort of infrastructure for free, I rarely directly use anything async in my web apps either. I do wonder how you're managing responsiveness in the forms apps though?

Where the async syntax really shines is dealing with external services. If you're calling a lot of database, web services, sockets etc it can make your life easier.

To be honest our webforms app is tiny and only does basic integration tasks between legacy software and our web app so there are no points it can block.

We do everything synchronously with respect to databases. Integration is all docoupled into workflows.

You raise a question, but there's a missing detail in your input - how many machines (and what kind) you are using.

If you have 80,000,000 machines serving 80,000,000 requests - that's not very impressive. Even if you have 80 servers, that's extremely unimpressive.

If you can do all of that on 1 server, then YOU probably don't need async. In fact, unless you're wasting time, 80,000,000 averages less than 10 requests/second, and peak is likely less than 100 requests/second. That's really not much - I do that with interpreted python.

But every language does need facility for async processing.

edit: it's 1000/second, not 10/second, as everyone down here corrected. Apparently, I cannot math before coffee. (I don't drink coffee). Apologies.

fun mnemonic: the number of seconds in a day is 864e2. (eight-six-four-(e)-two).

> In fact, unless you're wasting time, 80,000,000 averages less than 10 requests/second...

Please show your math.

I took 80000000 / (24 * 60 * 60) and I got 925.

So you seem off by about two orders of magnitude.

Please show your math.

I took 80,000,000/86400 and I got 926.

So you seem to be off by about 1.

I used bc. It's probably an issue of floor vs round....
$ echo "scale=7; 80000000 / (24 * 60 * 60)" | bc

925.9259259

Your math is off.

It's on a bell curve. We're regionally tied so we have 80,000,000 requests, 90% of which are over a 5 hour period so we can hit 5000 requests a second on a fun day :)

We do this on 10 servers. 4 front end, 4 back end, 2 database nodes. They are all hefty machines (FE=8xXeon,16Gb;BE=16xXeon,32Gb,DB=48xXeon,96Gb;Storage=12TiB SAN).

Our peak cluster load is about 20% (the sweet spot).

We don't need async, even if we lost half of our nodes.

>This begs the question: you really need async in the language?

Or even delegates. Or interfaces. Or LINQ.

Basically you could write the same thing in C.

Or assembly.

Strawman.

The parent isn't arguing that the new async constructs in the language don't make asynchronous programming easier.

He's questioning the need for using asynchronous programming models at all, for the common use case of ASP.NET applications.

>The parent isn't arguing that the new async constructs in the language don't make asynchronous programming easier. He's questioning the need for using asynchronous programming models at all, for the common use case of ASP.NET applications.

And, my comment questions this very idea that for language constructs to be added there has to be a "need".

They is never a hard need. We can do everything with assembly.

Language constructs are added to make things easier.

And that includes ASP.NET applications. His might not have a use for patterns made easier by async, but others do, especially those doing real time websockety stuff.

I think async is a systems programming tool rather than an application programming tool. If you're writing a web server or a web browser, it'd be useful. But for a web application, you're already sitting on a highly scalable and robust async library called IIS so when you need a background task the easiest thing to do is to make it a RESTful API call. I find many web developers now intuitively break up their application into parallelizable chunks without ever really thinking about async or threads, they just just say "this web page is taking a long time to load, maybe I can defer some of the work until after the page load" or (as you seem to have done) "maybe I can use tools like AppFabric Workflow to orchestrate all these small tasks".
The problem is that it's pretty useless in non-systems programming language.
It is pretty useful for ui stuff, you can start a wait cursor in a button event, do what you need to then put the cursor back when your done, all in one thread without having to use background threads or anything.

It is pretty simple for that scenario and avoids the cross thread marshalling you need to do in win forms.

Don't forget the SOA scenario: One service calls 0..n others to fulfill a request. Each of those calls is IO bound.
When you say that a background task should be made into a RESTful API call, I assume you mean that instead of spinning up a local thread to do the work you should defer it to another service. Certainly you can do that, but async/await is still relevant in that case.

The key question is: while said REST call is happening, what is the caller doing? Normally, he's sitting there and blocking his thread until the call finishes (i.e. consuming one thread from the connection thread pool). With async/await, the thread is released back into the thread pool until the result is ready, then it picks up where it left off.

So anything where you might want to deal with background operations that take a long time (including network / db / filesystem access, i.e. most applications that aren't pure functions) in a connection-oriented system can greatly benefit from async/await. Even if it's just a regular desktop app, it appears to be a useful mechanism for coordinating different background tasks.

Your point is a good one. Without wanting to get too deep into a design for some hypothetical system, I guess I was thinking that I'd either try and modify the current request such that it can return something to its caller (e.g. Render a web page saying "Loading..." with a setTimeout call in it). Or use .NET's HTTP request library which has a delegate callback option that lets you get on with other stuff.
One benefit to something like async/await is letting you get on with other stuff without having to murk up your code with callbacks/delegates which read more like GOTOs than async/await.
The problem with delegate/callback patterns with API calls is that you end up having to preserve a lot of state manually, which can be an error-prone process. Any metadata associated with a request must be persisted (and then read back) in painstakingly manual fashion with a delegate pattern, whereas with async (or closures, or...) this information is trivially available on your stack.

The nice thing about async vs. typical closures is that it neatly avoids a lot of nested scopes.

I was going to say the same thing. As a specific example I am currently working on a web application with multiple, complex views requiring several dropdowns. We noticed a dramatic speedup in these views when we refactored the controller to retrieve data for dropdowns using async/await.
If you're not blocking threads, either explicitly on I/O, or on perceptibly lengthy CPU operations[1], and you don't have a high proportion of kernel CPU% that can be put down to the cost of thread switching, then you personally don't really need async.

But it's a big leap to go from that to saying that nobody else needs async, so it need not go in the language.

[1] Taking the calculation off the main thread isn't the only way async can help here. The primitives also give you a way for taking better advantage of multiple cores (futures) more easily.

Maybe you don't need concurrency, but I certainly have in almost all my jobs thus far. async/await is pretty great for this, especially when you need to mix concurrency and UI. Making it a language feature (instead of a library) makes the concurrent operations a lot more readable by avoiding a lot of nesting.
> This begs the question: you really need async in the language?

I too have applications that do a lot of stuff relying of asynchronous toolsets but without much explicit asynchronicity. However, I did a simple experiment a couple weeks back - I coded one of our simple but very asynchronous programs in Go. Compared to the Python original (which uses Twisted), the Go version had 70% of the lines and was much more readable and understandable by even team mates who had never seen Go.

I believe that's the point of having async operation elegantly merged into the language.

Need? Of course not. It's always been possible to do the same thing "by hand". But you certainly do need to not have all your threads blocking while external calls happen (although from elsewhere in the thread it sounds like your peak is 500 requests/second/machine? That's not so many, maybe you can afford to make blocking DB calls in the request path), one way or another.

A better question is how many lines of code the async feature saves, compared to the code you would write without it.

There millions if not billions of lines of c code in production today doing everything from web servers/cgi web pages to low level embedded oses. A shit ton of it is not using oo programming: do we really need it?

It's about doing more, better, and easier. async allows a pretty complicated process to be much easier to write.

I didn't really get it until I watched Ander's original keynote on it.

To appreciate async and await, you have to go through the exercise of converting some synchronous code to an event driven model with callbacks. Then do it using async/await. It's more syntactic sugar than anything.

I appreciate it in how I appreciate the dynamic keyword. I know how to dynamically Invoke() methods at runtime using reflection. It's a lot of boilerplate and ceremony, and the dynamic keyword helps alleviate that.

Just fyi, you're raising the question, not begging it.
Despite the desires of pedantic prescriptivists obsessed with maintaining the "purity" of a poor translation into English of the Latin name of a logical fallacy as the sole use of the phrase "Begs the question" in English (a use which is always intransitive), the always-transitive (and thus, impossible to confuse with the fallacy name) use of "begs the question" in a way which makes more sense with the normal English definition of the words in the phrase is in widespread use, is well-understood, and actually provides an avenue for the intransitive fallacy-naming use to make some sense (as the intransitive use can be viewed as a special case of the transitive form where the object -- the question a call for the answer is made -- is the same question that the original proposition was attempting to answer.)

So, the pedantry on this point is pointless and counterproductive.

In some scenarios it simplifies the code greatly. It may or may not be applicable to your app.

Just a few weeks ago I was able to convert a nightmare-ish recursive asynchronous method to a `foreach` loop with `await`s inside.

It's cool when you can do this:

    var providerExceptions = new List<Exception> ();

    // Try each provider in turn
    foreach (var pi in providers) {
        token.ThrowIfCancellationRequested ();

        try {
            return await GetSession (provider, isLast, options, token);
        } catch (TaskCanceledException) {
            throw;
        } catch (Exception ex) {
            providerExceptions.Add (ex);
            // Fall back to next provider
        }
    }

    // Neither provider worked
    throw new AggregateException ("Could not obtain session via either provider", providerExceptions);

Or this:

    async Task<Session> GetSession (AccountProvider provider, bool isLast, LoginOptions options, CancellationToken token)
    {
        if (!SessionManager.NetworkMonitor.IsNetworkAvailable)
            throw new OfflineException ();

        var account = await GetAccount (provider, !isLast, options);
        if (account == null)
            throw new Exception ("The user chose to skip this provider.");

        var service = provider.Service;
        var session = new Session (service, account);

        if (service.SupportsVerification) {
            // For services that support verification, do it now
            try {
                await service.VerifyAsync (account, token);
            } catch (TaskCanceledException) {
                throw;
            } catch (Exception ex) {
                throw new InvalidOperationException ("Account verification failed.", ex);
            }
        }

        return session;
    }
Depending on the conditions, the method may or may not “freeze”, but the calling code doesn't care.
In your first code sample, I'm pretty sure you don't need the return await GetSession (provider, isLast, options, token);

Unless there's more to the method, just remove the async modifier and directly return the task returned by GetSession.

await will unwrap exceptions from the task object
Exactly, and thus I'll be able to try next provider in the loop.
If I did that, the code would always return at the first GetSession call.

Instead, it unwraps exceptions, and while it propagates cancelations, it ignores other exceptions and tries other providers in turn.

In fact, I could even use `break` or `continue` in the midst of async code, and there would be no problem.

Also I just love using `try` and `finally` in async code—with callbacks, you have to do finalization from all error and success paths (which can be a lot of places).

First note: Async should NOT be in the language. Support for such constructs should be in the language, and then libraries should add features such as async (like F# has done for years).

Second note: If your app has lots of short-ish lived requests, you can just achieve parallelism by having a decent-sized threadpool and just run each request on a thread. Who cares if you have 400 threads? It'll scale well enough (as you noticed).

However, if the app is doing long-running requests/clients, and you dedicate a thread to each one, then you end up with a lot of extra overhead. Using C#'s async model can simplify the code while keeping things lightweight.

Heuristically, anytime you're doing I/O (network or filesystem) could be an opportunity to increase concurrency with async/await. By using asynchrnous I/O and trickling that asynchronocity all the way up to ASP.NET, you make much more efficient use of the ThreadPool and allow more requests to be served. That's because you are recouping cycles otherwise lost on blocking.

This isn't the only way to increase concurrency. Obviously you can grow your IIS web farm as well. But async/await (or some other .NET asynchronous programming pattern[1]) is a one-time .NET development cost as opposed to a recurring infrastructure/hosting cost.

[1] http://msdn.microsoft.com/en-us/library/jj152938.aspx

The problem that I keep seeing is people think that TPL or Async/Await heck, even Rx is a Threading library....

As a result people still use monitors or any kind of blocking wait event. Including the TaskCompletetionSource in the framework I don't think helped this.

A lot of what I've seen on different projects using these newer tools is an attempt to re-create what they would do before. Doing a Producer Consumer, that's a blocking de-queue etc.

What's wrong with TaskCompletionSource? I use it constantly. It's great for translating other async styles (like passing a callback as the last argument) into tasks.

How would you implement Observable.LastAsync, without using something equivalent to TaskCompletionSource along the way?

I think your parent is arguing that, despite TaskCompletionSource exists, some people still don't use it much and resort to callbacks (presumably because they have not learned about it).

As in “adding TaskCompletionSource to the framework didn't help some folks adjusting to Task style, and they keep using callbacks”.

That's pretty bold statement. And it's not true.
"Everything .NET programmers know about Asynchronous Programming in ASP.NET is wrong" would perhaps be a better title.
I would've gone with, "Over 14 Month Old Podcast Makes Overly Broad Claims to Linkbait" as a headline.
After listening, I don't think they mention Task.Factory.StartNew, this is how I usually throw things into the background. Whats wrong with this approach?
Nothing per se, but the TPL uses the thread pool as well.
The single biggest problem I have with async/await in C# is how it requires an act of god to call an async(await) method from a non-async method. And even then, wrapping exceptions and such is a huge pain as well.

I would love async if it was completely optional and I could call async code from a synchronous context. Like if I could just say `var tmp=await Foo();` within a synchronous method (and it just ran Foo on the current thread and blocked)..

Instead, it requires one of the methods outlined here: http://stackoverflow.com/questions/5095183/how-would-i-run-a...

IMHO the C# team made the right decision. What you're suggesting is a very leaking abstraction. Running stuff by default on the current thread is a sure recipe for deadlocks.
So why don't you just define a non async method (A) and then one that is async (B) that calls A? If you want to run async then call B if not then call A.

If you could define a method as async and then call it either syncronously or not the whole point of the async keyword is lost. You would end up with the compiler doing a load of work because every developer will just slap async on every method just in case rather than actually thinking about what needs to be async and what doesn't.

Adding the async keyword is not a magic bullet, there's a huge amount of work going on under the hood every time you use it and it's important that developers appreciate that.

The problem with your two-method suggestion is that it's usually not /me/ I'm concerned about. The problem is usually things like "I need to open this file from a synchronous context in a Windows Store App". The problem is that with Windows Store (where async is suppose to shine), async is the rule, not the exception

Also, two methods sharing the same code sounds absolutely horrible.

Method B calls A it doesn't share any code and it is then explicit that one method is async and the other is not. You want to do two different things so why not have two methods?