Speaking of futures_unordered and similar patterns, I think a part of the "async promise" that has failed is the lack of concurrency for a single user request by default in most languages.
That is, the 'easy' path is to write code such as the following (in vaguely C# pseudocode):
var p = await GetUserPermission( username );
var c = await GetServerConfig();
var m = await GetMessageOfTheDay();
Assume each await call is potentially an expensive SQL query or REST API call.
The problem with that is that this is strictly sequential, synchronous code that is merely "dehydrated" and "rehydrated" to reduce overheads during the waiting periods. It is strictly slower when executed on a server that is not very busy! It must be, because it does the exact same work in the exact same order as the ordinary synchronous version, except now with extra state machinery and complex error handling woven throughout by the compiler.
Scalability is not everyone's concern. Scalability is for the FAANG sized companies. I care about the individual user experience, and async does nothing for that by default.
I mean, sure, you can write much more verbose code along the lines of:
var p_t = GetUserPermission(username);
var c_t = GetServerConfig();
var m_t = GetMessageOfTheDay();
await Task.WhenAll(new Task[] { p_t, c_t, m_t });
var p = p_t.Result;
var c = c_t.Result;
var m = m_t.Result;
But noone does this, for some values of noone. I've never seen code like this in the field.
In fact, let's test this. I'm reviewing an asynchronous ASP.NET application developed in 2020 right now. It's a large app, with literally thousands of uses of the "await" keyword, at least 3500 files use it.
The only use of "Task" static methods are seven uses of FromResult(). That's it. Zero uses of WaitAll(), WaitAny(), or ContinueWith()!
This is typical.
It's not that asynchronous programming is hard, it's that it is unergonomic to gain a latency benefit out of it. Most applications need lower latency, not higher throughput. Hence, for most programmers, most of the time, asynchronous programming is next to useless. It's just extra noise and more failure modes.
That .NET syntax using Task.WhenAll seems quite bad, which might be part of the reason why not many people bother (disclaimer: I don't do C# or ASP.NET). In Rust it would be:
let p, c, m = join!(
GetUserPermission(username),
GetServerConfig(),
GetMessageOftheDay()
);
(you don't even have to write await when using the join macro)
With such simple syntax available it seems obvious to me that one would want to use it as often as possible, and it's also much simpler (and probably cheaper) than dispatching those three tasks to a thread pool.
The original example by me did not assume that asynchronous functions all return the same result type.
Mist opportunities for concurrency are between unrelated tasks (because related tasks often dependencies between them). Unrelated tasks tend to have unrelated return types.
When tasks are unrelated, you also likely don't need them all at the same time for the next stage of pipeline. You can simply await it when you need it.
var p_t = GetUserPermission(username);
var c_t = GetServerConfig();
var m_t = GetMessageOfTheDay();
function_to_call1(await p_t, await c_t);
function_to_call2(await m_t);
This does not look any more complicated than a non-async function. Not sure how this example justifies your claims.
Besides, even if your example is valid, the usage of Task.WhenAll has nothing to do with your claims either. The use of async/await is majorly for scalability. Being able to make several network calls concurrently is not the major concern. Even if you await at each async call, you still achieve better scalability because threads won't be blocked for async calls and can work on something else.
I guess I am that no one. I come at all this from writing queues from scratch and using threads or processes for concurrency. I also had a lot of fun writing my own networking hot loops with select/poll/spill/kqueue when my work needed it, so I guess I am extra sensitive to making concurrent things actually concurrent. But I would not dream of making three independent requests like that sequentially. There are other patterns you can use besides waiting for all tasks to finish, especially if you can do some processing after the first are done, but all in all why wouldn’t you make them concurrent aside from liking seeing await/async all over the place?
All you have to do is wrap multiple futures into a single one and then await on the combined one. There is no programming language on earth that can prevent this.
My teams/company uses it all over, so maybe depends on the context you work in?
And FWIW, this explicit form is often unnecessary - if you kick off each task they will run in parallel and then just await each task only when the result is needed, it can look a lot cleaner:
var p_t = GetUserPermission(username);
var c_t = GetServerConfig();
var m_t = GetMessageOfTheDay();
var foo = isAuthorized(await p_t);
// more code here
var msg = ( (await c_t).ServerName + await m_t) );
True, this doesn't work in Rust though, because nothing at all happens before the first time you poll a future, so you need an explicit task (but as other pointed out, it's pretty straightforward thanks to the `join!` macro).
I write this kind of stuff all the time because parallelizing long-running tasks without dependencies is one of the easiest wins when it comes to wall-time.
But this kind of optimization is somewhat orthogonal to async/await. You don't need fine-grained async to optimize long-running tasks, you could just throw a bunch of closures into a threadpool for that purpose. Async only makes sense when you're interleaving thousands of tasks with readiness/completion based IO.
It's rare that I personally write these kind of optmisations in web apps, but quite often do for backend processing services (and then, only for "embarrassingly async" operations such as hitting a database or HTTP API).
You don't need to create a Task[] because WhenAll is set up for varargs. This is fine:
await Task.WhenAll(p_t, c_t, m_t);
Or you can just await the threads before you need them. They're already started and running at this point.
You also probably want to avoid using Result and just await the completed task for the nicer unwrap syntax. Plus, you don't want to get into the habit of using Result as its a blocking call. Same with WaitAll and WaitAny. Ideally you would never use those. ContinueWith is also not very needed if your style is to use the more plain await syntax. Those methods are more to bridge blocking and async code so an async from the start app might use async extensively and never those methods.
I have written a few programs like this - but not in languages which have async/await! In languages with manual async, getting here by refactoring is fairly easy.
I've been using C# for around 20 years, basically since it was first released.
I never personally had any issue with working with threads and locks, finding it simple enough to reason about them, though I understand lots of people felt differently. When async/await first came to C# around 10 years ago, I grumbled because I didn't see the point; I found it much harder to reason about the flow of code, and initially at least, stack traces were a shitshow (things are much improved, but there is still a lot of cruft in async stack traces).
But async/await was heavily pushed, and "real" threading is almost relegated to the sidelines for most developers. Although having said that, I find that junior devs in particular really struggle to really grok async/await.
Anyway, several more years on, and I have mixed feelings about async. Because Microsoft has gone all-in on async/await, I think it's really easy to work with when building web apps and APIs with ASP.NET Core/MVC - there is barely any "developer overhead" at all, really. Web apps very often hit things like HTTP APIs and databases, and with how easy it now is, there is little reason not to use async/await. Yes, for small loads there is a tiny performance loss due to the runtime setting up async state machines, but it really is almost always completely insignificant - even moreso with the advent of ValueTask, and again more recently with pooled ValueTasks. Yet the gains can be tremendous.
But for non-web apps/APIs, I feel differently. I spend a lot of time writing server-side processing services, and things like Windows services for desktops (in the infosec space), and I've gone all-in on async/await because Microsoft has gone async-first. Hell, a lot of stuff is async only now, so unless you want `.GetAwaiter().GetResult()` everywhere, you have little choice. Anyway, these systems are more complex than web apps, because with web apps, most of the real complexity is hidden away in the framework. But here you have to deal with work queues, caching, pooling, serialisation etc all by yourself. And with async/await, it can be hard to reason about the flow of code, and it's really easy to break things in ways that are really painful to diagnose. And it means that every.single.stacktrace contains async cruft that you need to sift through. Which is not fun.
Anyway, this is much longer than I meant, but my conclusion is that I'll continue to use async/await for web apps and REST APIs (because, why not), but for services, I'm going back to the threadpool, green threads and synchronization primitives, and only using async/await in a limited way where it provides clear value - not async all the way down from the entrypoint.
> Anyway, this is much longer than I meant, but my conclusion is that I'll continue to use async/await for web apps and REST APIs (because, why not), but for services, I'm going back to the threadpool, green threads and synchronization primitives, and only using async/await in a limited way where it provides clear value - not async all the way down from the entrypoint.
AFAIK, .Net doesn't support "green threads" and they repeatedly confirmed that there are no future plans to do so. Additionally, M:N threading model has serious interop issues as evident in Go, which is a no-go for system languages. Personally, I don't see a need for green threads since kernel threads are fast enough and don't use that much RAM as people tend to believe. And when they are not, sure, go async/await.
I find your comment about stack traces a bit weird: of course, when all your work is sequential and you can use only threads, you will have a nice stack trace for free, when async stack traces need a lot of support from the tooling.
But most of the time you not only use thread, but also several synchronization primitives (locks, channel, etc.) and when doing so, regarding stack trace you are in an even worst situation than what async stack traces gives you (“some thread changed this shared-memory value and now it's not what you expected, but you have no easy way to know which one did and when, good luck”).
Maybe if you spray threads around at random :), but in real-world use I find it much easier to pinpoint where the problem occurred, and the path taken to get there. Also, at least with threads you can get the thread ID and/or name.
Regarding shared, mutable state - if multiple async "threads" can access that state, then you still need to guard it, but usually with an async-capable means.
> Regarding shared, mutable state - if multiple async "threads" can access that state, then you still need to guard it, but usually with an async-capable means.
Sometimes, but not as often, because the scope of your async function is often the only “shared state” you need.
async/await is for the concurrent stuff and threads are for the parallel stuff. Two different things. If your code is I/O-bound, use async/await. If your code is processor-bound thing, use threads.
Async/await paradigms exist in several languages, but with C#, async/await is generally considered the "modern" and unified way to handle both IO bound and CPU bound tasks.
The runtime will generally schedule IO bound tasks to run on the threadpool.
> The runtime will generally schedule IO bound tasks to run on the threadpool.
Well, that's not correct. Unless you explicitly call Task.Run or Task.Start (or other similar methods) no new thread is created. The compiler generated state machines don't require the threading mechanism to work. In fact the overhead for async/await is mostly the extra code generated for the state machine and error handling. At runtime, there's no thread switching overhead.
Yes, I meant using Task.Run; I was simplifying, as I'd assumed (wrongly) you were familiar with async/await from another language.
Otherwise, from memory, the runtime spec doesn't actually guarantee that await won't run on a threadpool thread - it will under certain circumstances.
And then there are further nuances if there is a synchronisation context and ConfigureAwait(false) is used, as the continuation will be scheduled on a threadpool thread.
That is, the 'easy' path is to write code such as the following (in vaguely C# pseudocode):
Assume each await call is potentially an expensive SQL query or REST API call.The problem with that is that this is strictly sequential, synchronous code that is merely "dehydrated" and "rehydrated" to reduce overheads during the waiting periods. It is strictly slower when executed on a server that is not very busy! It must be, because it does the exact same work in the exact same order as the ordinary synchronous version, except now with extra state machinery and complex error handling woven throughout by the compiler.
Scalability is not everyone's concern. Scalability is for the FAANG sized companies. I care about the individual user experience, and async does nothing for that by default.
I mean, sure, you can write much more verbose code along the lines of:
But noone does this, for some values of noone. I've never seen code like this in the field.In fact, let's test this. I'm reviewing an asynchronous ASP.NET application developed in 2020 right now. It's a large app, with literally thousands of uses of the "await" keyword, at least 3500 files use it.
The only use of "Task" static methods are seven uses of FromResult(). That's it. Zero uses of WaitAll(), WaitAny(), or ContinueWith()!
This is typical.
It's not that asynchronous programming is hard, it's that it is unergonomic to gain a latency benefit out of it. Most applications need lower latency, not higher throughput. Hence, for most programmers, most of the time, asynchronous programming is next to useless. It's just extra noise and more failure modes.