User-level threads....... with threads. - Paul Turner - Google

Oh OK, I definitely could have written that better. From what I can tell, M:N threading has been thoroughly abandoned by Linux, but it might be worth revisiting given the horrendous I/O scaling of Linux kernel threads. I would imagine scheduling user threads cooperatively simplifies things a lot. Google has a very interesting talk about cooperative native thread scheduling, but they haven't upstreamed their code: [this video] And here's what linux devs are actually doing (it's underwhelming, something like 50% of what the hardware is capable of): [link] In the meantime, people who actually need the performance (HPC and even some enterprise servers) have to bypass the kernel entirely. It looks like that's going to be possible even in virtualized environments, thanks to hardware support: [link]

Google needs to get off its ass and open source Fibers (not to be confused with Google Fiber). Fibers sparked plenty of joy in me back when I was at Google. It's just a more dignified way of doing things with threads, IMO. Here's the talk which, unfortunately, does not show the higher level APIs I'm talking about: [this video] . If this were released today everyone would slap their foreheads and say "of course that's how it should be done". Google folks have already removed the obstacle in that direction by adding userspace switching to the kernel in recent kernel releases. Just go all the way now, and release the kraken. Last update I've seen on this is this article from June of this year: [link] Imagine how excruciating it must have been to all the Google people on the C++ Standard committee to work on this stuff that comes nowhere close to what they've been using internally for years, and not be able to talk about it in any kind of detail.

I am fully aware of the approach, especially in languages/systems like Erlang, and the freedom that very cheap threads give you. My first point was that you actually have much more of this freedom than most people are aware of, even with (comparatively) heavy kernel threads. For example, see the "Replace user threads with ... threads" by Paul Turned (Plumber's Conference): [this video] More on that point, I see a lot of user-threading/async craziness on clients such as iOS that would have been easily been handled by less than a dozen kernel threads, most of which would be sleeping most of the time anyhow. That's a number of kernel threads that is easily manageable and not particularly resource intensive. My second point is that there is one thread that this blocking-happy approach mostly doesn't apply to, and that is the UI thread. You really don't want that UI waiting for (network) I/O and therefore must employ some sort of asynchronous mechanism for data flow and/or notifications.

> Can we identify why exactly 1:1 threading is uncompetitive with async/await schemes, and fix that? I think we can identify it, but fixing it is not easy, at least at the kernel level. The easy part is the cost of scheduling. The kernel must schedule threads with very different behaviors -- say, encoding a video or serving requests over a socket -- so it uses a scheduling algorithm that's a compromise for all uses. But we can let the language's runtime take the scheduling of kernel threads with something like this: [this video] The harder part is managing the memory required for the stack. Even on 64-bit systems, the OS can't shrink and grow stacks finely enough. For example, I don't think there's a hard requirement that, say, memory below the sp is never accessed, so the OS can't even be sure about how much of the stack is used and can't uncommit pages. But even if there were such a requirement, or, that the language could tell the OS it follows such a requirement, still the OS can only manage memory at a page granularity, which is too much for lightweight concurrency. Any finer than that requires knowing about all pointers into the stack. We can do it in languages that track the location of all pointers in all frames, though, which is what we're attempting to do in Java, and this allows us to move stacks around, grow them and shrink them as necessary, even at a word granularity. > All this, by the way, is not to say that Rust made the wrong decision in focusing on async/await.

Although you are technically correct in that the cost of system calls on most systems is high, your post also implies to a casual reader that the cost is fundamental. This is not true. As demonstrated in the benchmark, the hardware cost of systems calls/trapping is quite low. Even accounting for hardware workarounds (which were not even known about when the common consensus decided system calls were slow), excepting the case of hardware virtualization which has much more expensive mitigations, the fundamental cost is a fraction of the actual cost. The majority of the cost (~90+%) is actually due to software. In the system call case this is the implementation of the system call. In the context switch case this is usually the cost of the scheduling decision. These operations take material amounts of time and access memory which evicts user caches. It is in speeding up the software component that real gains can be made. This is not to say that system calls could ever be made "free" since function calls/loads/stores will still be ~100x faster, but the cost would probably be ~10% of the current cost if they software component could magically be made instant. As an example of what I am talking about, there is this talk by Google from 2013 [1] which describes a "fast context switch" system call (benchmark at 18:40). This "context switch" knows what it is switching to which lets it bypass the scheduling decision in normal context switches. As a result it takes ~170 ns (which is prett

I don't think "fiber" has a universally agreed definition, which makes these things super confusing to read without clarification. It's hard to come up with a good definition and stick to it because people keep coming up with new variations. An interesting case is a different Google fiber library, not open-sourced but described in a public talk. [1] Real kernel threads but (mostly) scheduled in userspace via new syscalls [2] that suspend the current thread and unsuspend a userspace-chosen other thread. The idea is that most of the overhead of kernel threads is the scheduling and userspace has the ability to make a quick good choice. [3] Thus you can get most of the CPU performance of an async implementation but the relatively easy-to-read callback-free code of a threaded implementation. Anyway, people can disagree on what to call such a combination. [1] [this video] [2] the rseq stuff described in that talk has been upstreamed [4] but the switchto stuff still hasn't. [3] I'm not sure if the performance argument is as strong since the Meltdown/Spectre mitigations added to the performance cost of any syscall. [4] [link]

I think it's also worth mentioning that for certain specific use cases coroutines are much more efficient than a full goroutine, because switching to/from a coroutine doesn't require context switching or rescheduling anything. If you have two cooperating tasks that are logically synchronous anyway (e.g. an iterator) it's much more efficient to just run everything on the same CPU because the kernel doesn't have to reschedule anything or power down/wake up CPU cores, and the data is in the right CPU caches, so you'll get better cache latency and hit rates. With goroutines this may happen anyway, but it's not guaranteed and at the minimum you have the overhead of going through the Go runtime goroutine scheduler which is fast, but not as fast as just executing a different code context in the same goroutine. Coroutines offer more predictable scheduling behavior because you know that task A is switching directly to task B, whereas otherwise the goroutine scheduler could switch to another available task that wants to run. The last section of the blog post goes into this, where Russ shows that an optimized runtime implementation of coroutines is 10x faster than emulating them with goroutines. Google has internal kernel patches that implement cooperating multithreading this way (internally they're called fibers), and the patches exist for precisely this reason: better latency and more predictable scheduling behavior. Paul Turner gave a presentation about this at LPC ~10 years ago that