Does M:N threading model really utilizes CPU cores?

Question

There are several threading models that are used for scheduling threads within applications:

• 1:1 (kernel level threading): Each thread created by a user is mapped to a scheduled thread in the kernel.
• N:1 (user level threading): All the threads that are created by a user in a single application are actually mapped to a single scheduled kernel thread.
• M:N (hybrid threading): M threads created by a user in an application are mapped to N kernel threads.

User-level threads are considered faster than kernel-level threads, because context switching at the kernel level is more expensive than in user-level. One big disadvantage of user-level threads is that they don't utilize multiprocessor systems as they only use one kernel-level thread.

There are some articles telling that M:N threading model is best used with N as the number of CPU cores (here is an example). In this way we can achieve the advantage of both 1:1 and N:1 threading models.

My questions are:

1. When we use kernel-level threads we also get 'extra' time slice during execution (as opposed to user-level threads), so doesn't it make up for the slow context-switch?
2. Why is the number of CPU cores even relevant here? As I understand, the number of CPU cores is pretty transparent here, because even when we use the exact amount kernel-threads, nothing guarantee us that they are really executed simultaneously, because other cores can execute other threads from other processes, and 'our' threads might still use context switch afterwards. So they use context switch regardless of the how many CPU cores we have. Am I wrong here?.

Answer 1

User-level threads are considered faster than kernel-level threads

Both can be faster depending on workload, OS, hardware, and green threads implementation.

doesn't it make up for the slow context-switch?

Sometimes. Usually no. Kernel threads have a stack, when you have thousands of them they consume gigabytes of RAM, and when they context switch you're guaranteed to have plenty of cache misses. That's assuming your workload is IO heavy and you context switch a lot.

Why is the number of CPU cores even relevant here?

Irrelevant. You should use number of hardware threads, many modern CPUs have 2 hardware threads per core.

other cores can execute other threads from other processes

If they take considerable time, it means you have 2 processes loading the system. In this case, a better approach might be use 50% of hardware threads. When people design resource-demanding software, they usually assume it'll be the main workload of a computer.

creating a new kernel-level thread (on a system with a single hardware thread) usually won't give me extra CPU time because of the context switch

If there're other processes which also want 100% CPU, with 2 threads you will indeed get extra CPU time. But that's rare edge case, user gonna hit reset button due to an unresponsive system. Generally, unless blocking IO or security are involved, there's little point in creating more kernel threads than hardware threads.

Answer 2

Alan Cox once said, before multi-core architecture were common place, that: "A Computer is a state machine. Threads are for people who can't program state machines."

Kernel threads that gets scheduled, at least potentially, on different cores makes sense. It is my experience that in the vast majority of situations, user threads are nothing more of a needlessly costly abstraction designed to allow you to not think explicitly about managing the state machine which is a CPU core.

Which is fine, of course. We don't always, perhaps even usually, do not need top performance. But if your scenario does not require top performance as a first priority consideration, I would not bother worrying about threading models and just use the simplest.If you do care, I'd go with 1:1 kernel threading and handle a single core multiplexing explicitly.

Answer 3

Go(lang)?, as a case in point, utilizes this model for its concurrency. A goroutine may return from a system call in a different kernel thread than it dispatched from. Go aims and claims to be both efficient and achieve a high degree of utilization.

One problem is that the thread (hammer) has many applications (nail-like-objects). Concurrent programming is a form of program organization to match the operating characteristics of the system is quite different from parallel programming which aims to reduce execution time by maximizing resource utilization. There is considerable overlap, since often concurrent programs are faster and more responsive than their sequential equivalents.

(1): Extra time slices. There may be some schedulers where this is true, but what you are describing is an over-loaded system -- it has more work to do than available resources to do it. The overload may be transient, but to make this a design choice pits you at the mercy of a scheduler upgrade that balances between processes/jobs/sessions/??? rather than threads; your extra is an implementation detail.

(2): Yes, you are more right than wrong here. Just creating N kernel threads is not sufficient unless your machine has some form of coscheduling, but still could be victimized by system call that needs to synchronize on real io (eg. read(2)). At the risk of being a Go-fanboi, the Go scheduler circumvents this by keeping a system call slush fund of parked kernel threads in addition to the #execution units; so really has an L:M:N thread model.