Linux CPU isolation and mutexes

3
votes

I'm currentrly investigating the option of using isolcpus in order to isolate some cores and dedicate them to some process that has some real time constrains. Looking in the documentation of isolcpus it states this

Description Remove the specified CPUs, as defined by the cpu_number values, from the general kernel SMP balancing and scheduler algroithms.

My process has let's say 4 threads and I isolate 2 cores (ignore HT) and they need to sync using some sort of linux OS sync primitives (mutex, semaphor, cond var etc). If the cores are removed from scheduling algorithm as stated in documentation and one of the threads blocks and goes to sleep state, who will schedule the thread back to the isolated CPUs when it unblocks ?

Thank you.

1
clinuxlinux-kernelscheduler

1 Answers

3
votes

That CPUs are "Remove[d ...] from the general kernel SMP balancing and scheduler algroithms" means that the kernel will not choose those (logical) CPUs for scheduling general tasks, which effectively means those that have not been specifically assigned CPU affinity for those CPUs, such as via the taskset or cset command. It follows also that tasks scheduled on one or more of those CPUs will run only on the assigned CPUs, not even any others in the isolated set, if there are any, regardless of relative CPU load.

That does not mean, however, that the kernel abdicates responsibility for scheduling tasks explicitly assigned to the isolated CPUs. It schedules them just as it schedules any other task, but only on the CPUs for which they have been assigned affinity. You do not need to arrange for special scheduling of tasks assigned to the isolated CPUs.

Also, as an aside, when you say

I isolate 2 cores (ignore HT)

, I suspect that you may have a misconception. Linux handles cores on which hyperthreading is enabled as two separate cores each, even though in fact they share processing resources. Thus, you cannot safely ignore HT if it is enabled. You in fact need to take it very carefully into account, at least in choosing how many and which cores to isolate.