Linux userspace and kernel space scheduling

3
votes

I have a user space process running with scheduling policy SCHED_OTHER (0) and priority of 120 (default priority, top shows PR of 20). It runs a infinite while(1) loop without any system calls or waits etc. It is tied to a particular CPU say, 1.

In the kernel space I have a kernel thread which is also created with default scheduling parameters (policy: SCHED_NORMAL (0), and priority 120). It goes to sleep calling wait_event_interruptible(). A irq thread which happens at periodicity of say 1ms, wakes up the kernel thread. The kernel thread is not bound to any CPU.

In the case of kernel thread is scheduled on same CPU as userspace process, it does not get waken up though the wakeup call is done. If kernel thread is scheduled on other CPU which is free it gets waken up. Only when the kernel timer interrupt happens and ksoftirq thread is scheduled and while exiting it schedules kernel thread. Because of this, the kernel thread does not wakeup once in 1ms as expected.

I expected kernel thread to wake up by preempting the userspace process. That is not happening. Can someone help me with what is happening here?

BTW, if I change the scheduling of kernel thread to SCHED_FIFO and give RT priority, it works fine.

1
linuxlinux-kernel

1 Answers

0
votes

I expected kernel thread to wake up by preempting the userspace process. Can someone help me with what is happening here?

First of all, SCHED_OTHER is an alias for SCHED_NORMAL. Secondly, scheduler doesn't distinguish kernel and userspace threads (unless hierarchical scheduling via CGroups involved).

The only chance sleeper thread is allowed to preempt CPU-bound thread is that if it has smaller time spent on CPU (which is tracked via vruntime), but 1ms seem to be too small interval to make this effect significant, i.e. basis for computing timeslices in CFS is 6ms (sysctl_sched_latency).

You might try to use cfstrace.stp script (requires SystemTap) to trace scheduler state and try to understand why this is happening (it'll dump internal state of CFS including vruntime values).

I change the scheduling of kernel thread to SCHED_FIFO and give RT priority, it works fine.

Yes, if you need "real time", you need to use SCHED_FIFO or SCHED_DEADLINE