Kernel memory (virtual address entries) in TLB?

Question

Linux is the OS and ARM is the processor referred in this context.

Does the TLB contain both kernel and user space virtual addresses? Kernel memory starts at 0xc000_0000 and goes to 0xFFFF_FFFF where the first 3 GB belongs to userspace. Between context switching between processes, the TLB is flushed.

Does the TLB contain both kernel and user space virtual addresses?

Kernel memory (virtual) directly corresponds to physical memory (just offsetting with 0xC000_0000 will give us physical address). Is it necessary to have kernel memory (virtual) in TLB (if you say it is present in TLB)? It should have only the user space address present.

Answer 1

The main reason why we have virtual to physical address translation in modern CPUs is to make a more efficient and better controlled use of memory that lets us:

1. Allocate any physical memory, RAM, (contiguous or not) and make it accessible anywhere in the virtual address space (contiguously or not), without wasting memory to fragmentation.
2. Extend the physical memory, RAM, with disk or other memories.
3. Make certain portions of the address space only readable or not-executable or kernel-only, etc., etc. and protect them from unauthorized or erroneous accesses.
4. Isolate applications' memories from one another to further improve protection, security and reliability.
5. Share memory. ...

And page tables make this all possible.

You do want to be able to map and unmap physical memory in the virtual address space in the kernel as well and usually this translation mechanism works in the entire system. Of course, the translation comes at a price as you now need to consult and maintain the page tables and that incurs a performance hit. But all is not lost:

1. TLBs alleviate this problem to a certain degree. They cache translations.
2. Bigger pages (e.g. ARMv7-A's large pages and sections) help more since they need fewer TLB entries per unit of the translated memory.
3. There're also things like global pages. When you switch between applications and need to flush the current TLB, you can avoid invalidating global pages from the TLB by performing Invalidate TLB entries by ASID match with the application's ASID. If you mark the kernel's pages as global, you don't invalidate their translations, and the kernel itself doesn't suffer from unnecessary TLB invalidations.

See, for example, "ARM® Architecture Reference Manual ARM®v7-A and ARM®v7-R edition" for specific details related to the ARM Virtual Memory System Architecture (VMSA), page tables, TLB, etc.

Answer 2

There are two types of virtual addresses that the Linux kernel uses:

1. What you already mentioned in the line "Kernel memory (virtual) directly corresponds to physical memory (just offsetting with 0xC000_0000 will give us physical address)". This maps to contiguous physical addresses.
2. Using vmalloc.

The first one is done using a MACRO:

include/asm-x86/page_32.h

#define __pa(x) ((unsigned long)(x)-PAGE_OFFSET)
#define __va(x) ((void *)((unsigned long)(x)+PAGE_OFFSET))

_pa(x) does the virtual to physical translation. Please note that this translation happens inline at compile-time. No page-table translation happens. This last sentence is very important.

On the other hand, using the second method you can allocate memory which is contiguous in virtual memory, but may not be so in physical memory. Now, in this case full page table translation is required when you access a virtual address for the first time. The question is who does this?

In the case of CISC machines (like x86), the MMU (hardware) does that in case of a TLB miss (first-time access to virtual address) and updates the page table. For kernel virtual addresses (obtained via vmalloc) they are kept as TLB entries. They are called global entries and when a process context switch occurs, they are mostly ignored and not flushed like the rest of the process-address-space entries. However, when you do a vfree to release the virtual memory associated, those entries are deleted.

In case of a RISC machine (like MIPS), the page translation is handled by software. After a TLB miss, the hardware raise an exception. A trap handle runs in kernel mode to do the translation and updates the TLB using special instructions. After returning from the trap handler the same line of code is run and a TLB hit happens.

Please refer to: http://pages.cs.wisc.edu/~remzi/OSFEP/vm-tlbs.pdf

The bottom-line is that not all kernel addresses are mapped the way you described. For your case, the physical addresses are generated at compile-time itself. So, why add a TLB entry. For addresses from vmalloc, TLB entries are present. When context switch occurs between processes, the entire TLB need not be flushed, and the global entries made by kernel's vmalloc can be preserved. When you use vfree, the corresponding global entries are flushed.