How to use contiguous memory in linux kernel?

Question

I found that physical memory is split into ranks as follows (Memory Interleaving):

   rank0: [0-512KB] [2048KB-2560KB] [4096KB-4608KB] ...
   rank1: [512KB-1024KB] [2560KB-3072KB] [4608KB-5120KB] ... 
   rank2: [1024KB-1536KB] [3072KB-3584KB] [5120KB-...
   rank3: [1536KB-2048KB] [3584KB-4096KB] ...

Linux kernel is getting these interleaved memory. So, the physical memory seen by linux kernel is not contiguous. Correct me if I am wrong about this.

I have been looking at linux kernel source code for my course work.

While creating sysfs (/sys/devices/system/memory), linux kernel creates the sections(memory0, memory1,...) of certain size(128 MB on my system) of these available physical memory. Using state files in these directories I can make sections offline/Online. (Memory Hotplug)

So, physical memory represented by these sections is scattered. So, if I make any of the section offline that will make that scattered memory it is pointing to unavailable.

I want to make these sections refer to contiguous memory. something like this :

   memory0 : 0-128 MB
   memory1 : 128-256 MB
   ....

So, when I make some section offline then contiguous physical memory related to that section becomes unavailable. So, can I make linux kernel see physical memory as contiguous instead of interleaved?

Correct me if I am wrong about any of this.

Thank you.

What is your processor architecture? You said memory is split into ranks. In reality this heavily depends on how the processor memory management hardware work. — user2284570
Sorry for the late answer. I was busy with mid-terms. I was referring general architecture not specific to my machine. Sorry for the confusion. But for this discuss assume system supports this interleaving, so is there any way to disable it ? — Rajesh Golani
@SunEric I've no idea of which system Rajesh Golani run. So I can't tell the way to do it. — user2284570

vPraetor vPraetor · Accepted Answer · 2013-11-01T12:31:35

I think what you are seeing is due to interleave option for memory allocation under NUMA. There's a very good paper which talks about NUMA under Linux.

Without the details of the hardware that you are using it's very difficult to guess how the memory interleaving option can be disabled. You should also dig into the numactl utility to see if it has some setting that you can use on a running system. There are also some blog posts which mention changing the BIOS settings to control the memory interleaving option and the you may have to do the same. (not eligible to give 2 links right now :\ but a simple Google search on disabling memory interleaving in BIOS should turn up something useful for you)

How to use contiguous memory in linux kernel?

1 Answers