On a ARM based system running Linux, I have a device that's memory mapped to a physical address. From a user space program where all addresses are virtual, how can I read content from this address?
2 Answers
You can map a device file to a user process memory using mmap(2)
system call. Usually, device files are mappings of physical memory to the file system.
Otherwise, you have to write a kernel module which creates such a file or provides a way to map the needed memory to a user process.
Another way is remapping parts of /dev/mem to a user memory.
Edit: Example of mmaping /dev/mem (this program must have access to /dev/mem, e.g. have root rights):
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <unistd.h>
int main(int argc, char *argv[]) {
if (argc < 3) {
printf("Usage: %s <phys_addr> <offset>\n", argv[0]);
return 0;
}
off_t offset = strtoul(argv[1], NULL, 0);
size_t len = strtoul(argv[2], NULL, 0);
// Truncate offset to a multiple of the page size, or mmap will fail.
size_t pagesize = sysconf(_SC_PAGE_SIZE);
off_t page_base = (offset / pagesize) * pagesize;
off_t page_offset = offset - page_base;
int fd = open("/dev/mem", O_SYNC);
unsigned char *mem = mmap(NULL, page_offset + len, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, page_base);
if (mem == MAP_FAILED) {
perror("Can't map memory");
return -1;
}
size_t i;
for (i = 0; i < len; ++i)
printf("%02x ", (int)mem[page_offset + i]);
return 0;
}
busybox devmem
busybox devmem
is a tiny CLI utility that mmaps /dev/mem
.
You can get it in Ubuntu with: sudo apt-get install busybox
Usage: read 4 bytes from the physical address 0x12345678
:
sudo busybox devmem 0x12345678
Write 0x9abcdef0
to that address:
sudo busybox devmem 0x12345678 w 0x9abcdef0
Source: https://github.com/mirror/busybox/blob/1_27_2/miscutils/devmem.c#L85
mmap MAP_SHARED
When mmapping /dev/mem
, you likely want to use:
open("/dev/mem", O_RDWR | O_SYNC);
mmap(..., PROT_READ | PROT_WRITE, MAP_SHARED, ...)
MAP_SHARED
makes writes go to physical memory immediately, which makes it easier to observe, and makes more sense for hardware register writes.
CONFIG_STRICT_DEVMEM
and nopat
To use /dev/mem
to view and modify regular RAM on kernel v4.9, you must fist:
- disable
CONFIG_STRICT_DEVMEM
(set by default on Ubuntu 17.04) - pass the
nopat
kernel command line option for x86
IO ports still work without those.
See also: mmap of /dev/mem fails with invalid argument for virt_to_phys address, but address is page aligned
Cache flushing
If you try to write to RAM instead of a register, the memory may be cached by the CPU: How to flush the CPU cache for a region of address space in Linux? and I don't see a very portable / easy way to flush it or mark the region as uncacheable:
- How to write kernel space memory (physical address) to a file using O_DIRECT?
- How to flush the CPU cache for a region of address space in Linux?
- Is it possible to allocate, in user space, a non cacheable block of memory on Linux?
So maybe /dev/mem
can't be used reliably to pass memory buffers to devices?
This can't be observed in QEMU unfortunately, since QEMU does not simulate caches.
How to test it out
Now for the fun part. Here are a few cool setups:
- Userland memory
- allocate
volatile
variable on an userland process - get the physical address with
/proc/<pid>/maps
+/proc/<pid>/pagemap
- modify the value at the physical address with
devmem
, and watch the userland process react
- allocate
- Kernelland memory
- allocate kernel memory with
kmalloc
- get the physical address with
virt_to_phys
and pass it back to userland - modify the physical address with
devmem
- query the value from the kernel module
- allocate kernel memory with
- IO mem and QEMU virtual platform device
- create a platform device with known physical register addresses
- use
devmem
to write to the register - watch
printf
s come out of the virtual device in response
Bonus: determine the physical address for a virtual address
Is there any API for determining the physical address from virtual address in Linux?