AArch64 (Cortex-A53) - Understanding translation table

Question

I'm trying to understand the translation table from a Cortex-A53. Currently without success. Cannot enable the MMU. I would like to know how to correctly setup the MMU. Have tried this tutorial from ARM.

Relevant code (enabling):

__el2_cache:
    mrs x0, SCTLR_EL2
    bic x0, x0, #(1 << 0)  /* Disable MMU */
    bic x0, x0, #(1 << 2)  /* Disable D-Cache */
    bic x0, x0, #(1 << 12) /* Disable I-Cache */
    msr SCTLR_EL2, x0
    isb

    /* Invalidate and clean I/D-Cache */
    bl _cpu_icache_invalidate
    bl _cpu_dcache_l1_invalidate
    bl _cpu_dcache_l2_invalidate
    
__el2_pagetable:
    /* Create pagetable for EL2 */
    bl _cpu_el2_tlb_create
    
    /* Invalidate (old) Pagetable */
    tlbi ALLE2
    dsb sy
    isb
    
    mrs x0, SCTLR_EL2
    orr x0, x0, #(1 << 0)  /* Enable MMU */
    orr x0, x0, #(1 << 2)  /* Enable D-Cache */
    orr x0, x0, #(1 << 12) /* Enable I-Cache */
    msr SCTLR_EL2, x0 
    isb /* <-- CPU hangs here */
    nop
    nop
    nop
    nop

It also doesn't work when I don't enable the caches. The code in the sub-procedure call for invalidating the caches is also based on an ARM tutorial.

I create the translation table in _cpu_el2_tlb_create and set the relevant registers:

    ldr x1, =0x80803520
    msr TCR_EL2, x1
    ldr x1, =0x4400FF00
    msr MAIR_EL2, x1
    ldr x1, =_tlb_el2_tbb0_lv1
    msr TTBR0_EL2, x1
    mov x8, xzr
    dsb sy
    ret

Have used the same setup (memory type, sharable, ...) as U-Boot, but still doesn't work.

_cpu_el2_tlb_create creates the following tables:

Level 1          Level 2
0000000010006003 0000000000000711 0000000040000711 0000000080000711 00000000c0000711 00000000ff000401  
0000000010007003 0000000000200711 0000000040200711 0000000080200711 00000000c0200711 00000000ff200401     
0000000010008003 0000000000400711 0000000040400711 0000000080400711 00000000c0400711 00000000ff400401
0000000010009003 0000000000600711 0000000040600711 0000000080600711 00000000c0600711 00000000ff600401
000000001000a003 0000000000800711 0000000040800711 0000000080800711 00000000c0800711 00000000ff800401
0000000000000000 0000000000a00711 0000000040a00711 0000000080a00711 00000000c0a00711 00000000ffa00401
0000000000000000 0000000000c00711 0000000040c00711 0000000080c00711 00000000c0c00711 00000000ffc00401
0000000000000000 0000000000e00711 0000000040e00711 0000000080e00711 00000000c0e00711 00000000ffe00401
0000000000000000 0000000001000711 0000000041000711 0000000081000711 00000000c1000711 0000000000000000
[...]
0000000000000000 000000003ee00711 000000007ee00711 00000000bee00711 00000000fee00711 0000000000000000
0000000000000000 000000003f000711 000000007f000711 00000000bf000711 0000000000000000 0000000000000000
0000000000000000 000000003f200711 000000007f200711 00000000bf200711 0000000000000000 0000000000000000
0000000000000000 000000003f400711 000000007f400711 00000000bf400711 0000000000000000 0000000000000000
0000000000000000 000000003f600711 000000007f600711 00000000bf600711 0000000000000000 0000000000000000
0000000000000000 000000003f800711 000000007f800711 00000000bf800711 0000000000000000 0000000000000000
0000000000000000 000000003fa00711 000000007fa00711 00000000bfa00711 0000000000000000 0000000000000000
0000000000000000 000000003fc00711 000000007fc00711 00000000bfc00711 0000000000000000 0000000000000000
0000000000000000 000000003fe00711 000000007fe00711 00000000bfe00711 0000000000000000 0000000000000000

TTBR0_EL2 points also to the correct location and is 4KiB alignment.

The memory device map from the used board:

0x00000000 - 0xFF000000 DDR-RAM
0xFF000000 - 0xFFFF0000 Memory-Mapped I/O

Why do you have a 000000001000a003 entry in your L1 table? With T0SZ=32, that would fall outside your virtual address space. — Siguza
@Siguza Thanks for this hint. I made maybe a terrible mistake with the generation of the tables. Might be an off-one-error in the creation the table. Level 1 table is actual the level 0 table. The level 0 table points to 2MiB level 2 tables. But should be 1GiB entries. But I must verify this mistake. — krjdev
Mind you, with T0SZ=32 you wouldn't have an L0 table though... — Siguza
@Siguza Although I understand you, if TOSZ = 32 I can skip a level 0 table and setup TBBR0_EL2 to the level 1 table? — krjdev

krjdev krjdev · Accepted Answer · 2021-01-23T00:54:27

I solved the issue.

Unfortunately I didn't understand the translation table and the required settings.

Now my current setup for EL2:
Bare-Metal application is at 0x80000000 with a size of 512MiB defined in the linker script.

Extract from my current linker script:

ENTRY(_start)

MEMORY {
    RAM_KERN (rwx) : ORIGIN = 0x80000000, LENGTH = 512M
}

STACK_EL3 = 4K;
STACK_EL2 = 4K;
STACK_EL1 = 64M;
STACK_EL0 = 1M;

Some settings for TCR_EL2:
Currently I use a 1TiB range for TOSZ (0x17), so i also have a level 0 table with only one entry. But decrease this value later. Thanks to @Siguza for this hint. I use 4KiB granule for the tables.

Level 0:
Contains only one entry at index 0 which points to the first (and only) level 1 table. All other entries are invalid (zero).

Level 1 (Index 0):
These table contains two valid table entries. Because the bare-metal application starts at 0x80000000, the first 1GiB entries are currently invalid. So there is the bare-metal application at index 2. Index 3 in the level contains the entry to the MMIO table. All other subsequent entries are invalid.

Level 2 (Bare-Metal application):
These table contains 256 2MiB block entries because in linker script i defined 512MiB for the application. Start point is index 0 of the level 2 table. The subsequent entries (256) are invalid.

Level 2 (MMIO devices):
These table contains 8 2MiB block entries starting at index 503, because the IO is mapped at start point 0xFF000000. The first 502 entries are currently invalid.

Now I can successfully enable the MMU.

AArch64 (Cortex-A53) - Understanding translation table

1 Answers