what is the right way to update MMU translation table

Question

I enabled MMU on my s3c2440 board (3G - 4G memory :: the fault attribute),everything was just fine when I didn't read/write 3G - 4G memory .So to test the page fault vector ,I wrote to a 0xFF to the 3G address,as I expected ,I got the right value from FSR ,So I did this in _do_page_fault (), the step was like this :

.....                 // set new page to translation table 
.....
invlidate_icache ();  // clear icache 
clr_dcache ();        // wb is used ,clear dcache 
invalidate_ttb ();    // invalidate translation table 

and then ISR_dataabort returned ,I read the 3G address to get the 0xFF which I wote before . Unfortunately I got a data abort again .(I am sure the translation table value I set is ok)

So what is the correct way to update the MMU translation table .Any help is appreciate ! thanks

here is the Main code I used (just for some test), (I am a littlte strange to ARM ARCH,so this code might be urgly )

/* MMU TTB 0 BASE ATTR */
#define TTB0_FAULT          (0|(1<<4))          /* TTB FAULT */
#define TTB0_COARSE         (1|(1<<4))          /* COARSE PAGE BASE ADDR */
#define TTB0_SEG            (2|(1<<4))          /* SEG BASE ADDR */
#define TTB0_FINE           (3|(1<<4))          /* FINE PAGE BASE ADDR */

/* MMU TTB 1 BASE ATTR */ 
#define TTB1_FAULT          (0)
#define TTB1_LPG            (1)                 /* Large page */
#define TTB1_SPG            (2)                 /* small page */
#define TTB1_TPG            (3)                 /* tiny page */

/* domain access priority level */
#define FAULT_PL            (0x0)               /* domain fault */
#define USR_PL              (0x1)               /* usr mode */
#define RSV_PL              (0x2)               /* reserved */
#define SYS_PL              (0x3)               /* sys mode */

#define DOMAIN_FAULT        (0x0<<5)            /* fault 0*/
#define DOMAIN_SYS          (0x1<<5)            /* sys 1*/
#define DOMAIN_USR          (0x2<<5)            /* usr 2*/

/* C,B bit */
#define CB                  (3<<2)              /* cache_on, write_back */
#define CNB                 (2<<2)              /* cache_on, write_through */ 
#define NCB                 (1<<2)              /* cache_off,WR_BUF on */
#define NCNB                (0<<2)              /* cache_off,WR_BUF off */

/* ap 2 bits */
#define AP_FAULT            (0<<10)             /* access deny */
#define AP_SU_ONLY          (1<<10)             /* rw su only */
#define AP_USR_RO           (2<<10)             /* sup=RW, user=RO */
#define AP_RW               (3<<10)             /* su=RW, user=RW */

/* page dir 1 ap0 */
#define AP0_SU_ONLY         (1<<4)             /* rw su only */
#define AP0_USR_RO          (2<<4)             /* sup=RW, user=RO */
#define AP0_RW              (3<<4)             /* su=RW, user=RW */

/* page dir 1 ap1 */
#define AP1_SU_ONLY         (1<<6)             /* rw su only */
#define AP1_USR_RO          (2<<6)             /* sup=RW, user=RO */
#define AP1_RW              (3<<6)             /* su=RW, user=RW */


/* page dir 1 ap2 */
#define AP2_SU_ONLY         (1<<8)             /* rw su only */
#define AP2_USR_RO          (2<<8)             /* sup=RW, user=RO */
#define AP2_RW              (3<<8)             /* su=RW, user=RW */

/* page dir 1 ap3 */
#define AP3_SU_ONLY         (1<<10)             /* rw su only */
#define AP3_USR_RO          (2<<10)             /* sup=RW, user=RO */
#define AP3_RW              (3<<10)             /* su=RW, user=RW */


#define RAM_START           (0x30000000)
#define KERNEL_ENTRY        (0x30300000)        /* BANK 6 (3M) */
#define KERNEL_STACK        (0x3001A000)        /* BANK 6 (16K + 64K + 16K + 8K) (8k kernel stack) */
#define IRQ_STACK           (0x3001B000)        /* 4K IRQ STACK */
#define KERNEL_IMG_SIZE     (0x20000)            

#define IRQ_STACK           (0x3001B000)

/* 16K aignment */
#define TTB_BASE            (0x30000000)        
#define PAGE_DIR0           (TTB_BASE)
#define TTB_FULL_SIZE       (0x4000)        
#define PAGE_DIR1           (TTB_BASE+TTB_FULL_SIZE)                 

#define PAGE_DIR0_SIZE      (0x4000)            /* 16k */


    void _do_page_fault (void)
    {
        //
        ...........
        // 
        // read the FSR && get the vaddr && type here 
    volatile unsigned *page_dir = (volatile unsigned*)(TTB_BASE);  
    unsigned index = vaddr >> 20,i = 0, j = 0;
    unsigned page = 0;

        if (!(page_dir[index] & ~(0x3FF) && (type == 0x0B))) {          /* page_dir empty */
            i = index & ~0x03;                                          
            if ( (page_dir[i+0] & ~(0x3FF)) || (page_dir [i+1] & ~(0x3FF))         
              || (page_dir[i+2] & ~(0x3FF)) || (page_dir [i+3] & ~(0x3FF)) )
            {
                panic ( "page dir is bad !\n" );                        /* 4 continuous page_dir must be 0 */
            }

            if (!(page = find_free_page ())) 
                panic ( "no more free page !\n" );                      /* alloc a page page dir*/

            page_dir[i+0] = (page + 0x000) | DOMAIN_USR | TTB0_COARSE ; /* small page 1st 1KB */
            page_dir[i+1] = (page + 0x400) | DOMAIN_USR | TTB0_COARSE ; /* small page 2nd 1KB */
            page_dir[i+2] = (page + 0x800) | DOMAIN_USR | TTB0_COARSE ; /* small page 3rd 1KB */
            page_dir[i+3] = (page + 0xC00) | DOMAIN_USR | TTB0_COARSE ; /* small page 4th 1KB */

            if (!(page = find_free_page ())) 
                panic ( "no more free page !\n" );                      /* alloc a page page table*/

            volatile unsigned *page_tbl = (volatile unsigned*) (page_dir[index] & ~(0x3FF));

            *page_tbl = page|AP0_RW|AP1_RW|AP2_RW|AP3_RW| NCNB|TTB1_SPG;/* small page is used */


            invalidate_icache ();

            for (i = 0; i < 64; i++)
            {
                for (j = 0;j < 8;j ++)
                    clr_invalidate_dcache ( (i<<26)|(j<<5) );
            }


            invalidate_tlb ();
        }
        ........
        //

    }

    /* here is the macros */

    #define invalidate_tlb()    \
    {\
         __asm__  __volatile__ (\
            "mov    r0,#0\n"\
            "mcr    p15,0,r0,c8,c7,0\n"\
            :::"r0" \
        );\
    }

    #define clr_invalidate_dcache(index)    \
    {\
        __asm__  __volatile__ (\
            "mcr    p15,0,%[i],c7,c14,2\n"\
            :: [i]"r"(index)\
        );\
    }


    #define invalidate_icache() \
    {\
        __asm__  __volatile__ (\
            "mov    r0,#0\n"\
            "mcr    p15,0,r0,c7,c5,0\n"\
            ::: "r0"\
        );\
    }

    #define invalidate_dcache() \
    {\
        __asm__  __volatile__ (\
            "mov    r0,#0\n"\
            "mcr    p15,0,r0,c7,c6,0\n"\
            ::: "r0"\
        );\
    }



    #define invalidate_idcache()    \
    {\
        __asm__  __volatile__ (\
            "mov    r0,#0\n"\
            "mcr    p15,0,r0,c7,c7,0\n"\
            :::"r0"\
        );\
    }\

Answer 1

Note: I assume TTB_BASE is the primary ARM L1 page table. If it is some shadow, you need to show more code as per unixsmurf. Here is my best guess...

Your page_dir is functioning as both the primary L1 entries and as the L2 fine page table. The TTB_BASE should only contain sections, super sections or pointers to sub-page tables. You need to allocate more physical memory for the L2 page tables.

I guess your page_dir[i+0], page_dir[i+1], etc are overwriting other L1 section entries and Your invalidate_tlb() make this concrete to the CPU. You should be using the L2 page_tbl pointer to set/index the small/fine pages. Perhaps your kernel code is in the 3G-4G space and your are over-writing some critical L1 mappings there; any number of strange things could happen. The current use of page_tbl is unclear.

You can not double use the primary L1 tables as they have meaning to hardware. I guess that this is just a mistake?

There are only three types of entries in the primary L1 page tables,

1. Sections - a 1MB memory mapping straight virt to phys, with no 2^nd page table.
2. Super-sections - a 4MB memory mapping as per sections, but minimizing TLB pressure.
3. A 2^nd page table. Entries can be 1k,4k,and 64k. Fine page tables are 4k in table size and are not in modern ARM designs. Each entry is 1k of address in a fine page table.

The primary page table must be on a physical address that is 16k aligned, which is also it's size. 16k/(4bytes/entry)*1MB gives a 4GB address range of the L1 tables. So each entry in the primary L1 page table always refers to a 1MB entry. Coarse page tables are the only option on newer ARMs and they refer to a 1K L2 table.

1K_L2_size * 4K_entry / (4bytes_per_entry) gives 1MB address space.
1K_L2_size * 64K_entry / (16bytes_per_entry) gives 1MB address space.

There are four entries of 1MB each in the primary L1 for a super-section. There are four entries each for a 64k large page in the L2 table. If you are using super-sections, you do not have L2 entries.

I think you may have Super-sections and large pages mixed up? For certain having improperly formatted page tables will only manifest on a TLB invalidate, so that MMU mappings are re-fetched from the tables via a walk.

Finally, you should flush the D cache and drain the write buffer to ensure consistency with memory. You may wish to have a memory barrier as well.

static inline void dcache_clean(void)
{
    const int zero = 0;
    asm volatile ("" ::: "memory"); /* barrier */
    /* clean entire D cache -> push to external memory. */
    asm volatile ("1: mrc p15, 0, r15, c7, c10, 3\n"
                    " bne 1b\n" ::: "cc");
    /* drain the write buffer */
    asm volatile ("mcr 15, 0, %0, c7, c10, 4"::"r" (zero));
}

There are also co-processor commands to invalidate single TLB entries as you are only changing a portion of the vaddr/paddr mappings in the data fault.

See also: ARM MMU tutorial, Virtual Memory structures, and your ARM Architecture Reference Manual.