GCC arm instruction mode when compiling in thumb mode

Question

I'm wondering how GCC configured using --with-mode=thumb handles compiling/assembling code that makes use of ARM mode sections if the -marm flag is not specified. That is:

GCC is compiled with --with-mode=thumb
A program is compiled without -marm (defaults to thumb mode)
An assembly section of that program uses ARM mode

I tried compiling a small test program on Raspberry Pi 4 with Ubuntu 18.04.4 kernel 5.3.0-1018-raspi2 and noticed that the .arm section is being executed in 16-bit thumb instruction mode which prompted me to investigate this. This naturally causes a segmentation fault as the program counter is increment by 2 bytes instead of 4.

Here's what gdb in layout asm mode says when my program branches into the .arm assembly code and after I perform a single stepi command:

0x400900 <asm_maxfilter>        push   {r4, lr}
0x400904 <asm_maxfilter+4>      mov    r3, #0
0x400908 <filter_loop>          vld1.8 {d0-d1}, [r0]

pc 0x400902 0x400902 <asm_maxfilter+2>
^ The program counter is between instructions

My code is as follows:

#include <arm_neon.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>

void asm_maxfilter(unsigned char* upbuffer, unsigned char* longterm_buffer, int grid_size);

int main(int argc, char** argv) {

    const int pixels_per = 16;
    const int grid_reso = 256;
    const int grid_size = grid_reso * grid_reso;
    const int remainder = grid_size % pixels_per;
    const int work_count = grid_size - remainder;

    unsigned char* longterm_up = (unsigned char*)malloc(grid_reso * grid_reso);
    memset(longterm_up, 0, grid_reso * grid_reso);

    unsigned char* up_buffers[60];
    int u;
    int i;

    for (u = 0; u < 60; ++u) {
        up_buffers[u] = (unsigned char*)malloc(grid_reso * grid_reso);

        if (up_buffers[u] == NULL) {
            fprintf(stderr, "Failed mallocing\n");
            return 1;
        }

        memset(up_buffers[u], 0, grid_reso * grid_reso);
    }

    for (u = 0; u < 60; ++u) {

        asm_maxfilter(up_buffers[u], longterm_up, work_count);

        // non-SIMD version handles the remainder that did not fit in NEON registers
        for (i = grid_size - remainder; i < grid_size; ++i) {
            if (longterm_up[i] < up_buffers[u][i]) {
                longterm_up[i] = up_buffers[u][i];
            }
        }
    }

    for (u = 0; u < 60; ++u) {
        free(up_buffers[u]);
    }

    free(longterm_up);

    return 0;
}

Assembly:

@ ARM NEON version of a max filter. Performs the following operation:
@
@ for (int i = 0; i < buf_size; ++i) {
@   if (buf_b[i] < buf_a[i]) {
@       buf_b[i] = buf_a[i];
@   }
@ }

.arm
.section .text
.align 4
.globl asm_maxfilter

@ parameters
@ r0: buf_a
@ r1: buf_b
@ r2: buf_size, multiple of 16
asm_maxfilter:

    @ Store register states in stack. They must be restored before returning
    push { r4, lr }

    @ Reset counter
    mov r3, #0

    filter_loop:

        @ Load 16 bytes into vectors
        vld1.u8 {q0}, [r0]
        vld1.u8 {q1}, [r1]

        @ Find greater values in each vector
        vcgt.u8 q2, q0, q1

        @ Bitselect the greater value into q2
        vbsl.u8 q2, q0, q1

        @ Store the larger value in output buffer
        vst1.u8 {q2}, [r1]

        @ Increment counter by 16
        add r3, r3, #16

        @ Increment pointers
        add r0, r0, #16
        add r1, r1, #16

        @ Check if loop is done
        cmp r3, r2
        blt filter_loop

    @ Restore registers to their original state
    pop { r4, lr }

    @ lr register contains return address
    bx lr

.end

The code is compiled using:

gcc -Wall -Wpedantic -O0 -g -march=armv8-a -mfloat-abi=hard -mtune=cortex-a72 -mfpu=neon -c -o main.o main.c
gcc -Wall -Wpedantic -O0 -g -march=armv8-a -mfloat-abi=hard -mtune=cortex-a72 -mfpu=neon -o neon_test ./main.o ./asm_test.s

Based on what the ARM documentation says, if the processor needs to switch between thumb/arm the program should perform a branch using the BLX or BX instruction:

https://developer.arm.com/docs/100076/0100/instruction-set-overview/overview-of-aarch32-state/changing-between-a32-and-t32-instruction-set-states

Quoting:

To direct armasm to generate A32 or T32 instruction encodings, you must set the assembler mode using an ARM or THUMB directive. Assembly code using CODE32 and CODE16 directives can still be assembled, but Arm recommends you use the ARM and THUMB directives for new code.

These directives do not change the instruction set state of the processor. To do this, you must use an appropriate instruction, for example BX or BLX to change between A32 and T32 states when performing a branch.

After disassembling my program, I noticed this mode switching is not done. Is this something that the programmer must do themselves in their assembly code (even though the branching happens from the C code), or should the compiler/assembler handle this?

I also tried specifying __attribute__((target("arm"))) in the C file function declaration, that is:

__attribute__((target("arm")))
void asm_maxfilter(unsigned char* upbuffer, unsigned char* longterm_buffer, int grid_size);

However, this didn't seem to change anything. Everything works correctly as soon as I compile with -marm or use GCC that doesn't have --with-mode=thumb

you are mixing things, first off armasm is not related to gcc, you are using gas or gnu assembler. so anything you read with respect to armasm does not apply. for certain cores with latter versions of gcc thumb mode is default. newer cores that that would apply dont only need blx/bx to switch modes as the arm documentation for those cores indicates, armv7ar a lot of the instructions with pc as a destination I believe can switch modes. — old_timer
the toolchain works fine. what you need to do in asm is declare the label as a function if thumb then .thumb_func before the label for arm mode it is something like type label,%function or some such syntax, just compile a function in arm mode to assembly language and borrow that syntax, then if you use interwork then the tools will properly take care of mode switching for you. if that is your issue — old_timer

SirDifferential SirDifferential · Accepted Answer · 2020-03-11T08:07:53

As suggested by old_timer in a comment, the problem was that the assembly source code did not include .type asm_maxfilter, %function before the label. The working assembly code begins as follows:

.arm
.section .text
.align 4
.globl asm_maxfilter

.type asm_maxfilter, %function
asm_maxfilter:

    @ Store register states in stack. They must be restored before returning
    push { r4, lr }

    @ Reset counter
    mov r3, #0
    ...

If the situation was reversed (ARM mode program using a thumb function), then instead of .type asm_maxfilter, %function the type should have been .thumb_func.

As per Jester's response, I noticed that the C code object file indeed has a R_ARM_THM_CALL relocation segment, but without using the .type macro, the branch instruction was not replaced by a bx instruction.

If one implements an ARM function in a C file by using __attribute__((target("arm"))) without external assembly, ie:

#include <stdio.h>
#include <stdlib.h>

__attribute__((target("arm")))
void foo(int a) {
    int b = 6*a;
    fprintf(stderr, "%d\n", b*5);
}

int main(int argc, char** argv) {
    int asd = atoi(argv[1]);
    foo(asd);
    return 0;
}

Then one can obseve a blx instruction being correctly used in the generated binary. The problem I had is only a concern if one uses assembly code in a separate file that does not pass through a compiler.

GCC arm instruction mode when compiling in thumb mode

2 Answers