memory alignment within gcc structs

Question

I am porting an application to an ARM platform in C, the application also runs on an x86 processor, and must be backward compatible.

I am now having some issues with variable alignment. I have read the gcc manual for __attribute__((aligned(4),packed)) I interpret what is being said as the start of the struct is aligned to the 4 byte boundry and the inside remains untouched because of the packed statement.

originally I had this but occasionally it gets placed unaligned with the 4 byte boundary.

typedef struct  
{  
 unsigned int code;  
 unsigned int length;  
 unsigned int seq;  
 unsigned int request;  
 unsigned char nonce[16];  
 unsigned short  crc;  
} __attribute__((packed)) CHALLENGE;

so I change it to this.

typedef struct  
{  
 unsigned int code;  
 unsigned int length;  
 unsigned int seq;  
 unsigned int request;  
 unsigned char nonce[16];  
 unsigned short  crc;  
} __attribute__((aligned(4),packed)) CHALLENGE;

The understand I stated earlier seems to be incorrect as both the struct is now aligned to a 4 byte boundary, and and the inside data is now aligned to a four byte boundary, but because of the endianess, the size of the struct has increased in size from 42 to 44 bytes. This size is critical as we have other applications that depend on the struct being 42 bytes.

Could some describe to me how to perform the operation that I require. Any help is much appreciated.

Answer 1

If you're depending on sizeof(yourstruct) being 42 bytes, you're about to be bitten by a world of non-portable assumptions. You haven't said what this is for, but it seems likely that the endianness of the struct contents matters as well, so you may also have a mismatch with the x86 there too.

In this situation I think the only sure-fire way to cope is to use unsigned char[42] in the parts where it matters. Start by writing a precise specification of exactly what fields are where in this 42-byte block, and what endian, then use that definition to write some code to translate between that and a struct you can interact with. The code will likely be either all-at-once serialisation code (aka marshalling), or a bunch of getters and setters.

Answer 2

This is one reason why reading whole structs instead of memberwise fails, and should be avoided.

In this case, packing plus aligning at 4 means there will be two bytes of padding. This happens because the size must be compatible for storing the type in an array with all items still aligned at 4.

I imagine you have something like:

read(fd, &obj, sizeof obj)

Because you don't want to read those 2 padding bytes which belong to different data, you have to specify the size explicitly:

read(fd, &obj, 42)

Which you can keep maintainable:

typedef struct {
  //...
  enum { read_size = 42 };
} __attribute__((aligned(4),packed)) CHALLENGE;

// ...

read(fd, &obj, obj.read_size)

Or, if you can't use some features of C++ in your C:

typedef struct {
  //...
} __attribute__((aligned(4),packed)) CHALLENGE;
enum { CHALLENGE_read_size = 42 };

// ...

read(fd, &obj, CHALLENGE_read_size)

At the next refactoring opportunity, I would strongly suggest you start reading each member individually, which can easily be encapsulated within a function.

Answer 3

I've been moving structures back and forth from Linux, Windows, Mac, C, Swift, Assembly, etc.

The problem is NOT that it can't be done, the problem is that you can't be lazy and must understand your tools.

I don't see why you can't use:

typedef struct  
{  
 unsigned int code;  
 unsigned int length;  
 unsigned int seq;  
 unsigned int request;  
 unsigned char nonce[16];  
 unsigned short  crc;  
} __attribute__((packed)) CHALLENGE;

You can use it and it doesn't require any special or clever code. I write a LOT of code that communicates to ARM. Structures are what make things work. __attribute__ ((packed)) is my friend.

The odds of being in a "world of hurt" are nil if you understand what is going on with both.

Finally, I can't for the life make out how you get 42 or 44. Int is either 4 or 8 bytes (depending on the compiler). That puts the number at either 16+16+2=34 or 32+16+2=50 -- assuming it is truly packed.

As I say, knowing your tools is part of your problem.

Answer 4

What is your true goal?

If it's to deal with data that's in a file or on the wire in a particular format what you should do is write up some marshaling/serialization routines that move the data between the compiler struct that represents how you want to deal with the data inside the program and a char array that deals with how the data looks on the wire/file.

Then all that needs to be dealt with carefully and possibly have platform specific code is the marshaling routines. And you can write some nice-n-nasty unit tests to ensure that the marshaled data gets to and from the struct properly no matter what platform you might have to port to today and in the future.

Answer 5

I would guess that the problem is that 42 isn't divisible by 4, and so they get out of alignment if you put several of these structs back to back (e.g. allocate memory for several of them, determining the size with sizeof). Having the size as 44 forces the alignment in these cases as you requested. However, if the internal offset of each struct member remains the same, you can treat the 44 byte struct as though it was 42 bytes (as long as you take care to align any following data at the correct boundary).

One trick to try might be putting both of these structs inside a single union type and only use 42-byte version from within each such union.

Answer 6

As I am using linux, I have found that by echo 3 > /proc/cpu/alignment it will issue me with a warning, and fix the alignment issue. This is a work around but it is very helpful with locating where the structures are failing to be misaligned.