Arduino to Raspberry crc32 check

Question

I'm trying to send messages through the serial USB interface of my Arduino (C++) to a Raspberry Pi (Python).

On the Arduino side I define a struct which I then copy into a char[]. The last part of the struct contains a checksum that I want to calculate using CRC32. I copy the struct into a temporary char array -4 bytes to strip the checksum field. The checksum is then calculated using the temporary array and the result is added to the struct. The struct is then copied into byteMsg which gets send over the serial connection.

On the raspberry end I do the reverse, I receive the bytestring and calculate the checksum over the message - 4 bytes. Then unpack the bytestring and compare the received and calculated checksum but this fails unfortunately.

For debugging I compared the crc32 check on both the python and arduino for the string "Hello World" and they generated the same checksum so doesn't seem to be a problem with the library. The raspberry is also able to decode the rest of the message just fine so the unpacking of the data into variables seem to be ok as well.

Any help would be much appreciated.

The Python Code:

   def unpackMessage(self, message):
        """ Processes a received byte string from the arduino """

        # Unpack the received message into struct
        (messageID, acknowledgeID, module, commandType,
         data, recvChecksum) = struct.unpack('<LLBBLL', message)

        # Calculate the checksum of the recv message minus the last 4
        # bytes that contain the sent checksum
        calcChecksum = crc32(message[:-4])
        if recvChecksum == calcChecksum:
            print "Checksum checks out"

The Aruino crc32 library taken from http://excamera.com/sphinx/article-crc.html

crc32.h

#include <avr/pgmspace.h>

static PROGMEM prog_uint32_t crc_table[16] = {
    0x00000000, 0x1db71064, 0x3b6e20c8, 0x26d930ac,
    0x76dc4190, 0x6b6b51f4, 0x4db26158, 0x5005713c,
    0xedb88320, 0xf00f9344, 0xd6d6a3e8, 0xcb61b38c,
    0x9b64c2b0, 0x86d3d2d4, 0xa00ae278, 0xbdbdf21c
};

unsigned long crc_update(unsigned long crc, byte data)
{
    byte tbl_idx;
    tbl_idx = crc ^ (data >> (0 * 4));
    crc = pgm_read_dword_near(crc_table + (tbl_idx & 0x0f)) ^ (crc >> 4);
    tbl_idx = crc ^ (data >> (1 * 4));
    crc = pgm_read_dword_near(crc_table + (tbl_idx & 0x0f)) ^ (crc >> 4);
    return crc;
}

unsigned long crc_string(char *s)
{
  unsigned long crc = ~0L;
  while (*s)
    crc = crc_update(crc, *s++);
  crc = ~crc;
  return crc;
}

Main Arduino Sketch

struct message_t {
    unsigned long messageID;
    unsigned long acknowledgeID;
    byte module;
    byte commandType;
    unsigned long data;
    unsigned long checksum;
};

void sendMessage(message_t &msg)
{
    // Set the messageID
    msg.messageID = 10;
    msg.checksum = 0;

    // Copy the message minus the checksum into a char*
    // Then perform the checksum on the message and copy
    // the full msg into byteMsg
    char byteMsgForCrc32[sizeof(msg)-4];
    memcpy(byteMsgForCrc32, &msg, sizeof(msg)-4);
    msg.checksum = crc_string(byteMsgForCrc32);

    char byteMsg[sizeof(msg)];
    memcpy(byteMsg, &msg, sizeof(msg));

    Serial.write(byteMsg, sizeof(byteMsg));

void loop() {
    message_t msg;
    msg.module = 0x31;
    msg.commandType = 0x64;
    msg.acknowledgeID = 0;
    msg.data = 10;

    sendMessage(msg);

Kind Regards, Thiezn

Nathilion Nathilion · Accepted Answer · 2015-02-24T13:34:23

You are making the classic struct-to-network/serial/insert communication layer mistake. Structs have hidden padding in order to align the members onto suitable memory boundaries. This is not guaranteed to be the same across different computers, let alone different CPUs/microcontrollers.

Take this struct as an example:

struct Byte_Int
{
     int x;
     char y;
     int z;
}

Now on a basic 32-bit x86 CPU you have a 4-byte memory boundary. Meaning that variables are aligned to either 4 bytes, 2 bytes or not at all according to the type of variable. The example would look like this in memory: int x on bytes 0,1,2,3, char y on byte 4, int z on bytes 8,9,10,11. Why not use the three bytes on the second line? Because then the memory controller would have to do two fetches to get a single number! A controller can only read one line at a time. So, structs (and all other kinds of data) have hidden padding in order to get variables aligned in memory. The example struct would have a sizeof() of 12, and not 9!

Now, how does that relate to your problem? You are memcpy()ing a struct directly into a buffer, including the padding. The computer on the other end doesn't know about this padding and misinterprets the data. What you need a serialization function that takes the members of your structs and pasts them into a buffer one at a time, that way you lose the padding and end up with something like this: [0,1,2,3: int x][4: char y][5,6,7,8: int z]. All as one lengthy bytearray/string which can be safely sent using Serial(). Of course on the other end you would have to parse this string into intelligible data. Python's unpack() does this for you as long as you give the right format string.

Lastly, an int on an Arduino is 16 bits long. On a pc generally 4! So assemble your unpack format string with care.

Arduino to Raspberry crc32 check

2 Answers