MD5 vs CRC32 to validate integrity of 100kB of binary data transferred over cellular network

0
votes

I am trying to whether I can use CRC32 as the integrity checking mechanism for my firmware over the air updates, or whether I should be using a hashing algorithm such as MD5?

My situation:

The platform is an STM32F091RC (512KB Flash, 32KB RAM), which does not have hardware/peripheral for MD5 or other hashing algorithms. It does provide a CRC32 peripheral. The firmware download will be of an exact known length in the 100KB range, and will be transmitted over cellular data network (LTE - Cat M1 and 2G). I am not concerned about deliberate tampering - only concern is transmission error due to noise etc. All transmissions will use TCP as the transport protocol. To the best of my understanding both the cellular packets and the TCP packets they encapsulate each have their own packet level CRC. The TCP packets are CRC16 for packets no larger than 1500B, and I believe the cellular packets are smaller and have their own CRC (unsure of CRC size). So two sets of CRC checking has been done with necessary retransmission of bad packets.

CRC32 will be simpler to implement and have a much smaller / no footprint. I am assuming MD5 will provide a higher level of integrity assurance (if that is the right term!), however at a cost of complexity and footprint (It may also be slower, but this is not a real concern as these downloads are only very occasional).

I have read a number of pieces (on stack overflow and elsewhere), but still don't have a clear view on what transfer size limit I should apply before switching from CRC32 to MD5 or a similar hash algorithm? One post had:

EDIT: Looking at the Wikipedia page about CRC and Lott's answer, here' what we have: less than 64 bytes: 8-bit CRC less than 16K bytes: 16-bit CRC less than 512M bytes: 32-bit CRC

So, based on above I would think that CRC32 would be fine, but I don't have a high comfort level as this is the only piece I found with these types of numbers.

Very interested on any views as to what I should do - CRC32 or MD5/similar hash algorithm?

1
md5crc32stm32f0
Stack Overflow is a Q&A site and not for discussion of topics or asking opinion. You ask a specific question about a specific issue which will return a specific solution. What topics can I ask about here? and How to Ask – Rob
A CRC with a (x+1) factor is guaranteed to detect all 2 or odd number of bit errors for messages (data + CRC) up to 2^(n-1)-1 bits. This would translate into 4095 bytes for 16 bit CRC and 268,435,455 bytes for 32 bit CRC. For guaranteed 2 bit detection, then 8191 bytes for 16 bit CRC, 536,870,911 bytes for 32 bit CRC. For more bit errors, the probability of not detecting an error is about (1/(2^n)). Message size affects probability of an error (1 - (probability of zero packets in error)). – rcgldr

1 Answers

1
votes

Since you don't care about deliberate tampering, you don't need MD5. There are other, much faster, non-cryptographic hashes longer than 32 bits you could use.

All you need to figure out is what probability of false positive you would find acceptable. I.e., the probability of the integrity check falsely indicating a good packet when in fact there is an error. Take the negative of the base-2 logarithm of that probability. You need at least that many bits in your hash.