CRC error detection and undetected error probabilities

1
votes

If we have a large file, let 's say 1 Petabyte, what's the best CRC that can detect all the errors? Is 32bits enough?

I also heard that undetected error rate (packet or chunk) is= BitR* BER * 0.5^k which K is the FSC of the CRC. in CRC 32 k is 31

I want to know if we have bigger packets or smaller ones how this will affect the CRC... from this equation, it is not affecting at all.

1
crccrc32crc16error-detectioncrc64
A 32 bit CRC can detect a single 32 bit burst in a message up to 2^31-1 bits. However, only a few specific 32 bit CRC's can detect up to 7 random bit errors, and this is limited to messages up to 1024 bits, including the CRC, or 992 data bits + 32 CRC bits. Take a look 32 bit CRC Zoo and look at the notes for an explanation. Detecting 9 bit errors is limited to message length 100 data + 32 CRC bits. – rcgldr
Link to example of analysis of crc undetected error – rcgldr
Usually as in that link the odds of failure include the probability of an error: 1 - (1-BER)^n, where BER is bit error rate, and n is number of bits (which is calculating 1 minus the probability of zero bits in error). – rcgldr

1 Answers

2
votes

"Enough" depends on your tolerance for false positives. Given a CRC, or any other good hash, what probability would be acceptable to you of not detecting an error in any one message?

If you call that p, then the length of the CRC or hash you need in bits, n is n = ceiling(–log2(p)).

Note that this does not depend on the length of message. Kilobytes, Exabytes, whatever. Except to the extent that the expense of creating, sending, or storing the message impacts the p you would find acceptable.

For particularly expensive data or data sent over untrustworthy channels, you may want to consider error correcting codes, such as Reed-Solomon or BCH codes.