Optimal endianness conversions and signed integers

Question

I am looking for the fastest way to read numerical values stored in binary files.

I have done some functions that seem to work, but I'd like to get a feedback on whether or not my implementation is good.

Here is how I get a signed integer from a 4-bytes little endian block:

signed long int from4li(char const * const buffer)
{
    signed long int value = 0;

    value += (unsigned char) buffer[3];
    value <<= 8;
    value += (unsigned char) buffer[2];
    value <<= 8;
    value += (unsigned char) buffer[1];
    value <<= 8;
    value += (unsigned char) buffer[0];

    return value;
}

This would also work for unsigned integers, but I've originally made a different implementation for unsigned integers (that fails with signed integers, I don't know exactly why):

unsigned long int fromu4li(char const * const buffer)
{
    unsigned long int value = 0;

    value += (unsigned char) buffer[0] << 8 * 0;
    value += (unsigned char) buffer[1] << 8 * 1;
    value += (unsigned char) buffer[2] << 8 * 2;
    value += (unsigned char) buffer[3] << 8 * 3;

    return value;
}

I am more sure about the conversion from an integer to a little endian string buffer, which I think probably couldn't be optimized further:

void to4li(long int const value, char * const buffer)
{
    buffer[0] = value >> 8 * 0;
    buffer[1] = value >> 8 * 1;
    buffer[2] = value >> 8 * 2;
    buffer[3] = value >> 8 * 3;
}

I also think that it could be even faster usign memcpy, but to use memcpy I have to know the endianness of the host system.

I don't really want to rely on the endianness of the host system, as I think that my code should be independent of the internal data representation of the host system.

So, is this a proper way of doing those conversions, or can I improve my functions?

Answer 1

Using bitwise OR seems to be a good idea, but there is something strange:

After testing, it seems like c0, c1, c2, c3 need to be unsigned chars in order this solution to work. Again, I don't know why:

Take, for example, 0x8080, which is -3264 (signed) or 32896 (unsigned).

Using

char c0 = 0x80;
char c1 = 0x80;

I get:

uint16_t res = (c0 << 0) | (c1 << 8);
// res = 65408 ???

But

uint16_t res = ((unsigned char) c0 << 0) | ((unsigned char) c1 << 8);
// res = 32896 ok

Answer 2

An easier way, and one that avoids undefined behaviour due to bit-shifting a signed integral variable, is to just copy the data bitwise:

int32_t get(char const * const buf)
{
    int32_t result;
    char * const p = reinterpret_cast<char *>(&result);
    std::copy(buf, buf + sizeof result, p);
    return result;
}

This code assumes that the data has the same endianness as the machien. Alternatively, you can use std::copy_backward to reverse the endianness.

This method depends both on the stream data and the host machine endianness, and thus it isn't as elegant as the algebraic solution for unsigned integers, which only depends on the stream data. However, since signed integers are platform dependent anyway, this is supposedly an acceptable compromise.

(Just for comparison, for unsigned integers I'd prefer this machine-independent code:

template <typename UInt>
typename std::enable_if<std::is_unsigned<UInt>::value, UInt>::type
get_from_le(unsigned char * const buf)
{
    UInt result;
    for (std::size_t i = 0; i != sizeof(UInt); ++i)
        result += (buf[i] << (8 * i));
    return result;
}

Usage: auto ui = get_from_le<uint64_t>(buf);.

For the big-endian version, replace [i] by [sizeof(UInt) - i - 1].)