Convert a float array into little/big endian

Question

I am generating binary files for use in Unit Tests which simulate a network. Some of the data sent through the network is in Little Endian or Big Endian mode and I'd like to simulate that data with stub and creating binary files.

So, in other words, instead of using an active network for my unit tests, I would generate binary files that contain the equivelent data I would expect from the network sockets.

I am using Python 2.7 to generate the byte arrays and save them to files, but I am having trouble converting my float array to little endian mode.

from array import array
output_file = open(r"C:\temp\bin.dat", "wb")
float_array = array('d', [1, 1.2, 0.34, 9.8, 0.13, 1.1, 0.88, 72])
float_array.byteswap(); #This doesn't convert it to little endian!
float_array.tofile(output_file)
output_file.close()

So I wondering if anyone knows off how how to manipulate the array such that when I do float_array.tofile() it will write the binary data in little/big endian mode.

This code gives the following output:

Val = 8.6184E-41
Val = 0.0
Val = 4.1897916E-8
Val = 4.172325E-8
Val = -1.9212016E-29
Val = -490.3153
Val = -1.5834067E-23

Which doesn't match the wanted array('d', [1, 1.2, 0.34, 9.8, 0.13, 1.1, 0.88, 72])

When I comment out the byteswap method, I get the following

Val = 0.0
Val = 1.875
Val = 4.172325E-8
Val = 1.9
Val = 1.9023206E17
Val = 1.67
Val = -1.5881868E-23

However, in the real networking mode, it works great. But that doesn't help with unit testing!

I'm confused. Where do you get those Val = 0.0 lines? When I run your code, I get binary files, not anything in that format. And the first 8 bytes are 0, 0, 0, 0, 0, 0, 240, 63 if not swapped, 63, 240, 0, 0, 0, 0, 0, 0 if swapped. Is it possible that the files are being written just fine, but the code you use to read them in and display them and/or re-array-ify them (which you haven't shown us) is wrong? — abarnert

abarnert abarnert · Accepted Answer · 2013-05-08T22:01:10

The code you showed us is creating an array of 8 (64-bit) doubles. The code you didn't show us is reading back an array of 4 (32-bit) floats. Other than that problem, they're both correct.

The solution is to use 'f' here, or 'd' there, so your reading and writing match.

To verify:

>>> float_array = array('d', [1, 1.2, 0.34, 9.8, 0.13, 1.1, 0.88, 72])
>>> float_array.byteswap()
>>> s = float_array.tostring()
>>> new_float_array = array('f')
>>> new_float_array.fromstring(s)
>>> new_float_array
array('f', [8.618405945136922e-41, 0.0, 4.189791624753525e-08, 4.17232506322307e-08, -1.9212016359104883e-29, -490.3153076171875, -1.5834066576694083e-23, -6.352746725754018e-23, -360092172025856.0, -5.209340637631123e-17, -1.5917267736367743e-23, -6.352746725754018e-23, -2.141351348326541e+32, 4.89745031096063e-14, 2.950574046482335e-41, 0.0])

Look familiar? Now take out the byteswap:

>>> float_array = array('d', [1, 1.2, 0.34, 9.8, 0.13, 1.1, 0.88, 72])
>>> s = float_array.tostring()
>>> new_float_array = array('f')
>>> new_float_array.fromstring(s)
>>> new_float_array
array('f', [0.0, 1.875, 4.17232506322307e-08, 1.899999976158142, 1.902320558192722e+17, 1.6699999570846558, -1.5881868392106856e-23, 2.5562498569488525, 9.121204334167384e-33, 1.5049999952316284, -1.5881868392106856e-23, 1.8874999284744263, -71.68000030517578, 1.84499990940094, 0.0, 3.28125])

Note that I'm using tostring/fromstring for convenience, instead of tofile/fromfile, which means I don't specify a count. But if you specify a count of 8, you're obviously going to read the first 4 doubles as 8 floats. And if you then format them to look like your output:

>>> print '\n'.join('Val = {}'.format(val) for val in new_float_array[8])
Val = 8.6184E-41
Val = 0.0
Val = 4.1897916E-8
Val = 4.172325E-8
Val = -1.9212016E-29
Val = -490.3153
Val = -1.5834067E-23

Convert a float array into little/big endian

1 Answers