Decoding Scrambed Binary Data from GPS with Python

Question

I've got a raw binary file (1 KB↓) that is a serial data dump of a GPS stream (along with some associated metadata). I'm specifically trying to pull a value out of the binary file that represents the GPS time; I know its offset and width in the file (10 and 8 bytes respectively, with a total frame width of 28 bytes) but it's encoded in a very weird way as described in the quote below.

What's the most Pythonic way to read this data (into a list or array)?

GPS TIME - GPS Sensor time (time of week in seconds, starting at Saturday 2400 hours/ Sunday 0000 hours) if GPS Time Valid Message 3500 is set to 1, otherwise SDN500 system time since power up is reported. Data words are in the order 2, 1 (MSW), 4 (LSW), 3.

A message word length is 16 bits on the SDN500–HV interface. However, the SDN500–HV protocol, which uses a standard Universal Asynchronous Receiver Transmitter (UART), transmits data in 8-bit groups (bytes). This means that two bytes are required in order to make up one message word.

A byte of information is transmitted as a sequence of 11 bits: one start bit, 8 bits of data (least significant bit (LSB) first), one parity bit (odd), and one stop bit. For each 16-bit data word, the least significant byte is transmitted first, followed by the most significant byte. Integer and floating point data types consisting of more than one word are transmitted from the lowest numbered word to the highest numbered word. The one exception to this rule is the time tag, which is output in words 6-9 of each HV output message. The four 16-bit data words are in the following order: 2,1,4,3, where 1 represents the most significant word and 4 the least significant word. Each word is separately byte-reversed.

If you're more proficient in another language (like Matlab or C), I'd still love to see an example snippet of code that would read this data. This would help me understand how to better approach reading it in Python. — Nick
Hopefully someone else will give you better info, but the docs.python.org/2/library/struct.html module is where you want to start looking for binary file i/o. I only used it once for a short while, so can't help much more. — JL Peyret
@JLPeyret yeah I've been messing with struct.unpack variations and a hex editor all day trying to get reasonable values out of this but haven't figured out the right way to read the MSB/LSB structure. Though I was able to read some of the other fields that way. — Nick

Joran Beasley Joran Beasley · Accepted Answer · 2016-03-25T01:11:43

start by opening the file

fin = open("20160128t184727_pps","rb")

then read in a frame

def read_frame(f_handle):
    frame = f_handle.read(28) # 28 byte frame size
    start_byte = 10
    end_byte = 18 # 4 words each word is 2 bytes
    timestamp_raw = frame[start_byte:end_byte]
    timestamp_words = struct.unpack(">HHHH",timestamp_raw)

I could probably help more but I dont understand where the timestamp startbyte and endbyte is from your description as it does not seem to match the description you quoted ... I also do not know what the expected output value is ...if you provided those details I could probably help more

Decoding Scrambed Binary Data from GPS with Python

1 Answers