How do I parse a two's-complement fixed-point number from a primitive integer type with an arbitrary split of integer and fraction?

Question

I am working on parsing OpenType font files, and need to parse (and write) two kind of fixed-point numbers:

16-bit signed fixed number with the low 14 bits of fraction (2.14)
32-bit signed fixed-point number (16.16)

I assume that, in the end, it should be cast to/from f32

The F2DOT14 format consists of a signed, 2’s complement integer and an unsigned fraction. To compute the actual value, take the integer and add the fraction.

Examples of 2.14 values are:

Decimal Value   Hex Value   Integer     Fraction
1.999939        0x7fff      1           16383/16384
1.75            0x7000      1           12288/16384
0.000061        0x0001      0           1/16384
0.0             0x0000      0           0/16384
-0.000061       0xffff      -1          16383/16384
-2.0            0x8000      -2          0/16384

I have a solution that works but only for 2.14 values:

fn from(number: u16) -> f32 {
    let mut int = (number >> 14) as f32;
    if int > 1f32 {
        int -= 4f32;
    }
    let frac = (number & 0b11_1111_1111_1111) as f32 / 16384 as f32;
    int + frac
}

Because the integer value should be [-2, 2), I subtract 4 if the parsed integer is higher than 1 to achieve the negative numbers.

I am looking for a way of doing this for a any possible split of fixed-point numbers (like 2.14, 16.16, 3.5, 24.40, etc.) inside the standard range of Rust integer primitive types (u16, u32, u64, etc.).

Nirlah Nirlah · Accepted Answer · 2018-12-24T18:24:52

Was able to solve my issue, here is an example of parsing 16-bit fixed point number:

use std::mem::size_of;

fn from_u16(raw: u16, frac_count: usize) -> f32 {
  let bit_count = size_of::<u16>() * 8;
  let int_count = bit_count - frac_count;

  let unsigned = (raw >> frac_count) as isize;
  let sign_bit = unsigned >> (int_count - 1) & 1;
  let high_bits = if sign_bit == 1 { -1 } else { 0 };
  let signed = high_bits << int_count | unsigned as isize;

  let mut mask = 0;
  for i in 0..=frac_count {
    mask = mask << i | 1;
  }

  let frac = (raw & mask) as f32 / (1 << frac_count) as f32;
  signed as f32 + frac
}

How do I parse a two's-complement fixed-point number from a primitive integer type with an arbitrary split of integer and fraction?

1 Answers