Octave precision fluctuates

Question

According to Octave's documentation for format, the default for short is:

By default, Octave displays 5 significant digits in a human
readable form (option ‘short’ paired with ‘loose’ format for
matrices). If format is invoked without any options, this default
format is restored. 

short

    Fixed point format with 5 significant figures in a field that
    is a maximum of 10 characters wide. (default).

    If Octave is unable to format a matrix so that columns line up
    on the decimal point and all numbers fit within the maximum
    field width then it switches to an exponential ‘e’ format.

short e long e

    Exponential format. The number to be represented is split
    between a mantissa and an exponent (power of 10). The mantissa
    has 5 significant digits in the short format and 15 digits in
    the long format. For example, with the ‘short e’ format, pi is
    displayed as 3.1416e+00.

This is what I get:

octave:8> pi*1e3

   ans =  3141.6

octave:9> pi*1e4

   ans =  31415.92654

I can see how short yields the 1st output, but not how the 2nd output precision is arrived at. Can anyone please explain?

Steps taken

I think I'm already at default conditions other than for 1 statement in my ~/.octaverc: format compact. I only use Octave in CLI mode via octave --no-window-system --silent. The results of [a b c]=format and ver are:

octave:1> [a b c] = format
a = short
b = compact
error: element number 3 undefined in return list
octave:1> ver
----------------------------------------------------------------------
GNU Octave Version: 5.2.0 (hg id: eb46a9f47164)
GNU Octave License: GNU General Public License
Operating System: CYGWIN_NT-10.0-19041 3.1.7-340.x86_64 2020-08-22 17:48 UTC x86_64

Version 5.2 is the latest one from Cygwin.

I get the same results using the GUI (octave --gui), both in terms of the fluctuating precision and the outputs of format and ver.

Nick J Nick J · Accepted Answer · 2021-03-23T14:55:07

I am unable to recreate your result using Octave 6.2.0. Are you sure you reset to default conditions before getting the above output? you can call format with up to three output variables to verify your current state. E.g.:

octave:6> format
octave:7> [a b c] = format
a = short
b = loose
c = lowercase
octave:8> pi*1e3
ans = 3141.6
octave:9> pi*1e4
ans = 3.1416e+04

I get the same results in the CLI and the GUI.

Octave precision fluctuates

1 Answers