File size calculation, Int64, and differences between 32bit and 64bit

Question

I had problems with the following code:

var
  FileSize : Int64;
...
FileSize := Info.nFileSizeLow or (Info.nFileSizeHigh shl 32);

I expected it to work because of the Int64 type of the left side of the assignment. But it does not. The partial calculation containing the shl seems to produce an overflow.

So I changed it to:

FileSize := Info.nFileSizeLow or (Int64 (Info.nFileSizeHigh) shl 32);

which works on my 32 bit operating system, but does not work on Vista 64 bit!

Finally,

FileSize := Info.nFileSizeHigh;
FileSize := FileSize shl 32;
FileSize := Info.nFileSizeLow or FileSize;

works on both systems.

Can someone explain the differences in these three versions?

Answer 1

Generally speaking, the type of the expression a * b, where a and b are of type Integer and * is an operator that applies to Integer, is an integer type with the same range as Integer. (I say generally, as an exception is /.) In order for an operator to use 64-bit operations, one or more of the operands must have a range that is only expressible with a 64-bit type. That should cause all the operands to be promoted to 64-bit, and a 64-bit operation performed.

The fact that the left hand side of an assignment is a 64-bit location generally has no effect on the interpretation and typing of the expression on the right hand side of the assignment operator. This is the way it is in almost all languages that I'm aware of that have statically dispatched 32-bit and 64-bit operator overloads (as opposed to polymorphically dispatched operators on arbitrary precision integers or numeric towers etc.); making things behave otherwise would be very surprising behaviour.

For example, arguments to procedure calls are effectively implicit assignments to the parameters. If the left hand side of an assignment could change the interpretation of the expression on the right, we would not know how to interpret the argument to a procedure call without already knowing the definition:

var a, b: Integer;
// ...
P((a shl 16) or b); // 32-bit operation or 64-bit operation?

I do not know why you are seeing different behaviour with your second and third versions of the code. As far as I can see, they should be interpreted the same, and in my tests, they are interpreted the same. If you could provide sample code that works on 32-bit Windows but fails on 64-bit Windows, I could investigate further.

Answer 2

Actually, this is pretty well documented in Delphi 7's help file, under "Integer types":

In general, arithmetic operations on integers return a value of type Integer--which, in its current implementation, is equivalent to the 32-bit Longint. Operations return a value of type Int64 only when performed on one or more Int64 operand. Hence the following code produces incorrect results.

The code example provided:

var
  I: Integer;
  J: Int64;
  ...
I := High(Integer);
J := I + 1;

To get an Int64 return value in this situation, cast I as Int64:

 ...
J := Int64(I) + 1;

Answer 3

First of all FileSize must be defined as UInt64 and not Int64...

UInt64 (not available in early Delphi versions) is an unsigned 64 bit integer, aka a QWORD. This is the expected type for the FileSize (you won't expect a negative file size, won't you?).

IMHO you could have coded - using UInt64 because we don't want to have some values reported as negative:

FileSize := UInt64(Info.nFileSizeLow) or (UInt64(Info.nFileSizeHigh) shl 32));

But under Delphi 7 it produces the same exact code as yours.

FileSize := Info.nFileSizeLow or (Int64(Info.nFileSizeHigh) shl 32));

So there is perhaps some compiler regression. Could you take a look at the asm generated code (step debugger then Alt+F2), and see if there is a difference. But it's unlikely...

In all cases, here is a better (and faster) code:

with Int64Rec(FileSize) do
begin
  Lo := Info.nFileSizeLow;
  Hi := Info.nFileSizeHigh;
end;

The official MSDN documentation states about the WIN32_FIND_DATA Structure:

nFileSizeHigh: The high-order DWORD value of the file size, in bytes.

This value is zero unless the file size is greater than MAXDWORD.

The size of the file is equal to (nFileSizeHigh * (MAXDWORD+1)) + nFileSizeLow.

nFileSizeLow: The low-order DWORD value of the file size, in bytes.

Here is the resulting code:

FileSize := UInt64(Info.nFileSizeLow)+(UInt64(Info.nFileSizeHigh)*UInt64(1 shl 32));

Quite a funny definition, indeed...

Answer 4

This is not really an answer, but it's too long for a comment.

I noticed Delphi gets confused when the result of an expression is to be written into a 64 bit variable, but the operands are 32 bit. I ran into this bug when I was implementing a hash function returning an 64 bit number. Your third variant works because you're first assigning the 64 bit variable, helping Delphi figure out it really needs to do 64 bit arithmetic.

I'm tempted to say both variants (1) and (2) are actually failing because Delphi generates 32 bit arithmetic and then assignes the result to the 64 bit variable. I'm tempted to say the variant that works well on your 32 bit machine benefits from some sort of "unlucky non-failure" (ie: the code is bad, but none the less it produces good results for the given test). The trouble is, COMPILED code doesn't change when moved from a 32bit machine to a 64 bit machine. If the code is the same, the input is the same, you'd have to pin the error on the CPU, but you know you didn't find an bug in your CPU, so you have to fall back and re-think your tests, or pin it on the "unluck non-failure".

Answer 5

test on Delphi 7 and version 2 is OK. Must be bug of later version