21
votes

The What's New for Delphi XE2 contains the following.

Packed Now Forces Byte Alignment of Records

If you have legacy code that uses the packed record type and you want to link with an external DLL or with C++, you need to remove the word "packed" from your code. The packed keyword now forces byte alignment, whereas in the past it did not necessarily do this. The behavior change is related to C++ alignment compatibility changes in Delphi 2009.

I don't understand this. I'm struggling with this point: whereas in the past it did not necessarily do this. What I cannot reconcile is that packed has always resulted in byte alignment of records to the best of my knowledge. Can anyone give an example of a packed record that is not byte aligned? Obviously this would have to be in an earlier version.

Why do the docs say "if you want to link with an external DLL or with C++, you need to remove the the word packed from your code"? If the external code uses #pragma pack(1) then what are we to do if packed is off limits?

What about the $ALIGN directive? Are {$A1} and {$A-} equivalent to packed or is there some extra meaning with packed?

It seems that I'm missing something and would appreciate it if somebody could explain this. Or is the documentation just really poor?

Update

I'm reasonably convinced that the documentation is referring to alignment of the record itself rather than the layout of the record. Here's a little program that shows that the the use of packed on a record forces the alignment of the record to be 1.

program PackedRecords;
{$APPTYPE CONSOLE}
type
  TPackedRecord = packed record
    I: Int64;
  end;

  TPackedContainer = record
    B: Byte;
    R: TPackedRecord;
  end;

  TRecord = record
    I: Int64;
  end;

  TContainer = record
    B: Byte;
    R: TRecord;
  end;

var
  pc: TPackedContainer;
  c: TContainer;

begin
  Writeln(NativeInt(@pc.R)-NativeInt(@pc.B));//outputs 1
  Writeln(NativeInt(@c.R)-NativeInt(@c.B));//outputs 8
  Readln;
end.

This produces the same output on Delphi 6, 2010, XE and XE2 32 bit and XE 64 bit.

4
Keep in mind that there are two alignments in play: the alignment of the fields within the record (which is what packed affects) and the alignment of the record itself, say, in an array of these records. Historically, I believe packed did not affect the alignment of the record itself and that was later changed, or the other way around.dthorpe
Desperate googling revealed your question and the article it referred to. I don't get it either, I'm all agog for an example. Unpacked array in a packed record, or may be a variant record? On my linux box at the moment otherwise I'd be intrigued enough to experiment.Tony Hopkinson
@dthorpe Hi Danny. I'm aware of the difference between layout and alignment. The Delphi docs nowadays document that packed records have alignment 1. But my experience is that this has always been so. It was the case in D6 for sure. So packed does affect both layout and alignment. Are you saying that if you go far enough back, i.e. D1 say, that packed only affected layout?David Heffernan
I recall that it came up as an issue in Kylix that required quite a bit of internal discussion, but I don't recall the details of which way things went.dthorpe

4 Answers

5
votes

The latest updates to the documentation have removed all of the text on which this question was based. My conclusion is that the original text was simply a documentation error.

2
votes

As I am not the Delphi compiler guy I can also mostly guess as others did: The record alignment might not be meant for aligning the members inside the record, but the record itself instead.

If you declare a record variable it is aligned to some address in memory, that is most likely aligned on a 4-byte boundary. This has been the case (as tested in D2007) for packed and unpacked records.

Now in XE2 a packed record is placed in a 1-byte boundary, while unpacked records are placed on some even boundary, which can be controlled by the align keyword. Like this:

type
  TRecAligned = record
    b1: byte;
    u1: uint64;
  end align 16;

  TRecPackedAligned = packed record
    b1: byte;
    u1: uint64;
  end align 16;

The packed record is still aligned on a 1-byte boundary while the unpacked record is aligned to a 16-byte boundary.

As I said, it is only a guess. The wording of the Embarcadero quote isn't that much clear on the suject.

1
votes

As far as I remember, record used to be packed since a version of the compiler around Delphi 5-6.

Then, for performance reasons, plain record fields were aligned, according to the settings in the project option. If you define a packed record there won't be any alignment within the record.

The text you are quoting seems related not specifically to XE2, but to a Delphi 2009 change. See this Blog entry for historical purpose.

I guess that "'Packed' Now Forces Byte Alignment of Records" refers to the Delphi 2009 {$OLDTYPELAYOUT ON} feature - which may have some implementation issue before XE2. I agree with you: it sounds like a documentation issue.

-1
votes

pascal has always supported packed structures which are probably easiest explained using an example

Lets say you have two arrays x and y, and that we are using a 16 bit processor. That is, the 'word' size of the processor is 16 bits.

myrec = 
record
  b : byte;
  i : integer;
end;

x : array[1..10] of myrec;
y : packed array[1..10] of myrec;

Pascal only tells you that you have 10 elements to work with, each holding a 3 bytes of information (lets assume integer is the old 16 bit variety). It actually says nothing about how that information is stored. You may assume the array is stored in 30 consecutive bytes, however this is not necessary true and is completely compiler dependent (a really good reason to avoid pointer mathematics).

A complier may well put a dummy byte in between the field values b and i, in order to ensure that both b and i fall on word boundaries. In this instance the structure will take a total of 40 bytes.

The 'packed' reserved word instructs the compiler to optimise for size over speed, whereas leaving packed out, the compiler will generally optimise for speed over size. In this case the structure would be optimised and could take only 30 bytes.

Again I say 'could' because it is still compiler dependent. The 'packed' keyword just says to pack the data closer together, but doesn't actually say how. Some compilers therefore simply ignored the keyword, whereas others used packed to disable word alignment.

In the case of the latter, what typically could happen is that each element is aligned to the word size of the processor. You need to realise here that the word size of the processor is typically the register size, not simply '16 bits' as it has commonly come to mean. In the case of a 32 bit processor for example the word size is 32 bits (even though we generically define words as 16 bits - this is just a historical throwback) In a sense it is it the 'memory' equivalent of a disk sector.

By packing data, you could potentially have values such as integer crossing over word boundaries therefore requiring more memory manipulation (much like needing to read two disk sectors if a record crosses a sector boundary).

So basically what the change means, is the delphi is now fully using the packed keyword and forcing byte alignment (over word alignment) all of the time.

Hope this is enough to explain it. It is actually a bigger topic that I can reasonably answer in a few paragraphs.


UPDATE, continuation of comment thread below...

DavidH> I cannot imagine that you mean that the compiler is capable of producing different output given identical input

Yes David, that is exactly what I am saying. Not necessarily from one compile to the next on the same compiler, but certainly between versions of compiler, brands of compiler and different platforms.

TonyH > I think most of us always assumed packed = byte aligned, so now we are scratching our heads for a scenario where it doesn't. @David Heffernan is asking if anyone knows one

Tony, you have hit the nail on the head here. You have assumed that packed works like a compiler directive that specifies how the data memory is organised. This is not what it means. A better analogy is to consider the optimise directive for code generation. You know the code is being optimised, but you don't necessarily specifically what the underlying outputted code will be, nor do you need to.

I can give you plenty of examples. Lets say you have the following (I am using an array, it could apply just as easily to a record)

myarray = packed array[1..8] of boolean

If you then print a SizeOf(myarray) what would you expect? The answer is that delphi does not guarantee that packed is implemented in any particular way. So therefore the answer could be 8 bytes (if you are byte aligning each element). It is just as valid for delphi to pack these as bits, and so the entire array could fit into 1 byte. It could be 16 bytes if the compiler decides not to optimise on byte boundaries and is running on a 16 bit architecture. Yes it would be less efficient in terms of speed, but the whole point of packing is to optimise for size over speed. Whatever it does is invisible to the developer as long as they simply access the array elements and don't make any assumptions and try their own pointer mathematics to access the underlying data.

Similarly you cannot even guarantee the internal organisation of a primitive - it is platform dependent, not compiler dependent. E.g. If you have worked across multiple architectures you will be aware of issues surrounding how many bits make up an integer; whether or not the bytes within that integer are stored in byte order, or reverse byte order.

These types of architectural issues are exactly why various techniques such as comma delimited file, xml, etc have been developed. In order to provide a reliable common platform.

To summarise, David the issue is that you are actually asking the wrong question. If you consider the Embarcadero quote in the context of what I have said, you will see that it is consistent with what I am saying.