File format difference between Appx and Zip

Question

In general Microsoft claims appx packages should be basically zips with special content, but actually they are slightly different. If I take an appx package and extract it with 7-Zip, I already get a "Headers-Error" on extraction. if I then pack again the files into a Zip and change the extension, the package is invalid.

I was searching already for more details regarding the format but could not find something. Most answers to appx packaging related questions end with commands interfacing with the MakeAppx.exe. Does anybody know what exactly makes an appx different to a classical zip file and by this can possibly describe how you can create such a package without the use of the MakeAppx.exe ?

Goal is to be able to create appx packages programmatically on my own, without relying on the MakeAppx. Considering that they promote "Building on Standards" and the "Open Packaging Conventions" so much, I guess the difference is subtle. Most other OPC file formats appear to be 100% classical zips. Office documents, NuGet packages, Visual Studio extensions etc. can be easily un-zipped and repackaged without additional logic. Only appx containers seem to have some special header information contained.

Background of my question is that I am the author of a Signing Server used within our company. It accepts certain data formats and creates signed versions of them (like a 'SignTool' services). One functionality is the overwriting of existing signatures with our company signature (e.g. to replace internal testing certificates with the production one). Appx has quite some restrictions when it comes to signing, the manifest must be correct, correct hashing algorithms must be used etc. Microsoft provides the source on how to do the signing programatically, nice. But when it comes to a possible re-packaging there is no API available. I could try interfacing simply with the exe, invoking it, but this brings a lot of drawbacks that I want to prevent (error handling, temp file creation etc.)

Answer 1

Based on the link you provided (Designing a simple and secure app package – APPX) it appears that an appx file is indeed just a zip file that contains a well-defined set of documents.

To check that I downloaded a couple of example appx files. When I ran a unzip test on a Linux box they passed. I don't have 7-Zip.

Digging a bit deeper into the appx files I see that they all contain 8 extra non-standard padding bytes after each data descriptor section in the file. These bytes don't seem to be covered in the zip file specification (APPNOTE.TXT) or in the Open Office specification. That may account for the behavior you are seeing with 7-zip.

The key question for you then is whether the padding bytes are the root cause for you not being able to use 7-Zip to create a valid appx file.

Update

Looked at the makeup of the data descriptor used in appx files in more detail.

Here are some observations based on what I see with the first member in the Package.appx container, shown below

$ unzip -lv Package.appx 
Archive:  Package.appx
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
  444578  Defl:N   104066  77% 2019-06-17 11:01 4d706e98  OfflineFiles/Microsoft.Office.Smartlookup.Vendor_cfe9ce8c6334a9b3191bbc9bfc72ad56.js

Below is a hex dump of the data descriptor for the first member (assuming a standard 32-bit descriptor).

OFFSET BYTES       DESCRIPTION           VALUE
0196F4 50 4B 07 08 STREAMING DATA HEADER 08074B50
0196F8 98 6E 70 4D CRC                   4D706E98
0196FC 82 96 01 00 Compressed Length     00019682
019700 00 00 00 00 Uncompressed Length   00000000
019704 00 06 C8 A2 UNEXPECTED PADDING    ........
       00 00 00 00

1. The overall size of the data descriptor used is correct for a 64-bit data descriptor. (4 byte header + 4 byte CRC32 + 8 byte Compressed Length + 8 byte Uncompressed Length = 24 bytes). Problem is, this is not a well-formed 64-bit zip file.

2. The CRC value is correct.

3. The Compressed Length field in the data descriptor contains the correct 64-bit little-endian value -- 444578 (0x19682).

4. The first 32-bits for the Uncompressed Length field contains the Big-endian encoding of 19682. The second 32-bits are zero. The whole things should be little-endian.

So two issues

1. If the intention was to use a 64-bit data descriptor, then the zip file should conform to the Zip64 extensions. At a minimum there should be a Zip64 Extended Information Extra Field (0x0001) in the local header (see section 4.5.3 in Zip64). The Zip64 central directory headers should also be present.
2. The Uncompressed Length field is partially encoded in Big-endian rather than little-endian.