External signing PDF with iTextSharp - altered/corrupted document

Question

The goal is to implement a PDF signing process in which the server (.NET Core service) provides the hash to be signed to the client on request (Electron). The client then signs the given hash using a private key obtained from a smart card through a PKCS#11 interface. The signature is then sent back to the server for attaching into the PDF file using iTextSharp.

The process of signing the hash with the smart card token is pretty much straightforward with node-webcrypto-p11 at the moment (with a lot of trial and error needed to get here). The algorithm used is RSASSA-PKCS1-v1_5. I can successfully sign the hash and verify it afterwards.

I recently built on my previous implementation with the help of External signing PDF with iTextsharp (3) where I used getAuthenticatedAttributeBytes for getting the hash to be signed (as suggested by mkl).

On reviewing the signature in Acrobat Reader, I am presented with the dreaded document altered/corrupted, same as the OP pgkdev. As mentioned above the client-side is straightforward with the signing process and I do not suspect any problems arising there (I am open for review of that though).

pgkdev referred to Priyanka's question, where I found out that I am probably having issues regarding the 2-step process of signing the document in which the hash values are no longer the same.

If you check Grazina's question we can see that such an implementation is successful, when you do the process in one step.

mkl further mentions a way to do it successfully in 2 steps, but I am missing some more explanation on how exactly to achieve that.

Note: There is no way for me (that I know of) to do what I want in 1 step, as the signing is initiated by a client in an Electron app.

Clicking on Certificate details shows my full certificate details.

private const string SIG_FIELD_NAME = "sigField1";

    private byte[] GetPDFHash(string pdfFilePath, byte[] certificateValue)
    {
        var preparedSigPdfFilePath = $"{pdfFilePath}.tempsig.pdf";

        //Get certificates chain from certificate value
        ICollection<X509Certificate> certificatesChain = GetCertificatesChain(certificateValue);

        byte[] hash = CreatePDFEmptySignature(pdfFilePath, preparedSigPdfFilePath, certificatesChain);
        return hash;
    }

    private void SignPDFHash(string pdfFilePath, byte[] hash, byte[] signedHash, byte[] certificateValue)
    {
        var preparedSigPdfFilePath = $"{pdfFilePath}.tempsig.pdf";
        var signedPdfFilePath = $"{pdfFilePath}.signed.pdf";

        //Get certificates chain from certificate value
        ICollection<X509Certificate> certificatesChain = GetCertificatesChain(certificateValue);

        CreateFinalSignature(preparedSigPdfFilePath, signedPdfFilePath, hash, signedHash, certificatesChain);
    }

    private byte[] CreatePDFEmptySignature(string pdfFilePath, string preparedSigPdfFilePath, ICollection<X509Certificate> certificatesChain)
    {
        byte[] hash;

        using (PdfReader reader = new PdfReader(pdfFilePath))
        {
            using (FileStream baos = System.IO.File.OpenWrite(preparedSigPdfFilePath))
            {
                PdfStamper pdfStamper = PdfStamper.CreateSignature(reader, baos, '\0', null, true);
                PdfSignatureAppearance sap = pdfStamper.SignatureAppearance;
                sap.SetVisibleSignature(new Rectangle(36, 720, 160, 780), 1, SIG_FIELD_NAME);

                sap.Certificate = certificatesChain.First();

                var externalEmptySigContainer = new MyExternalEmptySignatureContainer(PdfName.ADOBE_PPKMS, PdfName.ADBE_PKCS7_DETACHED, preparedSigPdfFilePath, certificatesChain);

                MakeSignature.SignExternalContainer(sap, externalEmptySigContainer, 8192);

                hash = externalEmptySigContainer.PdfHash;
            }
        }
        return hash;
    }

    private void CreateFinalSignature(string preparedSigPdfFilePath, string signedPdfFilePath, 
        byte[] hash, byte[] signedHash, ICollection<X509Certificate> certificatesChain)
    {
        using (PdfReader reader = new PdfReader(preparedSigPdfFilePath))
        {
            using (FileStream baos = System.IO.File.OpenWrite(signedPdfFilePath))
            {
                IExternalSignatureContainer externalSigContainer = new MyExternalSignatureContainer(hash, signedHash, certificatesChain);
                MakeSignature.SignDeferred(reader, SIG_FIELD_NAME, baos, externalSigContainer);
            }
        }
    }

    public class MyExternalEmptySignatureContainer : ExternalBlankSignatureContainer
    {
        public string PdfTempFilePath { get; set; }
        public byte[] PdfHash { get; private set; }

        public ICollection<X509Certificate> CertificatesList { get; set; }

        public MyExternalEmptySignatureContainer(PdfName filter, PdfName subFilter, string pdfTempFilePath,
            ICollection<X509Certificate> certificatesChain) : base(filter, subFilter)
        {
            PdfTempFilePath = pdfTempFilePath;
            CertificatesList = certificatesChain;
        }

        override public byte[] Sign(Stream data)
        {
            byte[] sigContainer = base.Sign(data);

            //Get the hash
            IDigest messageDigest = DigestUtilities.GetDigest("SHA-256");
            byte[] messageHash = DigestAlgorithms.Digest(data, messageDigest);

            #region Log
            var messageHashFilePath = $"{PdfTempFilePath}.messageHash-b64.txt";
            System.IO.File.WriteAllText(messageHashFilePath, Convert.ToBase64String(messageHash));
            #endregion Log

            //Add hash prefix
            byte[] sha256Prefix = { 0x30, 0x31, 0x30, 0x0d, 0x06, 0x09, 0x60, 0x86, 0x48, 0x01, 0x65, 0x03, 0x04, 0x02, 0x01, 0x05, 0x00, 0x04, 0x20 };
            byte[] digestInfo = new byte[sha256Prefix.Length + messageHash.Length];
            sha256Prefix.CopyTo(digestInfo, 0);
            messageHash.CopyTo(digestInfo, sha256Prefix.Length);

            #region Log
            var messageHashWithPrefixFilePath = $"{PdfTempFilePath}.messageHash-with-prefix-b64.txt";
            System.IO.File.WriteAllText(messageHashWithPrefixFilePath, Convert.ToBase64String(digestInfo));
            #endregion Log

            var sgn = new PdfPKCS7(null, this.CertificatesList, "SHA256", false);
            var authenticatedAttributeBytes =
                sgn.getAuthenticatedAttributeBytes(messageHash, null, null, CryptoStandard.CMS);

            PdfHash = authenticatedAttributeBytes;

            return sigContainer;
        }
    }

    public class MyExternalSignatureContainer : IExternalSignatureContainer
    {
        public byte[] Hash { get; set; }
        public byte[] SignedHash { get; set; }
        public ICollection<X509Certificate> CertificatesList { get; set; }

        public MyExternalSignatureContainer(byte[] hash, byte[] signedHash, ICollection<X509Certificate> certificatesList)
        {
            Hash = hash;
            SignedHash = signedHash;
            CertificatesList = certificatesList;
        }

        public byte[] Sign(Stream data)
        {
            PdfPKCS7 sgn = new PdfPKCS7(null, this.CertificatesList, "SHA256", false);
            sgn.SetExternalDigest(this.SignedHash, null, "RSA");    
            return sgn.GetEncodedPKCS7(this.Hash, null, null, null, CryptoStandard.CMS);         
        }

        public void ModifySigningDictionary(PdfDictionary signDic)  {   }
    }

    private ICollection<X509Certificate> GetCertificatesChain(byte[] certByteArray)
    {
        ICollection<X509Certificate> certChain = new Collection<X509Certificate>();

        X509Certificate2 cert = new X509Certificate2(certByteArray);

        X509Certificate regularCert = new X509CertificateParser()
            .ReadCertificate(cert.GetRawCertData());

        certChain.Add(regularCert);

        return certChain;
    }

EDIT: Signed PDF

EDIT: Adjusted CreateFinalSignature to use messageHash which was saved into a .txt file. The result is the same. Signed PDF

private void CreateFinalSignature(string preparedSigPdfFilePath, string signedPdfFilePath, byte[] signedHash, ICollection<X509Certificate> certificatesChain)
    {
        var messageHashFilePath = $"{preparedSigPdfFilePath}.messageHash-b64.txt";
        string hashString = System.IO.File.ReadAllText(messageHashFilePath);
        byte[] hash = Convert.FromBase64String(hashString);

        using (PdfReader reader = new PdfReader(preparedSigPdfFilePath))
        {
            using (FileStream baos = System.IO.File.OpenWrite(signedPdfFilePath))
            {
                IExternalSignatureContainer externalSigContainer = new MyExternalSignatureContainer(hash, signedHash, certificatesChain);
                MakeSignature.SignDeferred(reader, SIG_FIELD_NAME, baos, externalSigContainer);
            }
        }
    }

Hashes are identical, as shown below. I put some breakpoints to try and catch the values before saving and after reading from the file.

Byte array before saving:

[0] = {byte} 133
[1] = {byte} 170
[2] = {byte} 124
[3] = {byte} 73
[4] = {byte} 225
[5] = {byte} 104
[6] = {byte} 242
[7] = {byte} 79
[8] = {byte} 44
[9] = {byte} 52
[10] = {byte} 173
[11] = {byte} 6
[12] = {byte} 7
[13] = {byte} 250
[14] = {byte} 171
[15] = {byte} 50
[16] = {byte} 226
[17] = {byte} 132
[18] = {byte} 113
[19] = {byte} 31
[20] = {byte} 125
[21] = {byte} 174
[22] = {byte} 53
[23] = {byte} 98
[24] = {byte} 68
[25] = {byte} 117
[26] = {byte} 102
[27] = {byte} 191
[28] = {byte} 109
[29] = {byte} 180
[30] = {byte} 88
[31] = {byte} 133

Byte array read from .txt file in CreateFinalSignature:

[0] = {byte} 133
[1] = {byte} 170
[2] = {byte} 124
[3] = {byte} 73
[4] = {byte} 225
[5] = {byte} 104
[6] = {byte} 242
[7] = {byte} 79
[8] = {byte} 44
[9] = {byte} 52
[10] = {byte} 173
[11] = {byte} 6
[12] = {byte} 7
[13] = {byte} 250
[14] = {byte} 171
[15] = {byte} 50
[16] = {byte} 226
[17] = {byte} 132
[18] = {byte} 113
[19] = {byte} 31
[20] = {byte} 125
[21] = {byte} 174
[22] = {byte} 53
[23] = {byte} 98
[24] = {byte} 68
[25] = {byte} 117
[26] = {byte} 102
[27] = {byte} 191
[28] = {byte} 109
[29] = {byte} 180
[30] = {byte} 88
[31] = {byte} 133

EDIT: Hashing authenticatedAttributeBytes and then returning that hash to be signed by the client.

Tried 3 different ways of hashing, with same result:

PdfHash = DigestAlgorithms.Digest(new MemoryStream(authenticatedAttributeBytes), messageDigest)

PdfHash = SHA256.Create().ComputeHash(authenticatedAttributeBytes)

PdfHash = SHA256Managed.Create().ComputeHash(authenticatedAttributeBytes)

Usage of GetPDFHash

byte[] bytesToSign = GetPDFHash(Path.Combine(_configuration["documentFileSystemStore:DocumentFolder"], "SignMeMightySigner.pdf"), Convert.FromBase64String(dto.base64certificateValue));

Usage of SignPDFHash

SignPDFHash(Path.Combine(_configuration["documentFileSystemStore:DocumentFolder"], "SignMeMightySigner.pdf"),Convert.FromBase64String(dto.base64signature), Convert.FromBase64String(dto.base64certificateValue));

EDIT (29.3.2020): I have checked my client side and can't find anything problematic. I choose the RSASSA-PKCS1-v1_5 alg to get the signature and verify it successfully afterwards. In some other questions I found that it could be a problem with transferring the byte array between the server and client, but I have checked and the values are the same, both base64 and the byte array.

Decided to open the PDF in a text editor and compare it to a regularly signed PDF (same text content, just signed directly through Adobe Reader).

What bothers and worries me, is that the PDF signed with iText is missing a huge chunk of "text" inside, that the directly signed one has.

Is there anything else I can provide for possible further analysis? I have seen a trend of people on Stack Overflow not being able to get past this problem, some even giving up on it completely. I do not want and can not do that and want to get to the bottom of it.

Directly signed through Adobe Reader Deferred signing with iText

EDIT 30.3.2020: As mentioned above I hash the AuthenticatedAttributeBytes

PdfHash = SHA256Managed.Create().ComputeHash(authenticatedAttributeBytes);

AuthenticatedAttributeBytes

49 75 48 24 6 9 42 134 72 134 247 13 1 9 3 49 11 6 9 42 134 72 134 247 13 1 7 1 48 47 6 9 42 134 72 134 247 13 1 9 4 49 34 4 32 122 115 111 54 139 240 60 168 176 67 64 158 55 107 233 48 77 220 19 208 139 187 42 1 141 149 20 241 151 80 31 79

AuthenticatedAttributeBytes - hashed

33 25 105 92 244 51 72 93 179 135 158 84 249 178 103 91 236 247 253 35 232 124 169 112 108 214 63 206 206 2 88 107

(returned to client) AuthenticatedAttributeBytes - hashed & base64 encoded

IRlpXPQzSF2zh55U+bJnW+z3/SPofKlwbNY/zs4CWGs=

Hash signed (signature)

76 13 184 229 123 212 2 8 140 24 34 88 95 31 255 142 105 220 204 186 172 110 61 75 156 44 185 62 81 209 238 226 67 133 115 247 76 24 182 144 38 164 71 92 124 140 77 16 212 43 52 156 173 90 163 116 0 124 119 119 103 8 12 74 147 1 207 51 156 104 52 231 112 125 115 140 28 105 160 117 235 199 224 166 30 220 111 35 165 49 18 85 253 194 112 254 142 117 46 58 87 13 110 161 151 228 95 238 115 171 70 117 203 103 204 222 233 42 163 37 105 91 177 117 190 238 135 137 162 6 54 125 108 64 148 219 7 198 93 117 12 164 130 123 213 197 233 173 145 77 209 11 166 91 29 137 142 25 20 96 90 130 251 169 234 9 44 245 230 20 46 243 254 98 179 98 148 87 104 151 228 246 231 23 94 134 144 84 177 219 235 90 11 130 33 139 94 155 73 112 60 88 53 150 59 49 184 100 210 82 32 71 66 168 21 167 91 141 94 239 221 156 96 23 132 147 237 15 237 232 112 214 224 61 117 46 143 208 41 64 13 128 44 69 135 172 113 58 8 85 5 176 192 254 107 92

(received from client) Hash signed (signature) - base64

TA245XvUAgiMGCJYXx//jmnczLqsbj1LnCy5PlHR7uJDhXP3TBi2kCakR1x8jE0Q1Cs0nK1ao3QAfHd3ZwgMSpMBzzOcaDTncH1zjBxpoHXrx+CmHtxvI6UxElX9wnD+jnUuOlcNbqGX5F/uc6tGdctnzN7pKqMlaVuxdb7uh4miBjZ9bECU2wfGXXUMpIJ71cXprZFN0QumWx2JjhkUYFqC+6nqCSz15hQu8/5is2KUV2iX5PbnF16GkFSx2+taC4Ihi16bSXA8WDWWOzG4ZNJSIEdCqBWnW41e792cYBeEk+0P7ehw1uA9dS6P0ClADYAsRYescToIVQWwwP5rXA==

The signature bytes (decoded from base64) match the logged uint8array on client side.

Please share a PDF signed by your code for analysis. As a result of that analysis one usually finds more quickly the cause of the issue. — mkl
@mkl Yes of course! drive.google.com/file/d/1DTnkSaKGqS6T-ugZDKkfSZgAKDMOHpn7/… — MSpan

mkl mkl · Accepted Answer · 2020-03-24T18:52:36

Your original code

In MyExternalEmptySignatureContainer.Sign you correctly determine the authenticated attributes using the naked hash of the PDF range stream:

var authenticatedAttributeBytes = sgn.getAuthenticatedAttributeBytes(messageHash, null, null, CryptoStandard.CMS);

Inspecting your example file, though, I find that the signed Message Digest attribute contains the hash embedded in a DigestInfo object, in other works with the sha256Prefix you apply to a copy of the message digest in MyExternalEmptySignatureContainer.Sign.

Apparently, therefore, when you re-create the autheticated attributes in MyExternalSignatureContainer.Sign

return sgn.GetEncodedPKCS7(this.Hash, null, null, null, CryptoStandard.CMS);

you do so with the wrong value in this.Hash. This has two effects, on one hand obviously the value of the Message Digest attribute now is incorrect and on the other hand the signature value created for the original, correct authenticated attributes does not match your incorrect ones. Thus, the resulting PDF signature is doubly incorrect.

To fix this use the correct hash value here, the hash of the PDF range stream without that prefix.

As you don't show how exactly you use your methods GetPDFHash and SignPDFHash, I cannot pinpoint the error more precisely.

Your updated code

Indeed, now the correct hash is in the Message Digest attribute but the signature still signs a wrong hash, in case of your new example:

Signed Attributes Hash: 54B2F135A542EEAA55270AB19210E363D00A7684405403E89B170591A7BCAB5F
Decrypted signature digest: 22D906E686A83FA1A490895A21CD6F9A9272C13FB9B16D8A6E862168458F3640

The cause probably is that the contents of your MyExternalEmptySignatureContainer property PdfHash is not a hash but the complete authenticated attributes bytes, cf. MyExternalEmptySignatureContainer.Sign:

var authenticatedAttributeBytes = sgn.getAuthenticatedAttributeBytes(messageHash, null, null, CryptoStandard.CMS);

PdfHash = authenticatedAttributeBytes;

You probably have to calculate the hash of authenticatedAttributeBytes and put that into PdfHash.

As you don't show how exactly you use your methods GetPDFHash and SignPDFHash, though, this can only be guessed.

Your logged hashes

On March 30th you shared logs of the relevant data transported in one run. In particular:

AuthenticatedAttributeBytes - hashed

33 25 105 92 244 51 72 93 179 135 158 84 249 178 103 91 236 247 253 35 232 124 169 112 108 214 63 206 206 2 88 107

and

Hash signed (signature)

76 13 184 229 123 212 2 8 140 24 34 88 95 31 255 142 105 220 204 186 172 110 61 75 156 44 185 62 81 209 238 226 67 133 115 247 76 24 182 144 38 164 71 92 124 140 77 16 212 43 52 156 173 90 163 116 0 124 119 119 103 8 12 74 147 1 207 51 156 104 52 231 112 125 115 140 28 105 160 117 235 199 224 166 30 220 111 35 165 49 18 85 253 194 112 254 142 117 46 58 87 13 110 161 151 228 95 238 115 171 70 117 203 103 204 222 233 42 163 37 105 91 177 117 190 238 135 137 162 6 54 125 108 64 148 219 7 198 93 117 12 164 130 123 213 197 233 173 145 77 209 11 166 91 29 137 142 25 20 96 90 130 251 169 234 9 44 245 230 20 46 243 254 98 179 98 148 87 104 151 228 246 231 23 94 134 144 84 177 219 235 90 11 130 33 139 94 155 73 112 60 88 53 150 59 49 184 100 210 82 32 71 66 168 21 167 91 141 94 239 221 156 96 23 132 147 237 15 237 232 112 214 224 61 117 46 143 208 41 64 13 128 44 69 135 172 113 58 8 85 5 176 192 254 107 92

Decrypting that latter signed hash value using the public key from your certificate, though, one gets a DigestInfo object containing this hash:

136 138 205 115 82 228 115 151 231 220 177 93 171 239 123 224 245 180 234 166 132 201 244 54 69 22 18 16 115 223 70 193

Thus, whatever your client side code does, it does not create a signature for your pre-hashed AuthenticatedAttributeBytes. Probably it hashes the hash bytes again, probably it hashes a base64 representation of them, probably it uses some random number, but it does not what you expect it to do.

You should find out what your client code actually does and fix it or feed it the data it requires.

E.g. if your client side code cannot be kept from re-hashing the data, feed it the un-hashed authenticated attribute bytes.

External signing PDF with iTextSharp - altered/corrupted document

1 Answers

Your original code

Your updated code

Your logged hashes