Sign multiple location with same response xml signature in PKCS7 (CMS)

Question

PDF document needs to signed with national digital identity.
National digital identity WebService provide facility to sign document, in my project I have integrated same.

Requesting Esign services give response in PKCS7(CMS) format. I want to append same response in multiple locations, So i am creating multiple empty signature container post i receive Response from service.

I referred this article : Sign Pdf Using ITextSharp and XML Signature

But in given article we only one signing location is present but i have multiple signing locations.

I am using itext sharp Library. Using MakeSignature.SignDeferred Method to append signature at multiple locations but it is showing PDF invalid.

Please find below response XML which i received from Webservice:

<?xml version="1.0" encoding="UTF-8"?>
<EsignResp errCode="NA" errMsg="NA" resCode="259A52453BE95D3A1071193995E062E3EAD796AD" status="1" ts="2019-03-18T14:26:59" txn="UKC:eSign:2998:20190318142602814">
    <UserX509Certificate>--Usercerti in base64--</UserX509Certificate>
    <Signatures>
        <DocSignature error="" id="1" sigHashAlgorithm="SHA256">--Signature in base 64 in PKCS7(CMS)---</DocSignature>
    </Signatures>
    <Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
        <SignedInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
            <CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315"></CanonicalizationMethod>
            <SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256"></SignatureMethod>
            <Reference URI="">
                <Transforms>
                    <Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"></Transform>
                </Transforms>
                <DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"></DigestMethod>
                <DigestValue>MrOfovytOIp/8qlEkgamrcyhGTSGTN5aS1P+08Fbwfk=</DigestValue>
            </Reference>
        </SignedInfo>
        <SignatureValue>BBexJyk47YaTdoDgXaFRCtJq1Gc3KsZNt48/I8X4TgNJ6gh2NI9Y5Y9Tc7bozrK/QRy1VYPOWYq5r/YdunjMQLmJJicyeqeqe2eD+TJ8oecpjCbmhPnDK2VgaJ2h00sfsfdsflIe/toKwAmV4PTBA1a5wkz77hj+HTkWXMkPEIsBUnBirVpHxe2bYaa7jcIIpWtJmqvcSurKTOeyFRa+AFWfwWHB/EzHJlDmgiMXzrNauxJ4HpphNaRU+bO5JdyzJs/8Zx4i6qwSEybkuprL3GdO9C7zMPiC98CTfO2dfUrbZWy1pSvwEqlVXQIfrkp+m2JRbFgT8EEIGfXUS+AJBPRwhY1Xsww==</SignatureValue>
        <KeyInfo>
            <KeyValue>
                <RSAKeyValue>
                    <Modulus>0o9vohWZ3ztI9ea8D/zUEUBRq6c82BE7sFmr1hNMeuGSJQFf39ceesRtGUzlUYVWXcU23P8sVZ5419CHh7ApFzUXaLD72i/2d5FFI0n3iRlTQec9PEUHyrvOCVDpqBhbnrO/EHBqRluUQJTQUtMu5mhPNFV7IIJMTEAsUhCL9adZXXQK9NeK0foRr29Oq7VdEGfSeLzHIibpQmhNPh89oJXqu0cmbNSW4J4i2GmwHQpmsmHaSQcgh4mgVrykO64pAKXPreAPipDHQM1l/e5hilYlWfLHxhC5OdfdfdsbTCTcydQ218IVulFOFhdQt7xVV61TOmoTC2elhWbDqoLJBVU5mBfQ==</Modulus>
                    <Exponent>AQAB</Exponent>
                </RSAKeyValue>
            </KeyValue>
            <X509Data>
                <X509SubjectName>CN=D-Random detail</X509SubjectName>
                <X509Certificate>--public certificate of provider--- </X509Certificate>
            </X509Data>
        </KeyInfo>
    </Signature>
</EsignResp>

EDIT: As per latest communication, Web Service provide response for whatever hash is being provided from my end. They do not validate it. Hash is any 64 character string. Kindly let me know what are the possible ways by which i can use this to append PKCS7 signature on a PDF document.

Below Code for generating request :

if (System.IO.File.Exists(tempPdf))
System.IO.File.Delete(tempPdf);

using (PdfReader reader = new PdfReader(pdfReadServerPath))
{
    using (FileStream os = System.IO.File.OpenWrite(tempPdf))
    {
        PdfStamper stamper = PdfStamper.CreateSignature(reader, os, '\0',null,true);

        PdfSignatureAppearance appearance = stamper.SignatureAppearance;

        appearance.SetVisibleSignature(new Rectangle(15, 15, 100, 100), 1, "sign1");

        appearance.CertificationLevel = PdfSignatureAppearance.NOT_CERTIFIED;
         AllPagesSignatureContainer external = new AllPagesSignatureContainer(appearance);

        MakeSignature.SignExternalContainer(appearance, external, 8192);
        Stream data = appearance.GetRangeStream();

       Stream data = appearance.GetRangeStream();
        byte[] hash = ReadFully(data); //Convert stream to byte
        _signatureHash = hash;


    }
}
//create sha256 message digest
using (SHA256.Create())
{
    _signatureHash = SHA256.Create().ComputeHash(_signatureHash);
}
bool check = false;
string hexencodedDigest = null;
//create hex encoded sha256 message digest
hexencodedDigest = new BigInteger(1, _signatureHash).ToString(16);
hexencodedDigest = hexencodedDigest.ToUpper();
if (hexencodedDigest.Length == 64)
{
    **Send this hexencoded hash to webservice**
}

Below code for appending signature:

//DLL Call
eSign2_1_Request_Response req_resp = new eSign2_1_Request_Response();

//// Response XML Digest process
string resp_xml = Request.Form["msg"].ToString();//signature response XML;
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.LoadXml(resp_xml);
XmlElement EsignResp = xmlDoc.DocumentElement;
if (EsignResp.Attributes != null && EsignResp.Attributes["status"].Value != "1")
{
    req_resp.WriteTextFileLog("errCode: " + EsignResp.Attributes["errCode"].Value + " & Error Message: " + EsignResp.Attributes["errMsg"].Value, "log", base_folder_path);
}
else
{
    req_resp.WriteTextFileLog(resp_xml, "xml", base_folder_path + "\\" + file_withoutExtn + "_responseXML.txt");
    //-------Continue to generate signed PDF by passing parameter to DLL

    XmlNodeList nodeList = xmlDoc.GetElementsByTagName("Signatures");

    string signature = nodeList[0].FirstChild.InnerText;

    string signedPdf = @"D:\POC Hosted\TryNSDL\TryNSDL\wwwroot\TempPath\signedPdf.pdf";
    string tempPdf = @"D:\POC Hosted\TryNSDL\TryNSDL\wwwroot\TempPath\tempPdf.pdf";
    using (PdfReader reader = new PdfReader(tempPdf))
    {

        using (FileStream os = System.IO.File.OpenWrite(signedPdf))
        {
            byte[] encodedSignature = Convert.FromBase64String(signature);

            IExternalSignatureContainer external = new MyExternalSignatureContainer(encodedSignature);

            MakeSignature.SignDeferred(reader, "sign1", os, external);
        }
    }
}

Code for Allsignature container:

public class AllPagesSignatureContainer : IExternalSignatureContainer
{
    public AllPagesSignatureContainer(PdfSignatureAppearance appearance)
    {
        this.appearance = appearance;

    }

    public void ModifySigningDictionary(PdfDictionary signDic)
    {
        signDic.Put(PdfName.FILTER, PdfName.ADOBE_PPKMS);
        signDic.Put(PdfName.SUBFILTER, PdfName.ADBE_PKCS7_DETACHED);

        PdfStamper stamper = appearance.Stamper;
        PdfReader reader = stamper.Reader;
        PdfDictionary xobject1 = new PdfDictionary();
        PdfDictionary xobject2 = new PdfDictionary();
        xobject1.Put(PdfName.N, appearance.GetAppearance().IndirectReference);
        xobject2.Put(PdfName.AP, xobject1);

        PdfIndirectReference PRef = stamper.Writer.PdfIndirectReference;
        PdfLiteral PRefLiteral = new PdfLiteral((PRef.Number + reader.NumberOfPages) + " 0 R");

        for (int i = 2; i < reader.NumberOfPages+1; i++)
        {
            var signatureField = PdfFormField.CreateSignature(stamper.Writer);

            signatureField.Put(PdfName.T, new PdfString("ClientSignature_" + i.ToString()));
            signatureField.Put(PdfName.V, PRefLiteral);
            signatureField.Put(PdfName.F, new PdfNumber("132"));
            signatureField.SetWidget(new Rectangle(15, 15, 100, 100), null);
            signatureField.Put(PdfName.SUBTYPE, PdfName.WIDGET);

            signatureField.Put(PdfName.AP, xobject1);
            signatureField.SetPage();
            Console.WriteLine(signatureField);

            stamper.AddAnnotation(signatureField, i);
        }
    }

    public byte[] Sign(Stream data)
    {
       return new byte[0];
    }

    PdfSignatureAppearance appearance;

}

I used append mode in create signature then signature doesn't come. Only empty signatures are visible in adobe reader : /Fileremoved/

If i try same without appendmode PdfStamper stamper = PdfStamper.CreateSignature(reader, os, '\0'); and PdfLiteral PRefLiteral = new PdfLiteral((PRef.Number + 1 + 2 * (reader.NumberOfPages - 1)) + " 0 R"); then it works fine : /Fileremoved/, but it can be used for single signer only. Nd if we again try to use same pdf to Resign then old signatures becomes invalid. (obviously since append mode is not used.)

I guess For signing to work in append mode, Change is required in line of PdfLiteral - I have less idea about same how it actually works.

Signed file :/Fileremoved/ Input file: /Fileremoved/

Answer 1

A first quick look through your code revealed two major errors.

Hashing twice

You hash the document data twice (using different APIs for that... weird!):

        Stream data = appearance.GetRangeStream();

        byte[] hash = DigestAlgorithms.Digest(data, "SHA256");

        [...]

        _signatureHash = hash;// signatureHash;
    }
}

[...]
using (SHA256.Create())
{
    _signatureHash = SHA256.Create().ComputeHash(_signatureHash);
}

This is wrong, this makes no sense.

Injecting the wrong signature container

You say

Requesting Esign services give response in PKCS7(CMS) format.

But instead of using the CMS signature container from the result as such, you try to build an own CMS container, injecting the Esign response CMS container as if it was a mere signed hash:

XmlNodeList UserX509Certificate = xmlDoc.GetElementsByTagName("UserX509Certificate");
byte[] rawdat = Convert.FromBase64String(UserX509Certificate[0].InnerText);
var chain = new List<Org.BouncyCastle.X509.X509Certificate>
{
    Org.BouncyCastle.Security.DotNetUtilities.FromX509Certificate(new X509Certificate2(rawdat))
};
var signaturee = new PdfPKCS7(null, chain, "SHA256", false);
_signature = signaturee;

_signature.SetExternalDigest(Convert.FromBase64String(signature), null, "RSA");

byte[] encodedSignature = _signature.GetEncodedPKCS7(_hash, null, null, null, CryptoStandard.CMS);

According to your comments in the XML

    <DocSignature error="" id="1" sigHashAlgorithm="SHA256">--Signature in base 64 in PKCS7(CMS)---</DocSignature>

this DocSignature element contains the CMS signature container.

Thus, remove the code segment above and instead put the content of the DocSignature element (don't forget to base64 decode) into the byte[] encodedSignature. Now you can inject it into the prepared signature as before:

IExternalSignatureContainer external = new MyExternalSignatureContainer(encodedSignature);

MakeSignature.SignDeferred(reader, "sign1", os, external);

---

After you fixed the issues above, two more became apparent:

Using the wrong file mode

You open the stream to write to like this:

using (FileStream os = System.IO.File.OpenWrite(signedPdf))

File.OpenWrite is documented on docs.microsoft.com to be

equivalent to the FileStream(String, FileMode, FileAccess, FileShare) constructor overload with file mode set to OpenOrCreate, the access set to Write, and the share mode set to None.

The file mode OpenOrCreate in turn is documented to specify

that the operating system should open a file if it exists; otherwise, a new file should be created.

Thus, if there already is a file at the given location, that file remains and you start writing into it.

If the new file you create is longer than the old one, this is no problem, you eventually overwrite all the old file content and then the file grows to house the additional new content.

But if the new file you create is shorter than the old one, you have a problem: After the end of the new file there still is data from the old, longer file. Thus, your result is a hodgepodge of two files.

This happened in case of the example files you shared, your new content of "signedPdf.pdf" is only 175982 bytes long but there appears to have been some older file with that name which was 811986 bytes long. Thus, the "signedPdf.pdf" file you shared is 811986 bytes long, the first 175982 bytes containing the result of your operation, the rest data from some other file.

If you cut down your shared "signedPdf.pdf" file to its first 175982 bytes, the result looks much better!

To solve this issue you should use the file mode Create which is documented to be

equivalent to requesting that if the file does not exist, use CreateNew; otherwise, use Truncate.

using (FileStream os = new FileStream(signedPdf, FileMode.Create, FileAccess.Write, FileShare.None))

An issue with your signing service - identity not yet valid

As mentioned above, if you cut down your shared "signedPdf.pdf" file to its first 175982 bytes, the result looks much better! Unfortunately merely better, not yet good:

The reason for your "identity has expired or is not yet valid" becomes clearer by looking at the details:

I.e. the signing time claimed by the PDF is 09:47:59 UTC+1.

But looking at the certificate:

I.e. your certificate is valid not before 09:48:40 UTC+1.

Thus, the claimed signing time is more than half a minute before your user certificate became valid! This obviously cannot be accepted by a validator...

Apparently your signing service creates a short-time certificate for you on demand, valid from just then for half an hour. And the time at which you started creating the PDF signature is not in that interval.

I doubt they will change the design of the signing service for your requirements. Thus, you'll have to cheat a bit and use a signing time slightly in the future.

By default the signing time is set to the current by the PdfSignatureAppearance constructor, i.e. when this line executes:

PdfSignatureAppearance appearance = stamper.SignatureAppearance;

Fortunately you can change this claimed signing time if you immediately use

appearance.SignDate = [some other date time];

The date time you should use here has to be shortly (I'd propose not more than 5 minutes) after the time you will call your signing service.

This of course implies that you cannot arbitrarily wait until executing that service call. As soon as you assigned the claimed signing time above, you are committed to have successfully called your signing service shortly before that claimed time!

Furthermore, if that signing service turns out to react only slowly or only after some retries, your software should definitively check the certificate in the signature container you retrieve from it and compare its validity interval with your claimed signing time. If the claimed signing time is not in that interval, start signing again!

---

Now it became apparent that the AllPagesSignatureContainer you used was designed for a very special use case and still had to be adapted to your use case.

Adapting the AllPagesSignatureContainer for append mode

The AllPagesSignatureContainer implementation essentially copied from this answer worked fine when not signing in append mode but when signing in append mode it failed.

This at first was plausible because that class has to predict the object number that will be used for the signature value. This prediction depends on the exact use case, and switching on append mode changes this use case considerably. Thus, my advice in a comment was

If you need append mode, try to replace the

PdfLiteral PRefLiteral = ...

line in the AllPagesSignatureContainer by

PdfLiteral PRefLiteral = new PdfLiteral((PRef.Number + reader.NumberOfPages) + " 0 R");

In my tests that worked but in your tests it still didn't. An analysis of your signed file turned up the cause: My test file was using cross reference tables while yours was using cross reference streams.

Adapting the AllPagesSignatureContainer for append mode and object streams

iText in append mode uses the compression features of the original file, i.e. in case of your file it creates an object stream as soon as storing an indirect object that allows storage in an object stream.

In case of your file iText reserved an object number for the object stream, and it did so between the time the AllPagesSignatureContainer predicted the signature value object number and the time the signature value actually was generated. Thus, in your file the actual signature value object number was higher than the predicted number by 1.

To solve this for PDFs with cross reference streams, therefore, one can simply replace the PdfLiteral PRefLiteral = ... line by

PdfLiteral PRefLiteral = new PdfLiteral((PRef.Number + reader.NumberOfPages + 1) + " 0 R");

i.e. by adding 1 to the originally predicted value. Unfortunately now the prediction is wrong for PDFs with cross reference tables...

A better way to fix this is to force iText to reserve an object number for the object stream for cross reference stream PDFs before predicting the signature value object number and then use the original prediction code. One way to do this is by creating and writing an indirect object right before the prediction, e.g. like this:

stamper.Writer.AddToBody(new PdfNull(), stamper.Writer.PdfIndirectReference, true);

PdfIndirectReference PRef = stamper.Writer.PdfIndirectReference;
PdfLiteral PRefLiteral = new PdfLiteral((PRef.Number + reader.NumberOfPages) + " 0 R");

The answer the AllPagesSignatureContainer implementation essentially was copied from has been updated accordingly.