OpenSSL signatures not verifying in Java

Question

I have a mobile application written in Objective C (iOS) which I need to generate signatures for the server to verify using the public key provided previously.

I'm struggling to get signatures to verify that are created by OpenSSL on my mobile application. I have Java to Java communication working and creating valid signatures, I just cannot get OpenSSL to create ones that are valid. There is a possibility my Java algorithm is incorrect but at this stage I'm fairly new to this.

Here is my OpenSSL code for generating a signature:

RCT_EXPORT_METHOD(signDataWithKey:(NSString *)dataToSign
                  privateKey:(NSString *)privateKey
                  callback:(RCTResponseSenderBlock)callback)
{
  int retEr;
  char* text = (char*) [dataToSign UTF8String];
  unsigned char *data;
  unsigned long dataLen;

  // converting nsstring base64 private key to openssl RSA key

  BIO *mem = NULL;
  RSA *rsa_private = NULL;
  char *private_key = (char*)[privateKey UTF8String];

  NSLog(@"Processing private key %s", private_key);

  mem = BIO_new_mem_buf(private_key, strlen(private_key));
  if (mem == NULL)
  {
    char buffer[120];
    ERR_error_string(ERR_get_error(), buffer);
    NSLog(@"Error loading private key %s", buffer);
  }

  rsa_private = PEM_read_bio_RSAPrivateKey(mem, NULL, NULL, NULL);
  BIO_free (mem);
  if (rsa_private == NULL)
  {
    char buffer[120];
    ERR_error_string(ERR_get_error(), buffer);
    NSLog(@"OpenSSL error: %s", buffer);
  } else {
    NSLog(@"Successfully loaded private key");
  }
  // end of convertion

  data = (unsigned char *) text;
  dataLen = strlen(text);


  //// creating signature
  // sha256
  unsigned char hash[SHA256_DIGEST_LENGTH];
  unsigned char sign[256];
  unsigned int signLen;



  SHA256(data, dataLen, hash);

  unsigned char *shaString = sha256_hash_string(hash);

  NSData* plainData = [NSData dataWithBytes:(const void *)shaString length:strlen(shaString)];
  NSString *base64String = [plainData base64EncodedStringWithOptions:0];

  NSLog(@"Base64 checksum %@", base64String);

  NSLog(@"SHA256 of %s is %send", text, shaString);


  //  signing
  retEr = RSA_sign(NID_sha256WithRSAEncryption, shaString, strlen(shaString), sign, &signLen, rsa_private);
  NSData* signatureData = [NSData dataWithBytes:(const void *)sign length:signLen];
  NSString *base64Signature = [signatureData base64EncodedStringWithOptions:0];
  NSLog(@"Got signed data %@", base64Signature);


  //  printf("Signature len gth = %d\n", signLen);
  NSLog(@"RSA_sign: %@ signature length = %u", (retEr == 1) ? @"RSA_sign success": @"RSA_sign error", signLen);

  NSLog(@"Got signed data %@", base64Signature);


  RSA_free(rsa_private);
  callback(@[base64Signature]);
}

unsigned char *sha256_hash_string (unsigned char hash[SHA256_DIGEST_LENGTH])
{
  unsigned char *outputBuffer = calloc(65, sizeof(char));

  int i = 0;

  for(i = 0; i < SHA256_DIGEST_LENGTH; i++)
  {
    sprintf(outputBuffer + (i * 2), "%02x", hash[i]);
  }

  outputBuffer[64] = 0;

  return outputBuffer;

}

You will see that I am actually using the string representation of the SHA256 hash - not the binary - this is on purpose. I then take that string and then sign it.

Now here is the Java code to verify the signature:

public static boolean verifySignature(PublicKey publicKey, byte[] signedData, String signature) {
      java.security.Security.addProvider(
              new org.bouncycastle.jce.provider.BouncyCastleProvider()
      );
      Signature signatureCheck = Signature.getInstance("SHA256withRSA", "BC");
      signatureCheck.initVerify(publicKey);
      signatureCheck.update(signedData);

      return signatureCheck.verify(CryptoUtil.Base64Decode(signature))
  }

Everything seems to be correct (I have manually validated the Private Key/Public Key pairs) and if I sign the same data using Java it will then verify (as follows):

  public static String getSignatureForString(PrivateKey privateKey, String data) {
        Signature signature = Signature.getInstance("SHA256withRSA", "BC");
        signature.initSign(privateKey);

        signature.update(data.getBytes());

        byte[] signed = signature.sign();

        return CryptoUtil.Base64Encode(signed);
    }

I'm really at a dead end here, I've tried all the various changing the RSA_sign method to use NID_sha256 on its own and other permutations, but at this stage I'm just guessing. I would appreciate some assistance if someone has done this kind of thing successfully before.

Answer 1

OK so I don't exactly know why this is any different from the RSA_sign method but after testing openssl_verify() and openssl_sign() in PHP and those functions were producing the correct results for the signatures created in Java I decided to read through the PHP source code to figure out how they do it. Anyway, PHP uses functions with EVP_ prefix (high level cryptographic functions according to OpenSSL documentation). Anyway, I adjusted to Objective C code to instead use these functions and now my signatures are matching in all languages. I hope this helps someone.

RCT_EXPORT_METHOD(signDataWithKey:(NSString *)dataToSign
                  privateKey:(NSString *)privateKey
                  callback:(RCTResponseSenderBlock)callback)
{
  int retEr;
  char* text = (char*) [dataToSign UTF8String];
  unsigned char *data;
  unsigned long dataLen;

  // converting nsstring base64 private key to openssl RSA key

  BIO *mem = NULL;
  EVP_PKEY *pkey;
  char *private_key = (char*)[privateKey UTF8String];

  EVP_MD_CTX md_ctx;

  NSLog(@"Processing private key %s", private_key);


  mem = BIO_new_mem_buf(private_key, strlen(private_key));
  if (mem == NULL)
  {
    char buffer[120];
    ERR_error_string(ERR_get_error(), buffer);
    NSLog(@"Error loading private key %s", buffer);
  }

  // CHANGED
  pkey = PEM_read_bio_PrivateKey(mem, NULL, NULL, NULL);

  BIO_free (mem);
  if (pkey == NULL)
  {
    char buffer[120];
    ERR_error_string(ERR_get_error(), buffer);
    NSLog(@"OpenSSL error: %s", buffer);
  } else {
    NSLog(@"Successfully loaded private key");
  }
  // end of convertion

  data = (unsigned char *) text;
  dataLen = strlen(text);


  //// creating signature
  // sha256
  unsigned char hash[SHA256_DIGEST_LENGTH];
  unsigned char sign[4096];
  int signLen;



  SHA256(data, dataLen, hash);

  unsigned char *shaString = sha256_hash_string(hash);

  NSData* plainData = [NSData dataWithBytes:(const void *)shaString length:strlen(shaString)];
  NSString *base64String = [plainData base64EncodedStringWithOptions:0];

  NSLog(@"Base64 checksum %@", base64String);

  NSLog(@"SHA256 of %s is %s end", text, shaString);

  // CHANGED - Using EVP to sign now.
  EVP_SignInit(&md_ctx, EVP_sha256());
  EVP_SignUpdate(&md_ctx, shaString, strlen(shaString));
  retEr = EVP_SignFinal(&md_ctx, sign, &signLen, pkey);


  //  OLD METHOD
  //retEr = RSA_sign(NID_sha256WithRSAEncryption, shaString, strlen(shaString), sign, &signLen, rsa_private);


  NSData* signatureData = [NSData dataWithBytes:(const void *)sign length:signLen];
  NSString *base64Signature = [signatureData base64EncodedStringWithOptions:0];
  NSLog(@"Got signed data %@", base64Signature);


  //  printf("Signature len gth = %d\n", signLen);
  NSLog(@"RSA_sign: %@ signature length = %u", (retEr == 1) ? @"RSA_sign success": @"RSA_sign error", signLen);

  NSLog(@"Got signed data %@", base64Signature);


  EVP_PKEY_free(pkey);
  callback(@[base64Signature]);
}

unsigned char *sha256_hash_string (unsigned char hash[SHA256_DIGEST_LENGTH])
{
  unsigned char *outputBuffer = calloc(65, sizeof(char));

  int i = 0;

  for(i = 0; i < SHA256_DIGEST_LENGTH; i++)
  {
    sprintf(outputBuffer + (i * 2), "%02x", hash[i]);
  }

  outputBuffer[64] = 0;

  return outputBuffer;

}