How to calculate RSA public key fingerprint using OpenSSL?

Question

I have a RSA public key in an EVP_PKEY structure (loaded from a PEM file starting with -----BEGIN PUBLIC KEY-----). Now I want to be able to display a fingerprint of that key from my code using the OpenSSL API. (The purpose is to allow the operator to verify the key for JWT using RS256 before trusting it).

Unfortunately all resources I found so far either use ssh-keygen on the commandline or do the fingerprint of an X.509 certificate but not the public key.

So how do I get RSA public key fingerprint of a public key stored in a EVP_PKEY structure using the OpenSSL API?

Answer 1

In my own code (C++), I have used the below, which takes an EVP_PKEY object and returns an RSA object:

using evp_pkey_ptr = std::unique_ptr<EVP_PKEY>;
using rsa_ptr = std::unique_ptr<RSA>;

rsa_ptr evp_get_rsa(const evp_pkey_ptr& evp) {
  RSA * rsa = EVP_PKEY_get1_RSA(evp.get());
  return rsa_ptr(rsa);
}

Regarding your purpose of verifying JWT.

This is how I did it:

// get jwt payload from jwt data
std::string payload_b64 = jwt[0] + "." + jwt[1];

// get jwt signature from jwt data
std::string signature_b64 = from_base64uri(jwt[2].c_str(), jwt[2].size());

// obtain RSA object from EVP
RSA * rsa = EVP_PKEY_get1_RSA(evp)

// create a digest of the jwt payload, using SHA256
digest = SHA256_Init, SHA256_Update, SHA256_Final, etc

// get byte representation of the jwt signature
sig_bytes = from_base64(payload_b64);

// call OpenSSL API to verify digest matches received signature
ERR_clear_error();
int rc = RSA_verify(NID_sha256,
                    (const unsigned char*) digest.data(), digest.size(),
                    (const unsigned char*) sig_bytes.data(), sig_bytes.size(), 
                    rsa);
if (1 != rc)
  printf("RSA_verify");

Answer 2

Although I'm still not sure whether this is "the official way"/ a commonly used way (suitable for writing "this is the key's fingerprint"), I found a reference at the "Deutsches Forschungsnetz" (https://www.dfn-cert.de/informationen/themen/verschluesselung_und_pki/openssl-kurzreferenz.html) where they use the hash of the key's modulus.

The modulus in hex presentation can be acquired from PRSA with BN_bn2hex(rsa.n) and must be prefxied with 'Modulus=' and suffixed with CR/LF to produce same output as command line. This is my resulting Delphi code:

function GetRsaModulusFingerprint(theRsa: PRSA; md: PEVP_MD; useLowerCase:
    Boolean; const separator: string): string;
var
  len: integer;
  modulusStr: PAnsiChar;
  buff: AnsiString;
  mdValue: array [0..EVP_MAX_MD_SIZE-1] of Byte;
  ctx: EVP_MD_CTX;
  mdLen: Cardinal;
  I: Integer;
begin
  // This func should produce same result as commandline
  // for public key:
  //  openssl rsa -noout -modulus -pubin -in server.pubkey |openssl sha256 -c
  // for private key:
  //  openssl rsa -noout -modulus -in server.privkey |openssl sha256 -c

  Result := '';
  if not Assigned(theRsa) then
    Exit;

  // get hex representation of modulus
  modulusStr := BN_bn2hex(theRsa.n);
  // bring it in same format as commanline
  buff := 'Modulus='+ AnsiString(modulusStr) + #13#10;
  // and free the string returned by BN_bn2hex using OPENSSL_free
  // (in openSSL1.0.2r crypto.h OPENSSL_free is a macro pointing to CRYPTO_free)
  CRYPTO_free(modulusStr);

  // calculate hash
  len := length(buff);
  if len >= 0 then
  begin
    EVP_MD_CTX_init(@ctx);
    EVP_DigestInit_ex(@ctx, md, nil);
    EVP_DigestUpdate(@ctx, PAnsiChar(buff), len);
    EVP_DigestFinal_ex(@ctx, @mdValue[0], mdLen);
    EVP_MD_CTX_cleanup(@ctx);

    //output as hex with separator
    result := '';
    if mdLen > 0 then
      result := IntToHex(mdValue[0]);
    for I := 1 to mdLen-1 do
      result := result + separator + IntToHex(mdValue[I]);

    if useLowerCase then
      result := result.ToLower();
  end;
end;