Powershell System.Security.Cryptography.RSACryptoServiceProvider doesn't accept encrypting password with length more than 50 chars

Question

When i'm trying to encrypt a 50 character string, using the System.Security.Cryptography.RSACryptoServiceProvider object I get a bad length error when calling encrypt. My educated guess is that the length of the string is too much (maybe limitation of [byte]?), because when I have a 39char string everything works fine.

Full errors below:

Exception calling "Encrypt" with "2" argument(s): "Bad Length. "At [omitted]
+ $encrypted = $rsa.Encrypt($bytes,$true) + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (:) [], MethodInvocationException + FullyQualifiedErrorId : CryptographicException

Exception calling "Decrypt" with "2" argument(s): "Error occurred while decoding OAEP padding." At [omitted] + $Password = [char[]]$rsa.Decrypt($encrypted, $true) -join "" | Co ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (:) [], MethodInvocationException + FullyQualifiedErrorId : CryptographicException

My code:

function Checkpassword([String] $Type) {
$pwpath = "$root\$Type.pw"
$encrypted = ''

if (test-path $pwpath -erroraction silentlycontinue) {
    $encrypted = Import-Clixml $pwpath
}

if(!($encrypted)) {
    write-host "No $Type password file found, create one now by entering your $Type password." -fore yellow

    # Create password file using local encryption
    $key = (2,3,56,34,254,222,1,1,2,23,42,54,33,233,1,34,2,7,6,5,35,43,6,6,6,6,6,6,31,33,60,23)
    $pass = Read-Host "Enter your $Type password" -AsSecureString
    $securepass = $pass |ConvertFrom-SecureString -Key $key
    $bytes = [byte[]][char[]]$securepass            

    $csp = New-Object System.Security.Cryptography.CspParameters
    $csp.KeyContainerName = "SuperSecretProcessOnMachine"
    $csp.Flags = $csp.Flags -bor [System.Security.Cryptography.CspProviderFlags]::UseMachineKeyStore
    $rsa = New-Object System.Security.Cryptography.RSACryptoServiceProvider -ArgumentList 5120,$csp
    $rsa.PersistKeyInCsp = $true

    $encrypted = $rsa.Encrypt($bytes,$true)
    $encrypted |Export-Clixml "$root\$Type.pw" -force
}

$key = (2,3,56,34,254,222,1,1,2,23,42,54,33,233,1,34,2,7,6,5,35,43,6,6,6,6,6,6,31,33,60,23)            

$csp = New-Object System.Security.Cryptography.CspParameters
$csp.KeyContainerName = "SuperSecretProcessOnMachine"
$csp.Flags = $csp.Flags -bor [System.Security.Cryptography.CspProviderFlags]::UseMachineKeyStore
$rsa = New-Object System.Security.Cryptography.RSACryptoServiceProvider -ArgumentList 5120,$csp
$rsa.PersistKeyInCsp = $true

$Password = [char[]]$rsa.Decrypt($encrypted, $true) -join "" |ConvertTo-SecureString -Key $key
$credential = New-Object System.Management.Automation.PsCredential ".",$Password }

The docs mention a size limit for the input byte array when combined with OAEP padding that is described as follows: Modulus size -2 -2*hLen, where hLen is the size of the hash. I hope you know what that means. — mklement0

Mathias R. Jessen Mathias R. Jessen · Accepted Answer · 2016-04-02T16:04:38

Keep in mind that you already implicitly encrypt the string with 3DES when you call ConvertFrom-SecureString -Key $Key.

This means that even though the string itself is only ~100 bytes, the resulting payload ($bytes) is >500 bytes in size, already beyond the maximum size for a plaintext to be encrypted with OEAP and a 4096 bit key.

You can either install a bigger key in the key container, or (rather) use another encryption algorithm better suited for encryption of long messages

Powershell System.Security.Cryptography.RSACryptoServiceProvider doesn't accept encrypting password with length more than 50 chars

1 Answers