Access self signed X509certificates in XamarinForms for mqtt TLS connection to a Mosquitto broker

Question

I desire to TLS secure with a self signed x509certificate a number of existing XamarinForms apps that connect to a mosquitto mqtt broker using the M2MqttDotnetCore client.

To that end I have created a simple sample XamarinForms pub/sub chat app to learn how to secure an XamarinForms mqtt client application that can be sound in this GitHub repository. jhalbrecht/XamarinFormsMqttSample

I have samples in Mosquitto_pub, python and a .net console app that accomplish this goal of successfully connecting to a mosquitto broker over port 8883 with TLS and a self signed certificate. The XamarinForms UWP app also works unsecured and secured. I'm having trouble getting the Android app to work with TLS on port 8883, The Android app does work unsecured on port 1883. This is the runtime log from Visual Studio 2017

[0:] M2Mqtt.Exceptions.MqttConnectionException: Exception connecting to the broker ---> System.AggregateException: One or more errors occurred. ---> System.Security.Authentication.AuthenticationException: A call to SSPI failed, see inner exception. ---> Mono.Btls.MonoBtlsException: Ssl error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED
  at /Users/builder/jenkins/workspace/xamarin-android-d15-9/xamarin-android/external/mono/external/boringssl/ssl/handshake_client.c:1132
  at Mono.Btls.MonoBtlsContext.ProcessHandshake () [0x00038] in <fb6d78e506844b3b96d5b35aa047fbbd>:0 
  at Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake (Mono.Net.Security.AsyncOperationStatus status) [0x0003e] in <fb6d78e506844b3b96d5b35aa047fbbd>:0 
  at (wrapper remoting-invoke-with-check) Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake(Mono.Net.Security.AsyncOperationStatus)
  at Mono.Net.Security.AsyncHandshakeRequest.Run (Mono.Net.Security.AsyncOperationStatus status) [0x00006] in <fb6d78e506844b3b96d5b35aa047fbbd>:0 
  at Mono.Net.Security.AsyncProtocolRequest+<ProcessOperation>d__24.MoveNext () [0x000ff] in <fb6d78e506844b3b96d5b35aa047fbbd>:0 
--- End of stack trace from previous location where exception was thrown ---
  at Mono.Net.Security.AsyncProtocolRequest+<StartOperation>d__23.MoveNext () [0x0008b] in <fb6d78e506844b3b96d5b35aa047fbbd>:0 
   --- End of inner exception stack trace ---
  at Mono.Net.Security.MobileAuthenticatedStream+<ProcessAuthentication>d__47.MoveNext () [0x00254] in <fb6d78e506844b3b96d5b35aa047fbbd>:0 
   --- End of inner exception stack trace ---
  at System.Threading.Tasks.Task.ThrowIfExceptional (System.Boolean includeTaskCanceledExceptions) [0x00011] in <d4a23bbd2f544c30a48c44dd622ce09f>:0 
  at System.Threading.Tasks.Task.Wait (System.Int32 millisecondsTimeout, System.Threading.CancellationToken cancellationToken) [0x00043] in <d4a23bbd2f544c30a48c44dd622ce09f>:0 
  at System.Threading.Tasks.Task.Wait () [0x00000] in <d4a23bbd2f544c30a48c44dd622ce09f>:0 
  at M2Mqtt.Net.MqttNetworkChannel.Connect () [0x000a8] in <72fbe921f857483bafbb8b397ec98dd1>:0 
  at M2Mqtt.MqttClient.Connect (System.String clientId, System.String username, System.String password, System.Boolean willRetain, System.Byte willQosLevel, System.Boolean willFlag, System.String willTopic, System.String willMessage, System.Boolean cleanSession, System.UInt16 keepAlivePeriod) [0x0001e] in <72fbe921f857483bafbb8b397ec98dd1>:0 
   --- End of inner exception stack trace ---
  at M2Mqtt.MqttClient.Connect (System.String clientId, System.String username, System.String password, System.Boolean willRetain, System.Byte willQosLevel, System.Boolean willFlag, System.String willTopic, System.String willMessage, System.Boolean cleanSession, System.UInt16 keepAlivePeriod) [0x00037] in <72fbe921f857483bafbb8b397ec98dd1>:0 
  at M2Mqtt.MqttClient.Connect (System.String clientId) [0x00000] in <72fbe921f857483bafbb8b397ec98dd1>:0 
  at MqttDataServices.Services.MqttDataService+<Initialize>d__5.MoveNext () [0x00266] in C:\jstuff\MqttSample\MqttDataServices\Services\MqttDataService.cs:183

The way I am currently loading and accessing the X509certificates is not secure or a best practice. It works. I hope to eventually learn how to access the device ca keystores for each mobile platform. I use the cross-platform plug-in FilePicker to load a cert, base64 encode it, and save it.

 FileData fileData = await Plugin.FilePicker.CrossFilePicker.Current.PickFile();
 if (fileData == null)
 return; // user canceled file picking

 string fileName = fileData.FileName;
 string content = Convert.ToBase64String(fileData.DataArray, 0, fileData.DataArray.Length,
 Base64FormattingOptions.None);

 string deviceFileName = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), fileName);
 File.WriteAllText(deviceFileName, content);

I have reached out to a few Xamarin folks via twitter. I have an open issue in my above mentioned repository discussing the problem where @baulig from Microsoft has I believe given me the answer however I don't currently know how to implement it.

I just looked at the certificate validation code and what it does is essentially

var certStore = KeyStore.GetInstance ("AndroidCAStore"); certStore.Load(null);

This is the entry point: https://github.com/mono/mono/blob/master/mcs/class/System/Mono.Btls/MonoBtlsX509LookupAndroid.cs, it calls this code https://github.com/mono/mono/blob/master/mcs/class/System/System/AndroidPlatform.cs#L101 which then calls into xamarin-android code here: https://github.com/xamarin/xamarin-android/blob/master/src/Mono.Android/Android.Runtime/AndroidEnvironment.cs

The KeyStore should be this class: https://developer.xamarin.com/api/type/Java.Security.KeyStore/.

So you should be able to do this via Java.Security.KeyStore.

What permissions are necessary to grant in AndroidManifest.xml?
What terms might I research to properly access the platform ca keystores?

Additions after initial posting

February 27, 2019 (MST) 2:51 PM
Added certs and mqtt client creation from MqttDataService.cs

 X509Certificate caCert = X509Certificate.CreateFromCertFile(Path.Combine(filesDirectoryBasePath, "ca.crt"));
 string thePfxPathOnDevice = Path.Combine(filesDirectoryBasePath, "xamarinclient.pfx");
 string theBase64EncodedPfx = File.ReadAllText(thePfxPathOnDevice);

 byte[] certificate = Convert.FromBase64String(theBase64EncodedPfx);
 X509Certificate2 clientCert = new X509Certificate2(certificate, "xamarin");
 _client = new MqttClient(
     GetHostName(_xpdSetting.MqttBrokerAddress),
     Int32.Parse(_xpdSetting.MqttBrokerTlsPort),
     _xpdSetting.UseTls,
     caCert,
     clientCert,
     MqttSslProtocols.TLSv1_2
     //MyRemoteCertificateValidationCallback
     );

Not as far as I know @SushiHangover I added the certs and mqtt client code to the original question. — Jeff Albrecht

SushiHangover SushiHangover · Accepted Answer · 2019-02-28T04:02:35

Since you are using .Net's/Mono Socket (via M2MqttDotnetCore), just use cert pinning and you only have to handle the RemoteCertificateValidationCallback. Thus no messing with Android's trusted stores, etc...

SslStream Usage on Android:

Note: There are issues with SslStream on Android, object allocations can go crazy... I believe(?) there is an open issue about this. (I had to use Java's SSLSocket a couple times to work around this issue)

Enable Native TLS 1.2+

Using BoringSSL via the Android project build options

Add your cert to the Android's Asset directory:

├── Assets
│   └── sushihangover.cert

This is your cert/.pem file (NOT your KEY!!)
Make sure that this is an ascii file with no unicode BOM header

Via openssl example (just change it to your host and secure port)

echo -n | openssl s_client -connect 10.1.10.250:5001 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p'

RemoteCertificateValidationCallback Implementation

Note: The following code can be in used in NetStd2.0 or Xamarin.Android

X509Certificate sushihangoverCert; // Class level var

bool CertificateValidation(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors certificateErrors)
{
    if (sushihangoverCert == null)
    {
        // There is no non-async version of OpenAppPackageFileAsync (via Xamarin.Essential) 😡 Why!!!
        using (var waitHandle = new EventWaitHandle(false, EventResetMode.AutoReset))
        {
            Task.Run(async () =>
            {
                using (var assetStream = await Xamarin.Essentials.FileSystem.OpenAppPackageFileAsync("sushihangover.cert"))
                using (var memStream = new MemoryStream())
                {
                    assetStream.CopyTo(memStream);
                    sushihangoverCert = new X509Certificate(memStream.ToArray());
                    waitHandle.Set();
                }
            });
            waitHandle.WaitOne();
        }
    }
    return sushihangoverCert.Equals(certificate) ? true : false;
}

SSLStream Usage Example:

Note: This is connecting to a NetCore Web API port using a self-signed cert

using (var tcpClient = new TcpClient("10.1.10.250", 5001))
using (var ssl = new SslStream(tcpClient.GetStream(), false, new RemoteCertificateValidationCallback(CertificateValidation)))
{
    ssl.AuthenticateAsClient("10.1.10.250", null, System.Security.Authentication.SslProtocols.Tls12, false);
    if (ssl.CanWrite)
    {
        var send = Encoding.ASCII.GetBytes("GET /api/item HTTP/1.1\r\nhost: 10.1.10.250\r\n\r\n");
        await ssl.WriteAsync(send, 0, send.Length);
        var buffer = new byte[4096];
        var count = await ssl.ReadAsync(buffer, 0, buffer.Length);
        Console.WriteLine(Encoding.UTF8.GetString(buffer, 0, count));
    }
    else
        throw new SocketException();
}

Server cert mismatch error:

If your server cert (self-signed or not) does not match the one that you are pinning to, you will receive:

{Mono.Btls.MonoBtlsException: Ssl error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED