Perl, IO::Socket::SSL, multi-threading

Question

I've implemented a small webserver in Perl. It is listening with IO::Socket::INET and IO::Socket::SSL parallel.

If a connect appears at the HTTP-port I start a thread and handle the IO::Socket::INET reference.

Because of thread-limitations in Net::SSLeay (IO::Socket::SSL says in the doc is is not thread-safe below 1.43) I did NOT parallelize the SSL. I just call the handler-function in the same context. In the parallelized case of HTTP the handler-function is the threads function.

All this is working as expected for a longer time.

I have now updated my system. Now my Net::SSLeay is 1.72 and I tried to paralellize the SSL too - the same way I do with the HTTP. But I get a segmentation fault at the first time I do a read.

use strict;
use warnings;
use IO::Handle;
use Fcntl ("F_GETFL", "F_SETFL", "O_NONBLOCK");
use Time::HiRes ("usleep");
use Socket;
use IO::Socket::SSL; 
use threads;
STDOUT->autoflush ();

my $port = "4433";
my $cer = "cer.cer";
my $key = "key.key";

my $sock = IO::Socket::SSL->new (Listen => SOMAXCONN, LocalPort => $port,
 Blocking => 0, Timeout => 0, ReuseAddr => 1, SSL_server => 1, 
 SSL_cert_file => $cer, SSL_key_file => $key) or die $@;

my $WITH_THREADS = 0;       # the switch!!

for (;;)
{
  eval
  {
    my $cl = $sock->accept ();
    if ($cl) 
    {   
        print ("\nssl connect");

          if ($WITH_THREADS == 0)   
          {
            # this is no multi-threading
             client ($cl);
          }
          else {    
            # with multithreading
            my $th = threads->create (\&client, $cl);   
            $th->detach (); 
         }
    }
 }; # eval
 if ($@)
 {
        print "ex: $@";
        exit (1);
 }
usleep (100000);        
}  # forever


sub client    # worker
{
  my $cl = shift;

  # unblock
  my $flags = fcntl ($cl, F_GETFL, 0) or die $!;
  fcntl ($cl, F_SETFL, $flags | O_NONBLOCK) or die $!;

  print "\n" . $cl->peerhost . "/" . $cl->peerport;

 my $ret = "";

 for (my $i = 0; $i < 100; $i ++)
 {
    $ret = $cl->read (my $recv, 5000);      
    # faults here if with threads!

    if (defined ($ret) && length ($recv) > 0)
    {   
        print "\nreceived $ret bytes";
    }
    else
    {
        print "\nno data";

    }   
    usleep (200000);
  }
  print "\nend client";
  $cl->close ();
}

I have also read some posts where they said IO::Socket::SSL is not threadsafe but I am not sure that is still the case.

Does anyone know if it is possible that way? Or maybe it is possible but I am handling it the wrong way...

Thanks, Chris

EDIT: I use Debian 8.3 with Perl 5.20.2. Net::SSLeay is 1.72, IO::Socket::SSL is 2.024. OpenSSL 1.0.1k

EDIT: Changed the code-sample to fully functional little sample-program.

I am not familiar with the prefork idea. Can you please explain very shortly how you handle the problem that you must listen at exactly one place (the socket listener) and push it forward to a worker. I can imagine a fork-afterwards (after the accept) but how can I do that before an accept? — chris01
Just had a look at httpd and what is the idea behind its prefork. If it is not working with perl threads maybe I will think about it. But I would prefere the current idea. — chris01
@chris: I'm the maintainer of IO::Socket::SSL. I'm not aware of any issues with thread support. Could you add OS, Perl version and version of IO::Socket::SSL? And if you could provide a minimal but complete example to reproduce the problem I could have a closer look at it. — Steffen Ullrich
Thank you, that is very kind. May I send you the details to your packet-maintainer-address?? — chris01

Steffen Ullrich Steffen Ullrich · Accepted Answer · 2016-02-25T18:42:14

TL;TR:
don't duplicate an established SSL socket into another thread.

Details:
You accept the SSL socket in the master thread into $cl and then create a new thread which works on the new socket. In effect this means you have the same file descriptor (kernel), the same OpenSSL data structure (user space) but two Perl variables using this single data structure (perl threads are shared nothing - so the Perl part is duplicated).

This only causes trouble because you then implicitly close the socket in the master ($cl gets out of scope) but continue to use it in the client thread. The close in the master thread causes an SSL shutdown and then frees the underlying OpenSSL structure. Thus $cl in the client thread points to some freed memory which causes the crash. You actually get something like this (without crash) also if you use forking instead of threading, because there is still the SSL shutdown done in the master process so the peer considers the socket closed and the child will not be able to make further use of the socket.

Instead of doing the SSL accept in the master thread you should move every SSL activity into the client thread. This will be done by doing the accept on a normal socket object and then upgrading it to SSL in the client thread. This is the preferred way anyway, see Basic SSL server in the IO::Socket::SSL documentation for details.

In the end your code would be changed like this:

my $port = "4433";
my $cer = "cer.cer";
my $key = "key.key";

# don't create a SSL socket but an INET socket
my $sock = IO::Socket::IP->new (
  Listen => SOMAXCONN, LocalPort => $port, Blocking => 0, ReuseAddr => 1
) or die $!;

my $WITH_THREADS = 1;       # the switch!!
....
sub client    # worker
{
  my $cl = shift;
  # upgrade INET socket to SSL
  $cl = IO::Socket::SSL->start_SSL($cl,
    SSL_server => 1,
    SSL_cert_file => $cer,
    SSL_key_file => $key
  ) or die $@;

Perl, IO::Socket::SSL, multi-threading

1 Answers