It is correct to switch the default perl's IO to utf-8 while using Plack and Middlewares?

Question

Two starting points:

In his answer to Why does modern Perl avoid UTF-8 by default? tchrist pointed out 52 things needed to ensure correct Unicode handling in Perl. The answer shows the boilerplate code with some use statements. A similiar question about the use of Unicode is How to make "use My::defaults" with modern perl & utf8 defaults?
The PSGI spec is by design byte oriented. It is my responsibility to encode/decode everything, so for the Plack apps the correct way is to encode output and decode input, e.g.:
```
use Encode;
my $app = sub {
    my $output = encode_utf8( myapp() );
    return [ 200, [ 'Content-Type' =>'text/plain' ], [ $str ] ];
};
```

Is it correct to use

use uni::perl; # or any similar

in the PSGI application and/or in my modules?

uni::perl changes Perl's default IO to UTF-8, thus:

use open qw(:std :utf8);
binmode(STDIN,   ":utf8");
binmode(STDOUT,  ":utf8");
binmode(STDERR,  ":utf8");

Will doing so break something in Plack or its middlewares? Or is the only correct way to write apps for Plack explicitely encoding/decoding at open, so without the open pragma?

Does Plack write to STDOUT or read from STDIN? If so, it's almost surely wrong (unless they're also a bug in Plack). I said "almost" because the use of binmode in Plack would make it not care. PS - now you know why it's not done by default; it breaks stuff. — ikegami
I'm in hope than @miyagawa gurusan will tell more.. :) And I understand why utf8 is not default, but, (IMO) the new CPAN modules should be developed with "perl -CSDA" or with env PERL_UNICODE in mind. And miyagawa sure uses it in japan environment, so, should know the right way.. ;) — cajwine
I think the “correct way” you list is broken. text/plain need a charset so the other side knows what the bytes represent and how to decode them. — Ashley
@Ashley, yes - and thanx. The fragment has another errors too ($str vs $output). But it is not really related to the question. — cajwine

LeoNerd LeoNerd · Accepted Answer · 2012-06-13T17:41:21

You really don't want to set STDIN/STDOUT to be UTF-8 mode by default on Plack, because you don't know for instance whether they will be binary data transports. E.g. if those filehandles are the FastCGI protocol connector they will be carrying encoded binary structures and not UTF-8 text. They therefore must not have an encoding layer defined, or those binary structures will be mangled or rejected as invalid.

It is correct to switch the default perl's IO to utf-8 while using Plack and Middlewares?

2 Answers