Override urlFormEncoded parser for charset - Play Framework 2.3.x

Question

how could I override urlFormEncoded parser to use another charset in Play 2.3.x (Scala)?

I'm writing BBS software which works in Japanese environment with Play framework 2.3.x in Scala.

The problem is about charset: Client POSTs request with url encoded (aka. "percent encoded") form parameters encoded in Shift-JIS -- famous charset in Japan -- but Play decodes it as UTF-8. That is predictable because it is hardcoded in https://github.com/playframework/playframework/blob/2.3.x/framework/src/play/src/main/scala/play/api/mvc/ContentTypes.scala#L515 .

I cannot modify BBS clients' encoding because it is de-facto standard.

Thus, I must rewrite or override url encoding decoder in Play to use Shift_JIS.

There may be some solutions:

1. Manually build Play from modified source code and use it.
2. Intercept request and decode body as Shift_JIS, then re-encode body as UTF-8.
3. Create custom parser which extends play.api.mvc.BodyParsers.parse and use it.

Most hopeful choice I thought is 3, but I cannot create object which extends play.api.mvc.BodyParsers.parse.

So, how could I override default parser and use it (or there are other better solutions)?

Thanks.

Answer 1

I resolved this problem renewing request using apache httpcomponents.

We can retrieve URL-encoded data as raw text, specifying parse.torelantText as body parser. Then manually parse URL-encoded data into Map[String, Seq[String]] and recreate Request.

I show the code below.

Define dependency in build.sbt:

libraryDependencies += "org.apache.httpcomponents" % "httpclient" % "4.4"

Define object to convert Request[String] into Request[Map[String, Seq[String]]]:

import play.api.mvc.{ Headers, Request }

object PercentEncoding {
  import java.net.URI
  import scala.collection.JavaConversions._
  import org.apache.http.client.utils.URLEncodedUtils
  import org.apache.http.NameValuePair
  def extractSJISRequest(request: Request[String]) = {
    val body = request.body
    val parsed: Seq[NameValuePair] = URLEncodedUtils.parse(new URI("http://example.com/?" + body), "Shift_JIS")
    val newBody = parsed.map { pair ⇒ (pair.getName, Seq(pair.getValue)) }.toMap
    new Request[Map[String, Seq[String]]] {
      override def body: Map[String, Seq[String]] = newBody

      override def uri: String = request.uri

      override def remoteAddress: String = request.remoteAddress

      override def queryString: Map[String, Seq[String]] = request.queryString

      override def method: String = request.method

      override def headers: Headers = request.headers

      override def path: String = request.path

      override def version: String = request.version

      override def tags: Map[String, String] = request.tags

      override def id: Long = request.id
    }
  }
}

Then convert original request using PercentEncoding and define implicit request value instead of original request:

def hogehoge = Action(parse.tolerantText) { request ⇒
    implicit val newRequest = PercentEncoding.extractSJISRequest(request)
...
}

And it works.

Answer 2

The default parser uses charset of RequestHeader if available. charset is determined by Content-Type header. So I assume that the client doesn't set Content-Type header.

One way is changing the charset by wrapping RequestHeader.

Define a wrapper.

class WrappedRequestHeader(rh: RequestHeader) extends RequestHeader {
  override def id = rh.id
  override def secure = rh.secure
  override def uri = rh.uri
  override def remoteAddress = rh.remoteAddress
  override def queryString = rh.queryString
  override def method = rh.method
  override def headers = rh.headers
  override def path = rh.path
  override def version = rh.version
  override def tags = rh.tags
}

Define a request header which returns a constant charset if an original header doesn't have it.

class DefaultCharsetRequestHeader(rh: RequestHeader, defaultCharset: String) extends WrappedRequestHeader(rh) {
  override lazy val charset = rh.charset orElse Some(defaultCharset) // Or always use Some(defaultCharset) if you want
}

Then you can create a BodyParser from any default parser.

  def createParser[A](parser: BodyParser[A], defaultCharset: String): BodyParser[A] = BodyParser[A] { rh =>
    parser(new DefaultCharsetRequestHeader(rh, defaultCharset))
  }

Or you can create a Filter if you want to apply it widely.

class DefaultCharsetFilter(defaultCharset: String) extends Filter {
  override def apply(f: (RequestHeader) => Future[Result])(rh: RequestHeader): Future[Result] =
    f(new DefaultCharsetRequestHeader(rh, defaultCharset))
}

I haven't tried above code yet, but I hope it works.