Gmail API - how to correctly parse message body data?

Question

I'm using the new Gmail API and am absolutely stuck on how to correctly handle the encoding of the [body][data] portion in Ruby/Rails for both the text/plain message and the text/html message.

Let's say data = the encoded message portion.

Calling Base64.decode64(data).unpack("M") on it returns an US-ASCII encoded text body with lots of missing characters as displayed on a web page.

Calling Base64.decode64(data).encode('UTF-8') throws a conversion error from US-ASCII to UTF-8

Yet if I do Base64.decode64(data).encode('UTF-8', {:invalid => :replace, :undef => :replace, :replace => '?'}), I'm still seeing a ton of question marks.

Can someone point me in the right direction on how to get the message body to be properly encoded and displayed in UTF-8?

The formatting of the email JSON response is as follows:

"parts": [
   {
    "partId": "0",
    "mimeType": "text/plain",
    "filename": "",
    "headers": [
     {
      "name": "Content-Type",
      "value": "text/plain; charset=UTF-8"
     },
     {
      "name": "Content-Transfer-Encoding",
      "value": "quoted-printable"

Answer 1

Use Base64.urlsafe_decode64 to decode the message body.

Answer 2

var base64toUTF8 = function base64toUTF8(str,urlsafe) {
  if(urlsafe) {
    str = str.replace(/_/g,"/");
    str = str.replace(/-/g,"+");
  }
  if(typeof window) {
    return decodeURIComponent(escape(window.atob( str )));
  }
  else if(typeof module !== 'undefined' && module.exports) {
    return new Buffer("SGVsbG8gV29ybGQ=", 'base64').toString('utf8');
  }
};

just the need to replace base64 encoded chars '-' with '+' and '_' with '/'

Answer 3

Based on Bhargav Krishna's answer, here's a nodeJS-friendly version:

var base64toUTF8 = function base64toUTF8(str, urlsafe) {
  if (urlsafe) {
    str = str.replace(/_/g,"/");
    str = str.replace(/-/g,"+");
  }
  return new Buffer(str, 'base64').toString('utf8');
};

I removed the references to window vs module, and also made it so that the NodeJS one actually uses str, not just "Hello World!"