Invalid multibyte char with Ruby 1.9, setting encoding: utf-8 does not fix it

Question

I just upgraded my Ruby version from 1.8.7 to 1.9.2 (using RVM). So Ruby is: ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-darwin12.2.0]. The app is on Rails 3.2.8. Since upgrading, I get an error regarding multi-byte characters.

I've found multiple very similar issues on stackoverflow, but none of the solutions in them worked for me:

Ruby 1.9 - invalid multibyte char (US-ASCII) and invalid multibyte char (US-ASCII) with Rails and Ruby 1.9 and Rails 3.1, Ruby 1.9.2-p180 and UTF-8 issues

When I start my app, and try to browse to it, I multiple errors, such as:

SyntaxError (/Users/antun/ror/parktwor3svn/app/helpers/application_helper.rb:35: invalid multibyte char (UTF-8) /Users/antun/ror/parktwor3svn/app/helpers/application_helper.rb:35: syntax error, unexpected tIDENTIFIER, expecting ']' ... States", "Afghanistan", "?land Islands", "Albania", "Algeri...

The offending code (in app/helpers/application_helper.rb) looks like this. For brevity, I've boiled it down to just the first few countries - the "Å" in "Åland Islands" is one of the characters that triggers it. When I look up that character in my editor, I get: 00c5. (There's a bunch more unicode characters in other country names).

  def countries
    return ["", "United States", "Afghanistan", "Åland Islands", "Albania"]
  end

So far, I have tried adding the following to app/helpers/application_helper.rb as the very first line to fix it:

# encoding: UTF-8

and

# encoding: utf-8

and

# -*- coding: utf-8 -*-

None of these worked.

My config/application.rb has:

config.encoding = "utf-8"

My text editor is VIM, and when I do :set encoding on that file, it returns: "encoding=utf-8".

parsifal parsifal · Accepted Answer · 2012-10-01T20:05:56

Use a tool such as od (assuming you're on Linux) to look at the actual contents of the file. If you have single byte C5, your file isn't in UTF-8. It's probably stored in ISO-8859-1 or similar, and Ruby is quite correctly complaining.

Invalid multibyte char with Ruby 1.9, setting encoding: utf-8 does not fix it

1 Answers