Ruby Encoding While File Writing

Question

I have a bunch of pdf/txt/msg files I'm trying to download in the ruby console and either move to a local directory or re-upload to another destination.

To start, I've just been trying to read the file and write it to a local directory like so:

path = File.join 'temp', doc_name
file = File.new(path, 'w')
file << Document.find(123).fetch_file  // this function retrieves and decrypts the file from s3

The exception I get is: Encoding::UndefinedConversionError: "\xB5" from ASCII-8BIT to UTF-8

I'm wondering how I might be able to get the encoding correct on the file write so that I can download it and open it. It seems like it should be trivial, the answer could be in the decryption or s3 call, but this seemed related to the file write.

peter peter · Accepted Answer · 2018-01-19T00:58:17

You need to open the file in binary to get the right encoding.

file = File.new(path, 'wb')

Check the encoding like this

puts file.encoding

It should be 'ASCII-8BIT'. Do the same with your decrypted filecontent, it should be the same encoding, other wise you need to convert it like this.

Document.find(123).fetch_file.force_encoding('ASCII-8BIT')

You could also use File.binread(file) and File.binwrite(file, content)

http://ruby-doc.org/core-2.3.0/IO.html#method-c-binread

http://ruby-doc.org/core-2.3.0/IO.html#method-c-binwrite

Ruby Encoding While File Writing

1 Answers