python - TypeError: 'str' does not support the buffer interface

Question

plaintext = input("Please enter the text you want to compress")
filename = input("Please enter the desired filename")
with gzip.open(filename + ".gz", "wb") as outfile:
    outfile.write(plaintext) 

The above python code is giving me following error:

Traceback (most recent call last):
  File "C:/Users/Ankur Gupta/Desktop/Python_works/gzip_work1.py", line 33, in <module>
    compress_string()
  File "C:/Users/Ankur Gupta/Desktop/Python_works/gzip_work1.py", line 15, in compress_string
    outfile.write(plaintext)
  File "C:\Python32\lib\gzip.py", line 312, in write
    self.crc = zlib.crc32(data, self.crc) & 0xffffffff
TypeError: 'str' does not support the buffer interface

Answer 1

If you use Python3x then string is not the same type as for Python 2.x, you must cast it to bytes (encode it).

plaintext = input("Please enter the text you want to compress")
filename = input("Please enter the desired filename")
with gzip.open(filename + ".gz", "wb") as outfile:
    outfile.write(bytes(plaintext, 'UTF-8'))

Also do not use variable names like string or file while those are names of module or function.

EDIT @Tom

Yes, non-ASCII text is also compressed/decompressed. I use Polish letters with UTF-8 encoding:

plaintext = 'Polish text: ąćęłńóśźżĄĆĘŁŃÓŚŹŻ'
filename = 'foo.gz'
with gzip.open(filename, 'wb') as outfile:
    outfile.write(bytes(plaintext, 'UTF-8'))
with gzip.open(filename, 'r') as infile:
    outfile_content = infile.read().decode('UTF-8')
print(outfile_content)

Answer 2

There is an easier solution to this problem.

You just need to add a t to the mode so it becomes wt. This causes Python to open the file as a text file and not binary. Then everything will just work.

The complete program becomes this:

plaintext = input("Please enter the text you want to compress")
filename = input("Please enter the desired filename")
with gzip.open(filename + ".gz", "wt") as outfile:
    outfile.write(plaintext)

Answer 3

You can not serialize a Python 3 'string' to bytes without explict conversion to some encoding.

outfile.write(plaintext.encode('utf-8'))

is possibly what you want. Also this works for both python 2.x and 3.x.

Answer 4

For Python 3.x you can convert your text to raw bytes through:

bytes("my data", "encoding")

For example:

bytes("attack at dawn", "utf-8")

The object returned will work with outfile.write.

Answer 5

This problem commonly occurs when switching from py2 to py3. In py2 plaintext is both a string and a byte array type. In py3 plaintext is only a string, and the method outfile.write() actually takes a byte array when outfile is opened in binary mode, so an exception is raised. Change the input to plaintext.encode('utf-8') to fix the problem. Read on if this bothers you.

In py2, the declaration for file.write made it seem like you passed in a string: file.write(str). Actually you were passing in a byte array, you should have been reading the declaration like this: file.write(bytes). If you read it like this the problem is simple, file.write(bytes) needs a bytes type and in py3 to get bytes out of a str you convert it:

py3>> outfile.write(plaintext.encode('utf-8'))

Why did the py2 docs declare file.write took a string? Well in py2 the declaration distinction didn't matter because:

py2>> str==bytes         #str and bytes aliased a single hybrid class in py2
True

The str-bytes class of py2 has methods/constructors that make it behave like a string class in some ways and a byte array class in others. Convenient for file.write isn't it?:

py2>> plaintext='my string literal'
py2>> type(plaintext)
str                              #is it a string or is it a byte array? it's both!

py2>> outfile.write(plaintext)   #can use plaintext as a byte array

Why did py3 break this nice system? Well because in py2 basic string functions didn't work for the rest of the world. Measure the length of a word with a non-ASCII character?

py2>> len('¡no')        #length of string=3, length of UTF-8 byte array=4, since with variable len encoding the non-ASCII chars = 2-6 bytes
4                       #always gives bytes.len not str.len

All this time you thought you were asking for the len of a string in py2, you were getting the length of the byte array from the encoding. That ambiguity is the fundamental problem with double-duty classes. Which version of any method call do you implement?

The good news then is that py3 fixes this problem. It disentangles the str and bytes classes. The str class has string-like methods, the separate bytes class has byte array methods:

py3>> len('¡ok')       #string
3
py3>> len('¡ok'.encode('utf-8'))     #bytes
4

Hopefully knowing this helps de-mystify the issue, and makes the migration pain a little easier to bear.

Answer 6

>>> s = bytes("s","utf-8")
>>> print(s)
b's'
>>> s = s.decode("utf-8")
>>> print(s)
s

Well if useful for you in case removing annoying 'b' character.If anyone got better idea please suggest me or feel free to edit me anytime in here.I'm just newbie

Answer 7

For Django in django.test.TestCase unit testing, I changed my Python2 syntax:

def test_view(self):
    response = self.client.get(reverse('myview'))
    self.assertIn(str(self.obj.id), response.content)
    ...

To use the Python3 .decode('utf8') syntax:

def test_view(self):
    response = self.client.get(reverse('myview'))
    self.assertIn(str(self.obj.id), response.content.decode('utf8'))
    ...