How do I change unicode u'\xb0C' from SQL to string °C in python?

0
votes

I SELECT data from SQL into temp_data dictionary but there is a unit value like "u't_unit': u'\xb0C'" which I can't convert to string °C in python.

Python Code:

temp_data = [{u'thresh_id': 508, u'app_guid': u'7D83E6F8D879BE17C95D14879159973A', 
u'override': 1, u'thresh_guid': u'BB8F3E8184783E0416EEA7D19D9C9FBC', 
u'app_id': 1102, u'edit_user': 1, u'h_range': Decimal('100.000'), 
u't_type': 1, u't_value': Decimal('0.000'), u'l_range': Decimal('0.000'), 
u't_unit': u'\xb0C', u'edit_date': datetime.datetime(2019, 4, 30, 4, 24, 44), 
u'name': u'Sensor Temperature High Major'}]

I used the following code to get my required values:

Python Code:

b = []  
for i in temp_data:  
    a = ('Dynamic', i['name'], i['thresh_id'], i['t_unit'], i['t_value'], i['l_range'], i['h_range'])  
    b.append(a)  
    b = [tuple(str(item) for item in t) for t in b]  
final_data = (",".join(map(str,b)))

The error message received:
UnicodeEncodeError: 'ascii' codec can't encode character u'\xb0' in position 0: ordinal not in range(128)

Expected result: 

('Dynamic', 'Sensor Temperature High Major', '508', '°C', '0.000', '0.000', '100.000')

I tried to use 'u'\xb0C'.encode('utf8')' to convert it, but it is not working.

How could I get the result as expected?
Thank you very much.

2
pythonpython-2.7character-encodingpython-unicode
You don't need to encode anything as your own question proves - SO uses UTF8 like almost all sites and stores text in nvarchar (Unicode) database fields. It doesn't perform any special encoding. What is the encoding of the database field? You may not have to do anything to store and retrieve °C - Panagiotis Kanavos

2 Answers

0
votes

When processing unicode data in Python 2, it's important to use the unicode and str types consistently. Ideally, use only unicode inside your application, and convert to and from str only when receiving input, or generating output, only when necessary.

The values in temp_data are either unicode or non-string types, so we want to process them as unicode objects rather than str.

>>> b = []
>>> for i in temp_data:
...     a = ('Dynamic', i['name'], i['thresh_id'], i['t_unit'], i['t_value'], i['l_range'], i['h_range'])
...     b.append(a)
...     # Cast items to unicode, not str
...     b = [tuple(unicode(item) for item in t) for t in b]
... 
>>> # Now all our items are unicode strings
>>> b
[(u'Dynamic', u'Sensor Temperature High Major', u'508', u'\xb0C', u'0.000', u'0.000', u'100.000')]
>>> # Join each tuple with unicode strings as the join value.
>>> final_data = (u','.join(u','.join(x) for x in b))  
>>> # Now the data prints correctly.
>>> print final_data
Dynamic,Sensor Temperature High Major,508,°C,0.000,0.000,100.000
0
votes

I suppose you use python 2.*.That's because you're printing the tuple. For example if you access the specific value you'll see that it is printed normally. Try this:

b = []
for i in temp_data:
    a = ('Dynamic', i['name'], i['thresh_id'], i['t_unit'], i['t_value'], i['l_range'], i['h_range'])
    print(i['t_unit'])  # prints °C
    print(a[3])  # prints °C