49
votes

How do I remove u chars from the following dictionary?

{u'name': u'A', u'primary_key': 1}  

This data is coming from Mongo Database find() query

so that it looks like

{'name': 'A', 'primary_key': 1}
7
The characters indicate that the strings are unicode strings. Do you have any reason for wanting to "remove" them? - Sven Marnach
when I use psycopg2 to execute statement it fails, I am assuming this is the reason ProgrammingError: 'syntax error at or near """"INSERT INTO p_dimension (id, name) VALUES (1, E\'Acro Energy\')""""\nLINE 1: """INSERT INTO p_dimension (id, name) VALUES (1, E\'Acr...\n ^\n' - daydreamer
@daydreamer You should probably post the actual code that is failing. - Amber
This sounds an awful lot like a problem with psycopg2 and not mongodb, considering that's where your error is. Post a full example with the problem you are really trying to solve - SingleNegationElimination

7 Answers

54
votes

Some databases such as Sqlite3 let you define converter and adapter functions so you can retrieve text as str rather than unicode. Unfortunately, MongoDB doesn't provide this option for any of the commonly needed types such as str, decimal or datetime:

Having eliminated Mongo options, that leaves writing Python code to do the conversion after the data is retrieved. You could write a recursive function that traverses the result to convert each field.

As a quick-and-dirty alternative, here is a little hack that may be of use:

>>> import json, ast
>>> r = {u'name': u'A', u'primary_key': 1}
>>> ast.literal_eval(json.dumps(r))
{'name': 'A', 'primary_key': 1}
22
votes

The u characters that you are seeing simply mean that they are unicode strings.

If you do not want them to be unicode, you can encode them as something else, such as ASCII.

>>> s = u'hi!'
>>> s
u'hi'

>>> s2 = s.encode('ascii')
>>> s2
'hi'
7
votes

If you simply want to convert the dict to json data string you can do:

>>> from bson.json_util import dumps
>>> data = {u'name': u'A', u'primary_key': 1}
>>> dumps(data)
'{"name": "A", "primary_key": 1}'
3
votes

You need to let psycopg2 encode your strings, not try to insert Python-syntax strings into your queries raw — you are putting yourself in danger of a SQL injection problem if some of the strings contain characters that SQL will interpret as ending the string. You should pass parameters to psycopg2 like this:

cursor.execute('INSERT INTO person (name, town) VALUES (%s %s)', (name, town))

Because psycopg2 knows SQL syntax very, very well, it will leave off the u characters as it gets your name and town strings and quotes and escapes them in exactly the way that this SQL statement needs.

2
votes

As sven mentions in his comment, the u is an indication of the types represented in mongodb (actually it's because json is defined to use unicode).

This fact should be totally transparent to you, in fact you can use str and unicode values interchangeably in the dicts.

>>> 'foo' in {u'foo': 5}
True
>>> {u'foo': 5}['foo']
5
>>> 
0
votes

You cannot simply remove the u from the strings, as it symbolizes, that the strings are in Unicode.

One solution is to use the encode function:

old_strings = {u'name':u'A', u'primary_key':1}
newstrings = {}
for k in old_strings.keys():
    newtsrings[k] = old_strings[k].encode('ascii','ignore')

This would just ignore non ascii characters.

0
votes

Similar to what has already been posted, but using the json library.

>>> import json
>>> udata = {u'name': u'A', u'primary_key': 1}
>>> json.dumps(udata)
'{"name": "A", "primary_key": 1}'