google app engine 1.7.6 memcache viewer (Python)

Question

I've been having some trouble with the memcache viewer after updating the Python Dev Appserver to Google App Engine 1.7.6 (with Python 2.7).

It appears that my memcache isn't updated or isn't readable. I have tried to view memcache with the app engine memcache viewer but when I input the memcache key I get an error.

when I flush the cache everything proceeds as normal until memcache needs to be read again...

The hit ratio and memcache size increases as normal, so there is something in the cache. Also when I revert back to app engine 1.7.5 everything works just fine. Perhaps someone else has had this issue?

When I input the memcache key I get the following:

Traceback (most recent call last):
  File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.1\webapp2.py", line 1536, in __call__
    rv = self.handle_exception(request, response, e)
  File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.1\webapp2.py", line 1530, in __call__
    rv = self.router.dispatch(request, response)
  File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.1\webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.1\webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\devappserver2\admin\admin_request_handler.py", line 80, in dispatch
    super(AdminRequestHandler, self).dispatch()
  File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.1\webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.1\webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\devappserver2\admin\memcache_viewer.py", line 145, in get
    values['value'], values['type'] = self._get_memcache_value_and_type(key)
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\devappserver2\admin\memcache_viewer.py", line 74, in _get_memcache_value_and_type
    except (pickle.UnpicklingError, AttributeError, EOFError, ImportError,
NameError: global name 'pickle' is not defined

I tried including an "import pickle" in my main.py but this was in vain.

I've included some samples of my code but hopefully this isn't necessary I hope its something more to do with the app engine update than my code...

some of my main.py file:

#import pickle
from google.appengine.api import memcache
from google.appengine.ext import db

and a sample function for how I handle memcache:

def mc_get(key):
    a = memcache.get(key)
    if a:
        val = a
    else:
        val = None
    return val

def mc_set(key, val):
    memcache.set(key, val)

and if I want to query the users in my db I use:

def get_users(update=False):
    mc_key = 'USERS'
    entries = mc_get(mc_key)
    if update or entries is None:
        a = User.all()
        logging.error('DB---Q - Users')
        entries = list(a)
        memcache.set(mc_key, entries)
    return entries

UPDATE: I added "import pickle" to the memcache_viewer.py file in Google\google_appengine\google\appengine\tools\devappserver2\admin\memcache_viewer.py (is this a bug??)

and now when I type in a memcache key I get the following error under the memcache key input field: Error fetching USERS: Failed to retrieve value from cache: No module named main

Any help would be greatly appreciated, thanks in advance.

I've tried to modify the memcache_viewer.py file in C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\devappserver2\admin\memcache_viewer.py, but I'm struggling with the permissions... However I struggle to believe that the update in 1.7.6 would require me to include something in one of the source files, I didn't add import pickle when using 1.7.5 — Emile Esterhuizen
This is definitely a bug. It happens when you store more complex structures (I think objects, but not fully tested). I did some simple tests with memcached strings and dictionaries and it works just fine. I'm getting pretty fed up with all of the recent bugs. — kevin
You should post this in Google App Engine's Issue tracker - code.google.com/p/googleappengine/issues/list — kevin
Ok cool, I'll post it to Google App Engine's Issue tracker, thanks for the input. I've been playing around with the deployed app and its quite fickle, still found no solution. If there's a solution I'll post it. — Emile Esterhuizen

Emile Esterhuizen Emile Esterhuizen · Accepted Answer · 2013-04-06T12:02:33

I changed from the old datastore API to NDB(a bit of a chore changing the code). The automatic caching seems to have solved the problem, this may suggest that the problem was with my code, but still doesn't explain why everything worked fine when using app engine 1.7.5 and not with 1.7.6.

I'll remove this answer if anyone has an alternative, just thought I'd post my progress in case anyone else is having the same problem.

google app engine 1.7.6 memcache viewer (Python)

1 Answers