Sharing Memcache with PHP and Python

Question

I am trying to share a Memcache key between Python and PHP. Python writes the key and PHP reads it. I am using the Cakephp framework, with php-pecl-memcache (not php-pecl-memcached), and the python-memcache all python library.

Python:

  mc = memcache.Client( ["127.0.0.1:11211"])
  key = "key1"
  value = 1323779849
  mc.set(key, value)

PHP:

  echo Cache::read('key1', 'memcached');

PHP can't read the variable, I get weird "MemcachePool::get() [http://php.net/memcachepool.get]: Failed to uncompress data" errors; I suspect it has to do with memcached flags that are set differently in each library.

This is what happens when I telnet to memcached:

Python sets key:

 get key1
 VALUE key1 1 12
 1323779849
 .
 END

PHP sets key:

 get key1
 VALUE key 1 0 12
 1323779849
 END

Is there a way to override these flags and just 'do your own thing'?
Are there php/python memcache libraries that play better together?
Alternatively, is there a simple way to create a common memory space between python/php?

thanks Will, but all I am trying to do is share a simple data type, either an int or a string, so there is no need to serialize, pickle or even compress. And I would rather not write to a file because of the performance impact although yes, that would do the trick. — Michael Franze
If you've tried integers and strings I don't think you can get away from using (consistent) serialization. — Will Bickford
You might missing some points, memcache has an option to compress and serialize data (even is just a single byte). This is more for internal usage. I guess python API is enable compression by default, while your PHP api did not enable it by default. You did not mention what exactly API you used for memcache, So, I guess nobody can really dig the documentation and provide an insight answer. — ajreal
Thanks Ajreal, but in this case it was simpler because all I was trying to do was store an Integer and neither PHP or Python were trying to compress the data. It really comes down to PHP and Python using different flags to represent the same thing. I have explained all in my Answer below. — Michael Franze

Michael Franze Michael Franze · Accepted Answer · 2011-12-14T11:22:41

Finally got it to work. Lot's of stuff wasn't working as expected.

One problem is that php and python use different flags to do different things. Not a problem in an all-python or all-php solution, but for inter-environment communication a real problem. A useful resource is http://www.hjp.at/zettel/m/memcached_flags.rxml, which shows that python-memcache flags long integer as '2', which php-memcache does not understand, hence the compression error. I amended python-memcache to include a 'flag-override' in the set function. This variable simply forces a particular flag irrespective of what python memcache thinks it ought to be. This allowed me to re-flag Int from 2 to 0. I will prob branch the current version of python-memcache and submit it to Github. This allowed me to force the python long int flag (2) to something php would understand (0).
CakePhp prior to 1.3.3 stores all keys in memcached with an additional key_expires key, flagged as 768, etc, etc. Without this additional key it cannot find the key you are looking for. Thankfully this behaviour was dumped in later Cakephp version (I simply upgraded to 1.3.13) and it works well now.

Sharing Memcache with PHP and Python

3 Answers