ERROR oslo_messaging.rpc.dispatcher [req-xxxxx] Exception during message handling: 'metadata'

0
votes

I installed Openstack(Kilo) on Centos7

The error occurred in adding the security group rules

I'm using Nova-network.

I got this error form nova-compute.log

2015-07-31 16:55:05.669 5950 ERROR oslo_messaging.rpc.dispatcher [req-76584058-8dbd-4860-a048-e9dbad712779 843c47b4a71b4ac3a6c4375d558aa423 3f4e979ea9b9409a9425442a8b096457 - -
-] Exception during message handling: 'metadata'
2015-07-31 16:55:05.669 5950 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2015-07-31 16:55:05.669 5950 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2015-07-31 16:55:05.669 5950 TRACE oslo_messaging.rpc.dispatcher     executor_callback))
2015-07-31 16:55:05.669 5950 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2015-07-31 16:55:05.669 5950 TRACE oslo_messaging.rpc.dispatcher     executor_callback)
2015-07-31 16:55:05.669 5950 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
2015-07-31 16:55:05.669 5950 TRACE oslo_messaging.rpc.dispatcher     result = func(ctxt, **new_args)
2015-07-31 16:55:05.669 5950 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 6716, in refresh_instance_security_rules
2015-07-31 16:55:05.669 5950 TRACE oslo_messaging.rpc.dispatcher     return self.manager.refresh_instance_security_rules(ctxt, instance)
2015-07-31 16:55:05.669 5950 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 434, in decorated_function
2015-07-31 16:55:05.669 5950 TRACE oslo_messaging.rpc.dispatcher     args = (_load_instance(args[0]),) + args[1:]
2015-07-31 16:55:05.669 5950 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 425, in _load_instance
2015-07-31 16:55:05.669 5950 TRACE oslo_messaging.rpc.dispatcher     expected_attrs=metas)
2015-07-31 16:55:05.669 5950 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/objects/instance.py", line 492, in _from_db_object
2015-07-31 16:55:05.669 5950 TRACE oslo_messaging.rpc.dispatcher     instance['metadata'] = utils.instance_meta(db_inst)
2015-07-31 16:55:05.669 5950 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/nova/utils.py", line 816, in instance_meta
2015-07-31 16:55:05.669 5950 TRACE oslo_messaging.rpc.dispatcher     if isinstance(instance['metadata'], dict):
2015-07-31 16:55:05.669 5950 TRACE oslo_messaging.rpc.dispatcher KeyError: 'metadata'
2015-07-31 16:55:05.669 5950 TRACE oslo_messaging.rpc.dispatcher

Thank you warm-hearted!

2
openstackopenstack-nova

2 Answers

0
votes

I also faced the same error.

Try adding the rules to the security group first and then create the instance. It worked for me.

This is a work around only. Since I don't have enough privileges to comment I am adding this as an answer.

If I get to know about any solution I will post it here.

0
votes

Although it's been 4 years, probably nobody even cares what the root cause is but I put it here anyway, since I did some (re)search.

The object_compat decorator expects to get the Instance object with 'metadata' and 'system_metadata' attributes but if those aren't in the db instance dict object, Instance._from_db_object will fail with a KeyError.

In Kilo this happens per refresh_instance_security_rules because in the compute API code, the instance passed to refresh_instance_security_rules comes from the call to get the security group(s) which joins on the instances column, but that doesn't join on the metadata/system_metadata fields for the instances. So when the instances get to object_compat in the compute manager and the db instance dict is converted to the Instance object, it expects fields that aren't in the dict and we get the KeyError.

Refer to the following code commit for details.

https://opendev.org/openstack/nova/commit/9369aab04e37b7818d49b00e65857be8b3564e9e