Fiware Orion batch duplicate

Question

I'm using APPEND_STRICT, but having trouble understanding a certain concept.

Example, I have a single entity in Fiware Orion(already created) and want to create let's say 1000 entities in batch using APEND_STRINCT(v2/op/update).

In 1000 entities there is 1 duplicate(an entity I mention that is already in Orion).

So is this correct, Orion will throw error 422 without any information in term of the id of an entity that already exists, error talk about attributes of the entity (I understand why it is the concept of APPEND_STRICT) but showing it would really help.

And another part is if the entity which is duplicate was on position 400 then Orion send error but continue to write remaining entities, this is really hard to manage because I cannot know when a total write is done and have to show some response while Orion still works on them in the background.

Are my assumptions correct and can be something done to avoid this, something I failed to notice.

Thanks.

Edit

Error message:

{ error: 'Unprocessable',
    description: 'one or more of the attributes in the request already exist: 
    [ family, serialNumber, refSortingType, description, refType, storedWasteOrigin, location, address, fillingLevel, cargoWeight, temperature, methaneConcentration, regulation, responsible, owner, dateServiceStarted, dateLastEmptying, nextActuationDeadline, actuationHours, openingHours, dateLastCleaning, nextCleaningDeadline, refDepositPointIsle, status, color, image, annotations, areaServed, dateModified, refDevice ]' } } }

Example request:

{ method: 'POST',
 headers:
  { 'Content-Type': 'application/json',
    'Fiware-Service': 'waste4think',
    'Fiware-ServicePath': '/d',
    'X-Auth-Token': 'DssfKZe82e1dyJof416EmrQPdFQ3QK1' },
 uri: 'http://localhost:1026/v2/op/update',
 body: { actionType: 'APPEND_STRICT', entities: [Array] }

     {"actionType":"APPEND_STRICT","entities":[{"id":"xxx","type":"xxx","family":{"value":"Agent","type":"String","metadata":{}},"serialNumber":{"value":"","type":"String","metadata":{}},"refSortingType":{"value":"SortingType:2","type":"String","metadata":{}},"description":{"value":"","type":"String","metadata":{}},"refType":{"value":"DepositPointType:0","type":"String","metadata":{}},"storedWasteOrigin":{"value":"","type":"String","metadata":{}},"location":{"value":{"type":"Point","coordinates":[xxx]},"type":"geo:json"},"address":{"value":"xxxxx.","type":"String","metadata":{}},"fillingLevel":{"value":0,"type":"Float","metadata":{"unit":{"value":"C62","type":"String"}}},"cargoWeight":{"value":0,"type":"Float","metadata":{"unit":{"value":"KGM","type":"String"}}},"temperature":{"value":0,"type":"Float","metadata":{"unit":{"value":"CEL","type":"String"}}},"methaneConcentration":{"value":0,"type":"Float","metadata":{"unit":{"value":"59","type":"String"}}},"regulation":{"value":"Municipal association","type":"String","metadata":{}},"responsible":{"value":"","type":"String","metadata":{}},"owner":{"value":"xxx","type":"String","metadata":{}},"dateServiceStarted":{"value":"","type":"String","metadata":{}},"dateLastEmptying":{"value":"","type":"String","metadata":{}},"nextActuationDeadline":{"value":"","type":"String","metadata":{}},"actuationHours":{"value":[],"type":"List","metadata":{}},"openingHours":{"value":[],"type":"List","metadata":{}},"dateLastCleaning":{"value":"","type":"String","metadata":{}},"nextCleaningDeadline":{"value":"","type":"String","metadata":{}},"refDepositPointIsle":{"value":"","type":"String","metadata":{}},"status":{"value":"ok","type":"String","metadata":{}},"color":{"value":"","type":"String","metadata":{}},"image":{"value":"","type":"String","metadata":{}},"annotations":{"value":"","type":"String","metadata":{}},"areaServed":{"value":"","type":"String","metadata":{}},"dateModified":{"value":"","type":"String","metadata":{}},"refDevice":{"value":"","type":"String","metadata":{}}}]}

As for the request, I split the post part and body part. As you can see by error msg is not possible to know what entity caused this

To be fully sure about your question, could you edit it to include a draft of the request (not all the 1000 entities of course, but a fragment of it) and the response you get, please? Thanks! — fgalan

fgalan fgalan · Accepted Answer · 2019-03-14T16:52:46

I think the functionality is as you describe. Orion responses with a list of the attributes that already exist but not to which entity they belong. A response like this could be probably more useful:

'one or more of the attributes in the request already exist: 
   entity23: [ family, serialNumber], entity 42: [refSortingType, description]'

with some capping (e.g. as much as 20 entities) to preclude too bigs responses.

If you think implementing something like that could be insteresting, please create a new issue in the Orion repository about it, please.

Some additional comments:

APPEND_STRICT is deprecated. The right keyword is appendStrict.
Regarding "Orion send error but continue to write remaining entities, this is really hard to manage because I cannot know when a total write is done and have to show some response while Orion still works on them in the background". Orion doesn't response until it finishes to process the whole batch in the POST /v2/op/entity request. So your REST client can be sure that when the response is received everything has been processed (although that processing could involve errors due to duplicated attributes in some entities, as we have been discussing). Have you experience a different behaviour? In that case, how did you get it? (it Orion is behaving that way it could be a bug and I'd like to know about in order to debug it).

Fiware Orion batch duplicate

1 Answers