How can I bulk insert with Elastic low-level-client for nested typed object?

Question

Im trying to insert whole data with nested data types, but low-level-client(c#) couldn't insert.

Latest elasticsearch documents doesn't explain how can we do this.

So, I'm not sure what I need to do. Could someone plase help me about that ?

VendorProduct was created as nested type.

Mapping:

"properties": {

            "vendorProducts": {
                "properties": {
                    ...
                },
                "type": "nested"
            },

        }

Bulk Insert Request on Bulk API;

{"index":{"_index":"productindextemp-local82","_type":"elasticproduct","_id":"1715"}}
{
    "id": 1715,
    "productTypeId": 5,
...

    "vendorProducts": [
        {
            "id": 124550,
            "productId": 1715,
            "vendorId": 8,
..
        },
        {
            "id": 451542,
            "productId": 1715,
            "vendorId": 15,
            ..
        }
    ]
}

Server: AWS Elastic Search Service v7.1

c# References: Elasticsearch.Net v7.4.1 Nest v7.4.1

Our C# side code is ;

var settings = new ConnectionSettings(new System.Uri(ElasticUrl));
settings.DefaultMappingFor<ElasticProduct>(d => d.IndexName(newTempIndexName));
settings.RequestTimeout(TimeSpan.FromHours(5));
var client = new ElasticClient(settings);




if (!client.Indices.Exists(newTempIndexName).Exists)
{
    client.Indices.Create(
        newTempIndexName,
        c => c.Map<ElasticProduct>(m => m
            .AutoMap()
            .Properties(p => p
                .Nested<List<ElasticVendorProduct>>(n => n
                    .Name(nn => nn.VendorProducts)
                    .AutoMap()
                )
             )
        )
    );
}



StringBuilder stringBuilder = new StringBuilder();
foreach (var data in productBulkDataItems)
    stringBuilder.Append(data).Append("\n");
var result = client.LowLevel.Bulk<BulkResponse>(newTempIndexName, stringBuilder.ToString());

We've received this error after request has been sent. Because of that, result is null. I couldnt handle this problem. We have extracted Json request from Nest client on .net then we try to sent request manually on kibana. This method is also gave to us same result.

Exception:

 {index returned 400 _index: productindextemp-local82 _type: elasticproduct _id: 1715 _version: 0 error: Type: illegal_argument_exception Reason: "object mapping [vendorProducts] can't be changed from nested to non-nested"}

Please can you edit your question to show what you're doing to get the exception you see — Russ Cam
What is productBulkDataItems? What are the definitions for ElasticProduct and ElasticVendorProduct? — Russ Cam

eyupgevenim eyupgevenim · Accepted Answer · 2019-11-22T10:41:28

When you are trying to create index, you have to chagnge index name from "elasticproduct" to "_doc". Your output data should look like that:

    "mappings": {
        "_doc": {
            "properties": {
            ...
            }
        }
    }

Try "_type":"_doc" instead of "_type":"elasticproduct".

How can I bulk insert with Elastic low-level-client for nested typed object?

1 Answers