C# MongoDB Azure Cosmos Pull Filter

Question

Issue Summary - When doing a pull filter against a nested array of objects using the Lt operator, Azure Cosmos Mongo API does not remove the expected elements.

.NET Version - .NET Core 3.1

MongoDB.Driver Version - 2.10.2

Here are the details for what I am trying to do and what is happening.

I have the following document

{
    "_id" : ObjectId("5e80b59326773581903da606"),
    "item" : "journal",
    "instock" : [
        {
            "warehouse" : "A",
            "qty" : 15
        },
        {
            "warehouse" : "B",
            "qty" : 5
        },
        {
            "warehouse" : "C",
            "qty" : 2
        }
    ]
}

What I want to do is remove any elements from the "instock" array where qty is less than 10. Here is my C# code using the MongoDB.Driver nuget package where I try to do this.

static void Main(string[] args)
{
    var item = "journal";
    var collection = GetCosmosCollection();
    collection.DeleteOne(i => i.item == item); // Delete the existing document
    InsertSampleData(collection, item); // Insert the sample JSON document shown above

    var filter = Builders<InventoryItem>.Filter.Eq("item", item);

    // Remove from the instock array where qty is less than 10
    var stockPullFilter = Builders<InventoryItem>.Update.PullFilter(
        "instock", 
        Builders<Stock>.Filter.Lt(s => s.qty, 10)
    );

    collection.FindOneAndUpdate<InventoryItem>(
        filter, 
        stockPullFilter
    );
}

When I go view the collection in the Azure Cosmos data explorer, I still see all of the objects listed in "instock".

If I change the operator to "Eq" instead of "Lt", and match on an exact qty, then it works.

// Remove from the instock array where qty equals 5
var stockPullFilter = Builders<InventoryItem>.Update.PullFilter(
    "instock", 
    Builders<Stock>.Filter.Eq(s => s.qty, 5)
);

So to me this seems to be a bug with using the Lt operator. I tried with Lte, Gt, and Gte and experience the same issue.

I then decided to spin up a collection using Mongo Atlas and see what happened. The same exact C# code works against Atlas using the Lt operator. So at this point it appears to just be an issue with Cosmos.

Next, I recreated this logic just using mongo shell commands in order to figure out if this is a problem with the mongo .net driver or with Cosmos.

// Delete the test item to make sure we start fresh
db.inventory.deleteOne( { "item": "journal" } )

// Insert a test item, with 3 instock elements
db.inventory.insertOne( { "item": "journal", "instock": [{warehouse: "A", qty: 15}, {warehouse: "B", qty: 5}, {warehouse: "C", qty: 2}] } );

// Find the test item to confirm it insert successfully
db.inventory.find( { "item": "journal" } ).pretty()

// Pull (remove) from the instock array where the quantity is less than 10
db.inventory.update({ item: "journal" }, { $pull: { instock: { qty: { $lt: 10 } } } }, { multi: true })

// Find the test item to confirm the stock is removed as expected
db.inventory.find( { "item": "journal" } ).pretty()

Executing this against Atlas works fine while executing this against Cosmos does not remove any instock elements.

Obviously this is an issue with Azure Cosmos' Mongo API implementation, but my question is this:

Is there some other way to write this pull command or will I need to reach out to Microsoft support and raise this issue with them?

Thanks

Is it possible you only have a hash index on that property in your Cosmos collection? That would explain why equality works but less than does not. — Mark Brown
Good idea, but that doesn't appear to be the issue. I tried adding an index and that didnt help. I also listed out the indexes between Cosmos and Atlas and they are the same. — enowapiseals

Mark Brown Mark Brown · Accepted Answer · 2020-04-01T18:37:39

This appears to be a bug in our server version 3.2 that was fixed in 3.6. If you are willing to upgrade to 3.6 you can either create a new Cosmos account and select 3.6 or file a support ticket and we will do an inplace upgrade. Thanks.

C# MongoDB Azure Cosmos Pull Filter

1 Answers