CosmosDB Entity with the specified id does not exist

Question

I am developing an ASP.NET Core MVC API to call resources in an Azure Cosmos DB. When I try to perform a GET for any specific ID, I receive DocumentClientException: Entity with the specified id does not exist in the system. I can confirm that the entity does exist in the system, and the connection is successful because I can successfully perform other methods and requests. The partition key is _id .

Debugging with breakpoints in Visual Studio, I can see where the correct ID is received at the API, but I can't confirm what specifically it is sending to Azure

The controller methods: (the ID field is a random string of numbers and text)

        //controller is MoviesController decorated with [Route(api/[controller])]
    //sample GET is to localhost:port/api/Movies/5ca6gdwndkna99
        [HttpGet("{id}")]
        public async Task<MoviesModel> Get(string id)
        {
            MoviesModel movie = await _persistence.GetMovieAsync(id);
            return movie;
        }

The data handling method:

public async Task<MoviesModel> GetMovieAsync(string Id)
        {

            string _id = Id;
            RequestOptions options = new RequestOptions();
            options.PartitionKey = new PartitionKey(_id);
            var documentUri = UriFactory.CreateDocumentUri(_databaseId, "movies", Id);
            Document result = await _client.ReadDocumentAsync(documentUri,options);

            return (MoviesModel)(dynamic)result;
        }

Other methods, like getting a list of all movies and returning to a table are working fine, so we can rule out network issues

public async Task<List<MoviesModel>> GetMoviesAsync() 
{
var documentCollectionUri = UriFactory.CreateDocumentCollectionUri(_databaseId, "movies");

            // build the query
            var feedOptions = new FeedOptions() { EnableCrossPartitionQuery = true };
            var query = _client.CreateDocumentQuery<MoviesModel>(documentCollectionUri, "SELECT * FROM movies", feedOptions);
            var queryAll = query.AsDocumentQuery();

            // combine the results
            var results = new List<MoviesModel>();
            while (queryAll.HasMoreResults)
            {
                results.AddRange(await queryAll.ExecuteNextAsync<MoviesModel>());
            }

            return results;
        }


        public async Task<List<GenresModel>> GetGenresAsync()
        {
            await EnsureSetupAsync();

            var documentCollectionUri = UriFactory.CreateDocumentCollectionUri(_databaseId, "genres");

            // build the query
            var feedOptions = new FeedOptions() { EnableCrossPartitionQuery = true };
            var query = _client.CreateDocumentQuery<GenresModel>(documentCollectionUri, "SELECT * FROM genres", feedOptions);
            var queryAll = query.AsDocumentQuery();

            // combine the results
            var results = new List<GenresModel>();
            while (queryAll.HasMoreResults)
            {
                results.AddRange(await queryAll.ExecuteNextAsync<GenresModel>());
            }

            return results;
        }

Answer 1

Firstly, I would suggest to re-look at your cosmosDb design once, bcz of the following reasons...

Problems:

1. If your _id is random string of numbers and text, then its not good to have the entire _id as your partition key, bcz this would create a new partition for each entry.(although azure will range parition it later)

2. Querying just by partition key is not efficient, for pin point queries we should have both partition key and row key.

Solution:

1. Make the first one or two letters of your _id as your partition key. (so your partitions will be finite).

2. Make your _id as your row key.

3. If your _id = "abwed123asdf", then your query should be..

   RequestOptions options = new RequestOptions();

   options.PartitionKey = new PartitionKey(_id.Substring(0,1));

   options.RowKey = _id;

This way, your look up will pin point to the exact required entry with the help of partition and row key. (saves lot of RUs)

Please refer docs for choosing a better partition keys for your needs https://docs.microsoft.com/en-us/azure/cosmos-db/partitioning-overview

Answer 2

I was able to get this to work by completely refactoring to the dotnet v3 SDK. My code for the solution is in the comments of the gitHub link:

using Microsoft.Azure.Cosmos;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using VidlyAsp.DataHandlers;

namespace VidlyAsp.DataHandlers
{
    public class PersistenceNew
    {

        private static string _endpointUri;
        private static string _primaryKey;
        private CosmosClient cosmosClient;

        private CosmosDatabase database;
        private CosmosContainer movieContainer;
        private CosmosContainer genreContainer;
        private string containerId;
        private string _databaseId;


        public PersistenceNew(Uri endpointUri, string primaryKey)
        {
            _databaseId = "Vidly";
            _endpointUri = endpointUri.ToString();
            _primaryKey = primaryKey;
            this.GetStartedAsync();
        }
        public async Task GetStartedAsync()
        {
            // Create a new instance of the Cosmos Client
            this.cosmosClient = new CosmosClient(_endpointUri, _primaryKey);
            database = await cosmosClient.Databases.CreateDatabaseIfNotExistsAsync(_databaseId);
            CosmosContainer moviesContainer = await GetOrCreateContainerAsync(database, "movies");
            CosmosContainer genresContainer = await GetOrCreateContainerAsync(database, "genres");
            movieContainer = moviesContainer;
            genreContainer = genresContainer;
        }

        public async Task<GenresModel> GetGenre(string id)
        {

            var sqlQueryText = ("SELECT * FROM c WHERE c._id = {0}", id).ToString();
            var partitionKeyValue = id;

            CosmosSqlQueryDefinition queryDefinition = new CosmosSqlQueryDefinition(sqlQueryText);


            CosmosResultSetIterator<GenresModel> queryResultSetIterator = this.genreContainer.Items.CreateItemQuery<GenresModel>(queryDefinition, partitionKeyValue);

            List<GenresModel> genres = new List<GenresModel>();

            while (queryResultSetIterator.HasMoreResults)
            {
                CosmosQueryResponse<GenresModel> currentResultSet = await queryResultSetIterator.FetchNextSetAsync();
                foreach (GenresModel genre in currentResultSet)
                {
                    genres.Add(genre);

                }
            }
            return genres.FirstOrDefault();
        }

        public async Task<MoviesModel> GetMovie(string id)
        {

            var sqlQueryText = "SELECT * FROM c WHERE c._id = '" + id + "'";
            var partitionKeyValue = id;

            CosmosSqlQueryDefinition queryDefinition = new CosmosSqlQueryDefinition(sqlQueryText);


            CosmosResultSetIterator<MoviesModel> queryResultSetIterator = this.movieContainer.Items.CreateItemQuery<MoviesModel>(queryDefinition, partitionKeyValue);

            List<MoviesModel> movies = new List<MoviesModel>();

            while (queryResultSetIterator.HasMoreResults)
            {
                CosmosQueryResponse<MoviesModel> currentResultSet = await queryResultSetIterator.FetchNextSetAsync();
                foreach (MoviesModel movie in currentResultSet)
                {
                    movies.Add(movie);

                }
            }
            return movies.FirstOrDefault();
        }

        /*
    Run a query (using Azure Cosmos DB SQL syntax) against the container
*/
        public async Task<List<MoviesModel>> GetAllMovies()
        {
            List<MoviesModel> movies = new List<MoviesModel>();

            // SQL
            CosmosResultSetIterator<MoviesModel> setIterator = movieContainer.Items.GetItemIterator<MoviesModel>(maxItemCount: 1);
            while (setIterator.HasMoreResults)
            {
                foreach (MoviesModel item in await setIterator.FetchNextSetAsync())
                {
                   movies.Add(item);
                }
            }

            return movies;
        }

        public async Task<List<GenresModel>> GetAllGenres()
        {
            List<GenresModel> genres = new List<GenresModel>();

            // SQL
            CosmosResultSetIterator<GenresModel> setIterator = genreContainer.Items.GetItemIterator<GenresModel>(maxItemCount: 1);
            while (setIterator.HasMoreResults)
            {
                foreach (GenresModel item in await setIterator.FetchNextSetAsync())
                {
                    genres.Add(item);
                }
            }

            return genres;
        }

        private static async Task<CosmosContainer> GetOrCreateContainerAsync(CosmosDatabase database, string containerId)
        {
            CosmosContainerSettings containerDefinition = new CosmosContainerSettings(id: containerId, partitionKeyPath: "/_id");

            return await database.Containers.CreateContainerIfNotExistsAsync(
                containerSettings: containerDefinition,
                throughput: 400);
        }
    }
}