Azure DevOps: Cosmos DB MongoError document does not contain shard key

When creating documents in Azure Cosmos databases for MongoDB API you might get the error message MongoError: document does not contain shard key.

The issue occurs for partitioned collections that have been created via the Azure CLI, because of the way the partitionKey path is being stored in the collection settings.

Reproduction steps

1. From the Azure Portal, create a Cosmos DB account with the Azure Cosmos DB for MongoDB API.

Account Name: xyzreprodbaccount

API: Azure Cosmos DB for MongoDB API

2. Create a database.

Database name: xyzreprodatabase

3. Create a collection with a shard key.

Collection id: collection1

Shard key: tid

4. Create a new document and save it.

a. Browse to Data Explorer > Database > Collection > Documents.

b. Click on New Document.

c. In the editor type in the following text:

{
     "id" : "1",
     "tid": "aaa"
}

d. Click Save.

e. Observe that the document has been created.

5. Create another collection using the Azure CLI.

You can create the collection also from the Azure Shell (https://shell.azure.com/). Use the following command:

$partitionKeyPath = '/tid'

az cosmosdb collection create --resource-group-name XYZ-repro --name xyzreprodbaccount --db-name xyzreprodatabase --collection-name collection2 --throughput 400 --partition-key-path $partitionKeyPath 

6. From the Azure Portal, browse the collection2 created at step 5 and insert a new document.

a. Browse to Data Explorer > Database > Collection > Documents.

b. Click on New Document.

c. In the editor type in the following text:

{
     "id" : "1",
     "tid": "aaa"
}

d. Click Save.

Expected result: the document to be saved

Actual result: error message “Command insert failed: document does not contain shard key”

When saving a document from an application, the error message is “MongoError: document does not contain shard key”.

Problem analysis

Looking at the collection properties using the command

az cosmosdb collection show --resource-group-name XYZ-repro --name xyzreprodbaccount --db-name xyzreprodatabase --collection-name collection1

we can see that for the first collection created from the Azure Portal, the path for the partitionKey is stored as:

"partitionKey": {
      "kind": "Hash",
      "paths": [
        "/'$v'/tid/'$v'"
      ]
    } 

While for the second collection created via Azure CLI, the path for the partitionKey is stored as:

"partitionKey": {
      "kind": "Hash",
      "paths": [
        "/tid"
      ]
    } 
Workaround

When creating collections via Azure CLI, specify the partition key as /’$v’/tid/’$v’

Here is the AZ CLI command:

$partitionKeyPath = '/tid'
$Bugfix_partitionKeyPath = '/''$v''' + $partitionKeyPath + '/''$v'''

az cosmosdb collection create --resource-group-name XYZ-repro --name xyzreprodbaccount --db-name xyzreprodatabase --collection-name collection2 --throughput 400 --partition-key-path $Bugfix_partitionKeyPath 

Azure DevOps: list all collections from all Azure Cosmos DB accounts

While creating an Azure Pipeline to backup all Azure Cosmos databases in a subscription, I had to list all collections from all databases. For that I wrote a script.

Feel free to adapt it in order to meet your needs! Enjoy!

#Retrieve database accounts
$databaseAccounts = Get-AzResource | where ResourceType -Like "*database*"

foreach($databaseAccount in $databaseAccounts){
    Write-Host "#RG:"  $databaseAccount.ResourceGroupName -ForegroundColor Green
    Write-Host "#   databaseAccountName:" $databaseAccount.Name -ForegroundColor Green
    
    #Retrieve databases
    $databases = az cosmosdb database list --resource-group-name $databaseAccount.ResourceGroupName --name $databaseAccount.Name
    $databasesConverted = $databases | ConvertFrom-Json
    foreach($database in $databasesConverted){
        $databaseName = $database.id
        Write-Host "#     databaseName:" $databaseName -ForegroundColor Green
        
        #Retrieve collections
        $allcollections = ""
        $collections = az cosmosdb collection list --resource-group-name $databaseAccount.ResourceGroupName --name $databaseAccount.Name --db-name $databaseName
        $collectionsConverted = $collections | ConvertFrom-Json
        foreach($collection in $collectionsConverted){
            $collectionName = $collection.id
            Write-Host "#       collectionName: " $collectionName -ForegroundColor Green
        }
    }
}

Create Azure Cosmos Databases programmatically using ARM templates and PowerShell

I have recently deployed tens of Azure resources programmatically. While most of the Azure resources and settings you can define in the ARM templates in JSON format, there are scenarios where you need to benefit from the power of PowerShell and the Azure CLI.

In this article I will demonstrate how to deploy an Azure Cosmos DB Account using an ARM template and then create the databases and collections using PowerShell.

To get the Azure templates for Visual Studio, install the Azure development SDK:

Create in Visual Studio a new project from type Azure Resource Group:

Edit the file azuredeploy.json like this:

{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"environmentName": {
"type": "string",
"defaultValue": "ranari"
}
},
"variables": {},
"resources": [
{
"apiVersion": "2015-04-08",
"name": "[concat(parameters('environmentName'),'mongodb')]",
"type": "Microsoft.DocumentDB/databaseAccounts",
"kind": "MongoDB",
"location": "[resourceGroup().location]",
"scale": null,
"properties": {
"ipRangeFilter": "",
"enableAutomaticFailover": false,
"enableMultipleWriteLocations": false,
"isVirtualNetworkFilterEnabled": false,
"virtualNetworkRules": [],
"databaseAccountOfferType": "Standard",
"consistencyPolicy": {
"defaultConsistencyLevel": "Session",
"maxIntervalInSeconds": 5,
"maxStalenessPrefix": 100
},
"locations": [
{
"locationName": "[resourceGroup().location]",
"failoverPriority": 0
}
],
"capabilities": []
},
"dependsOn": [],
"tags": {}
}
],
"outputs": {}
}

Deploy the solution by following these steps:

Browse the Azure site and observe that the Azure Cosmos DB Account has been created:

At this point the Azure Cosmos DB account is empty. To add a database and collection, run following PowerShell script:

#####################################################

#Parameters & variables

param(
[Parameter(Mandatory=$true)][String]$environmentName,
[Parameter(Mandatory=$false)][String]$subscriptionId = "e9c6149c-f5f6-44c7-aa8b-b8d2fa7f6140"
)

$resourceGroup = $environmentName.ToLower() + "-databases"
$mongoDBAccount = $environmentName.ToLower() + "mongodb"
$mongoDBdatabase = $environmentName.ToLower() + "-cars"

$mongoDBdatabaseCollections=@("VW",
"BMW",
"Mercedes",
"Audi",
"Skoda",
"Renault",
"Ford")

#####################################################

#Connect to Azure
Connect-AzAccount
az account set --subscription $subscriptionId

#####################################################
#Create databases and collections

az cosmosdb database create --resource-group-name $resourceGroup --db-name $mongoDBdatabase --name $mongoDBAccount

foreach($collection in $mongoDBdatabaseCollections)
{
az cosmosdb collection create --resource-group-name $resourceGroup --name $mongoDBAccount --db-name $mongoDBdatabase --collection-name $collection --throughput 400
}
#####################################################

Now check in Azure that the database and collection were successfully created: