Using Microsoft Azure CosmosDB

You can connect Vonk to CosmosDB the same way you connect to MongoDB. There are a few limitations that we will work out later. They are listed below.

Attention

You cannot use CosmosDb for the Vonk Administration database. Use SQLite instead.

  1. Create a CosmosDB account on Azure, see the Quickstart Tutorial

  2. Make sure you choose the MongoDB API

  3. In the Azure Portal, open your CosmosDB account and go to the ‘Connection Strings’ blade. Copy the ‘Primary Connection String’ to your clipboard.

  4. Now on your own machine, navigate to your Vonk working directory

  5. In a text editor open appsettings.json to find the Repository setting:

    "Repository": "Sqlite",
    
  6. Change the setting to CosmosDb

  7. If you have your own database in CosmosDB already, change the CosmosDbOptions to reflect your settings:

    "CosmosDbOptions": {
        "ConnectionString": "<see below>",
        "EntryCollection": "vonkentries",
        "SimulateTransactions": "false"
    },
    

    Paste the ConnectionString from step 3, and add the databasename that you want to use. The connectionstring looks like this:

    mongodb://<accountname>:<somerandomstring>==@<accountname>.documents.azure.com:10255?ssl=true&replicaSet=globaldb
    

    You can add the databasename after the portnumber, like this:

    mongodb://<accountname>:<somerandomstring>==@<accountname>.documents.azure.com:10255/vonk?ssl=true&replicaSet=globaldb
    
  8. If your CosmosDB account does not have a database or collection by this name, Vonk will create it for you.

  9. You can set SimulateTransactions to “true” if you want to experiment with FHIR transactions. Vonk does not utilize the CosmosDB way of supporting real transactions across documents, so in case of an error already processed entries will NOT be rolled back.

CosmosDB Request Units

If you upload a lot of data in a short time (as is done on reindexing), you quickly exceed the default maximum of 1000 Request Units / second. If you encounter its limits, the Vonk log will contain errors stating ‘Request rate is large’. This is likely to happen upon reindexing or when using Vonkloader. Solutions are:

  • Raise the limit to at least 5000 RU/s. See the Microsoft documentation for instructions.
  • Lower the load
    • on Reindexing, lower the MaxDegreeOfParallelism, see this warning
    • with Vonkloader, lower the value of the -parallel parameter.

Limitations

  1. Request size for insertions to CosmosDB is limited to around 5 MB. Some bundles in the examples from the specification exceed that limit. Then you will get an error stating ‘Request size too large’. You can avoid this by limiting the size of incoming resources in the SizeLimits setting.
  2. The CosmosDB implementation of the MongoDB API is flawed on processing $not on arrays. This inhibits the use of these searches in Vonk:
    • Using the :not modifier
    • Using :missing=true