Is there a way to control when Azure CosmosDB writes get picked up by reads (stage data)?


Author: Hari Durairaj

Originally Sourced from: https://stackoverflow.com/questions/68474576/is-there-a-way-to-control-when-azure-cosmosdb-writes-get-picked-up-by-reads-sta

Scenario - We have multiple containers within a DB instance and each of the containers are being upserted by independent Data Factory pipelines daily (with a timestamp). Among the container instances, when the data is read, we want to ensure that certain set of containers always return the same date viz., if pipeline1 completed upserting into container1 on date 1 and date 2 (making date2 the latest) but pipeline2 is still in the process of upserting date 2 (making date1 the latest), we do not want to container1 to return date2 info until pipeline2 has completed

One possible way this can be achieved is if there was a way for us to control when the write region updates get reflected on the read regions (basically an automation which looks at pipeline runs and then if all looks good, turn on the sync from write to read region). Is this possible? What we are looking for is a way to 'stage' data before it can be pushed to the read regions.