The introduction of transactions in MongoDB 4.0 represents possibly the most significant change in MongoDB's architecture since its original release. The lack of a transactional capability previously defined the capabilities of the database: Without transactions, MongoDB was blocked from consideration for a wide range of application scenarios. With the implementation of transactions, MongoDB can for the first time truly claim to be a general purpose DBMS.
Now that we have access to the MongoDB 4.0 documentation and beta software, it’s a good time to look at the internal implementation of MongoDB transactions.
In general, MongoDB has not reinvented the wheel in its implementation of transactions. Rather, it has adopted most of the familiar transactional patterns implemented in MySQL, Oracle, and other ACID relational databases.
Let’s look briefly at these key elements of MongoDB transactions:
From a programming standpoint, MongoDB transactions must be explicitly enabled by issuing a start_transaction method against a session object.
This session object is a relatively new feature of MongoDB. One of Mongos simplifying optimizations was to regard every MongoDB operation as essentially independent. This is still the case in MongoDB 4.0 by default, but applications can optionally create a session and then associate operations with that session. Operations from a single session can then be combined into transactions.
In ACID relational databases, a System Change Number (SCN) or transaction ID is used to keep data consistent within and without transactions. When a transaction commences this SCN number is incremented. Database sessions use the SCN to determine if changes to the database should be visible. Typically a query cannot see a change in the database if the change’s SCN is higher than the SCN that existed when the query commenced. MongoDB implements a mechanism similar to the SCN using timestamps.
In databases such as Oracle, read consistency (“cursor stability”) is implemented for queries transparently. A query only ever sees data that existed when the query commenced. That is, if a transaction alters data after the query starts, but before the query finishes, this data will not be returned by the query. In MongoDB 4.0 this behavior is optional and requires that that a “snapshot” transaction be commenced before the query is executed. These “read only” transactions will always return versions of the data that existed when the transaction was commenced.
The ability to read some previous version of a data item requires that multiple versions of a document must be maintained: the famous Multi Version Concurrency Control (MVCC) pattern. The MongoDB WiredTiger storage engine caches previous versions of any document that might be required by an open transaction. This might represent a lot of documents! However, because read-consistency is implemented only for declared transactions, and because transactions can only persist for one minute, the amount needed to be cached is limited.
MongoDB will issue a “WriteConflictException” if you attempt to modify a document that has changed since the start of a transaction. This transparently implements the familiar pessimistic locking model. Implementing an optimistic model is a little complicated because MongoDB does not currently implement an “intent” lock that can be used to prevent data being modified by another transaction after you have read it.
The initial MongoDB transaction implementation has a few limitations. Currently transactions must issue all commands to the same replica set member. This means in practice that transactions are not available for sharded collections. Transactions are also limited to one minute in duration.
MongoDB has not created anything ground-breaking in the 4.0 release. Indeed, the key concepts in the 4.0 transactional architecture are at least 30 years old. However, by implementing a tried and true transactional system, MongoDB has still taken a giant leap forward in capability.