On 25th of September 2019, the LiveDataset team joined the UK and European users of MongoDB at MongodDB.local London.
Keynote speaker Eliot Horowitz, co-founder and CTO announced a whole host of new features in MongoDB 4.2 – Here are the ones we found most interesting:
- Client-side Encryption
- Full text search capabilities (only in Atlas for now)
- Full support for transaction
Security -> Client-Side Field Level Encryption
MongoDB always supported data transport encryption using TLS/SSL protocol as well as data at rest encryption using AES256.
Client-Side Field Level Encryption is a new feature, an upgrade to the MongoDB drivers (version 4.2), that allows applications to encrypt some fields in documents prior to transmitting the data to MongoDB backend server.
Only applications with the correct keys can decrypt and read the data.
This is a very powerful addition to MongoDB, dealing with scenarios whereas there are some fields that store highly confidential or personal information that does not even System or DB administrator (that have access to the MongoDB Server) can see.
Using this new feature is straightforward for developers, no application changes are required, simply define the fields to encrypt in the database schema. Obviously, the limitation on these encrypted fields is that they can only be searched with exact matches, but that’s a trade-off that is acceptable in these cases.
Currently MongoDB supports the following Key Management Service (KMS):
- AWS KMS
- Locally Managed Keyfile
Support for major KMS provider such as HashiCorp and Vormetic underway.
Atlas -> Full Text Search
Typically, a modern application provides “google search” style capabilities for any text search alongside the more traditional filters for exact matches.
The go to tools in the industry that provides such a feature are Apache Solr and Elasticsearch.
Since, under the hood, both these tools use Apache Lucene, a well established 20 years old project, MongoDB decided to cut the middleman and directly integrate with it.
For application developers, already using MongoDB, having such an option is a great addition without having to integrate with another technology.
Currently is only available for Atlas, looking forward to seeing it available in the other MongoDB editions.
Being based on Lucene and its inverted index data structure, it requires an initial full document scan as well as single document processing every time a document is changed or stored. Architecturally this is achieved by deploying additional mongot nodes that do not interfere with the operational side provided by the mongod nodes.
Core -> ACID Transactions in MongoDB
With the addition of multi-document ACID transactions, even across sharded clusters, MongoDB is officially the only database that fully combines the power of document json model and a distributed architecture with ACID guarantees, once only available in traditional RDBMS.
Summary
Overall, we found the conference very informative and left with the feeling that we are using the right tool, reassured by the fact that the team behind MongoDB is always on top of the latest and important trends in the industry.
Many sessions were also very technical and from the point of view of the customers having to deal with common problems. We particularly enjoyed the talk on how to query and index data and the one on methodologies to apply to data modelling and its effect on memory and disk utilization (two critical factors for application performance).
LiveDataset already leverages core MongoDB features such as flexible json schemas for easy dataset creation and manipulation and data clustering for resiliency, the team will look into integrating new features provided by MongoDB in upcoming releases.
References