MongoDB 7.0 introduces an exciting feature called queryable encryption (with Equality Queries), providing a robust method to enhance the security of your data while retaining query capabilities.
Queryable Encryption empowers client applications to secure data during network transport through fully randomized encryption, all while preserving the ability to perform queries. This ensures that sensitive data is automatically encrypted and decrypted by the client, guaranteeing that it is only sent to and received from the server in an encrypted format.
MongoDB supports two types on queryable encryption.
Implicit Queryable Encryption - Available on MongoDB Enterprise & MongoDB ATLAS
Explicit Queryable Encryption - Available on MongoDB Enterprise & MongoDB ATLAS & MongoDB Community
In this blog post, we will delve into the concept of explicit queryable encryption, highlighting its advantages. Additionally, we will offer practical, hands-on examples to illustrate the operational aspects of this security feature.
Explicit Queryable Encryption
Explicit queryable encryption is a security feature that allows you to encrypt sensitive data in your MongoDB database while still allowing the database to perform queries on the encrypted data.
Important Key elements of Explicit Queryable Encryption
Data Encryption Keys (DEKs)
The Data Encryption Key (DEK) is the key used to encrypt fields within MongoDB documents. This DEK is securely stored in a MongoDB collection known as the keyVault collection.
Key Vault Collections
The KeyVault collection functions as the repository for storing and managing encryption keys, including Data Encryption Keys (DEKs) and other keys crucial for cryptographic operations within MongoDB's encryption features
Cryptographic Tokens
Cryptographic tokens are used to represent encrypted data in a format that allows for querying without the need for decryption. They are essentially encrypted versions of specific values or fields within a document. When you insert data into a collection with queryable encryption enabled, the specified fields are encrypted and stored as cryptographic tokens or Cypher Text in the namespace. These tokens are created using encryption algorithms (AES-CBC-256), such as random encryption.
Cryptographic tokens preserve the order of values, allowing for equality and range queries(Upcoming release) on the encrypted data without exposing the plaintext values. Clients can perform queries using these tokens without the need to decrypt the data first.
Cryptographic Tags
Cryptographic tags are metadata associated with encrypted fields. They provide information about the encryption key and algorithm used to encrypt a specific field. These tags help the MongoDB server understand how to handle the encrypted data when performing query operations. Cryptographic tags are stored alongside the data in the database, allowing the server to identify which encryption key to use when querying or decrypting data.
When a query is executed, the MongoDB server uses the cryptographic tags to determine the appropriate decryption key and algorithm for the field in question. Cryptographic tags are crucial for proper query execution while maintaining data security.Customer Master Keys (CMKs). The Customer Master Key (CMK) serves as the key used to encrypt Data Encryption Keys (DEKs) in MongoDB. MongoDB's automated process involves encrypting DEKs with the designated CMK at the time of DEK creation.
Key Management System (KMS)
Queryable Encryption is compatible with the following Key Management System (KMS) providers:
Amazon Web Services KMS
Azure Key Vault
Google Cloud Platform KMS
Any KMIP Compliant Key Management System
Local Key Provider
The structural framework for implementing Explicit Queryable Encryption with a Local Key Provider.
MongoDB 7.0 Queyable Encryption
How MongoDB 7.0 Queryable Encryption works?
Write Operation
MongoDB 7.0 Queryable Encryption - Read operation
When the application submits a query, MongoDB drivers initially analyze the query.
By using AES-CBC-256 encryption algorithm, DEKs (Data Encryption Keys) will encrypt specific fields in a document.
Each DEK is unique and associated with a specific field in the document.
AES - Advanced Encryption Standard
CBC-256 - 256 bit Cipher Block Chaining
AES-CBC-256 encryption algorithm creates a cryptographic token and its respective tags (cryptographic tags). Cryptographic token having encrypted data (using AES-CBC-256 encryption algorithm) and an IV(Initialization Vector) is used to ensure the uniqueness of ciphertexts.
DEKs (Data Encryption Keys) are protected by encrypting them with the CMK (Customer Master Key).
Following this, the driver sends the query to the MongoDB server, representing the encrypted fields as ciphertext.
Finally, Encrypted data (Cryptographic Tokens and its respective Cryptographic tag) are stored in the MongoDB server.
1. Create a directory
mkdir Encryption
2. Switch into the newly created folder
cd Encryption
3. Create index.py file under Encryption folder
touch index.py
4. Update the below information in the index.py file and save it
import os
path = "master-key.txt"
file_bytes = os.urandom(96)
with open(path, "wb") as f:
f.write(file_bytes)
5. Execute the index.py file
python3 index.py (This command will create cmk key and it placed in the master-key.txt file)
Read Operation
MongoDB 7.0 Queryable Encryption - Read operation
When the application submits a query, MongoDB drivers initially analyze the query.
By using AES-CBC-256 encryption algorithm, DEKs (Data Encryption Keys) will encrypt specific documents fields.
DEKs (Data Encryption Keys) are protected by encrypting them with the CMK (Customer Master Key).
Following this, the driver sends the query to the MongoDB server, representing the encrypted fields as ciphertext.
By using the cryptographic tags MongoDB will fetch the respective cryptographic token (Having encrypted data) and send it to the driver.
Finally, the query results are decrypted using the keys held by the driver, and the decrypted data is then returned to the client, presented in plaintext form.
Setting Up a MongoDB Instance with Explicit Queryable Encryption
Walk through the steps of setting up a MongoDB instance with explicit queryable encryption. This includes configuring the key vault, enabling explicit queryable encryption, and establishing the necessary security settings.
To configure explicit queryable encryption, you need to have one of the following drivers installed:
MongoDB Node.js Driver: Install the MongoDB Node.js driver to work with MongoDB in a Node.js environment while configuring explicit queryable encryption.
MongoDB Java Driver: Install the MongoDB Java driver to interact with MongoDB in a Java-based environment while setting up explicit queryable encryption.
MongoDB Python Driver: Install the MongoDB Python driver if you're working with Python and want to configure explicit queryable encryption in your MongoDB application.
These drivers provide the necessary tools and interfaces to interact with MongoDB while enabling queryable encryption for enhanced data security.
I've configured explicit queryable encryption using the python driver. Below is a list of the required packages and the corresponding commands to set up explicit queryable encryption in the python driver:
1. Create one directory like belowmkdir Encryption
2. Switch into the newly created foldercd Encryption
3. Create index.py file under Encryption foldertouch index.py
4. Update the below information in the index.py file and save it
import os
path = "master-key.txt"
file_bytes = os.urandom(96)
with open(path, "wb") as f:
f.write(file_bytes)
5. Execute the index.py file python3 index.py (This command will create cmk key and it placed in the master-key.txt file)
Create a Unique Index on the Key Vault collection
Create __keyvault collection under encryption DB
mongo-encryption [direct: primary] test> use encryption
switched to db encryption
mongo-encryption [direct: primary] encryption> db.getCollection("__keyVault").createIndex({ keyAltNames: 1 },
... {
... unique: true,
... partialFilterExpression: { keyAltNames: { $exists: true } }
... }
... )
mongo-encryption [direct: primary] encryption>
Create Data Encryption Keys and Encrypted Collection
mydbops@ip-x-x-x-x:~/Encryption$ cat create_collection.py
import os
from pymongo.mongo_client import MongoClient
from pymongo.encryption import ClientEncryption
from bson.binary import STANDARD
from bson.codec_options import CodecOptions
from pymongo.encryption_options import AutoEncryptionOpts
provider = "local"
path = "./master-key.txt"
with open(path, "rb") as f:
local_master_key = f.read()
kms_providers = {
"local": {
"key": local_master_key
},
}
"""
This code provides information about the DEK key, encompassing the database and collection (encryption.__keyVault), in addition to the encrypted database and collection (medicalRecords.patients).
"""
connection_string = "mongodb://mongousername:mongopassword@x.x.x.x:27017,x.x.x.x:27018/?authSource=admin&replicaSet=mongo-encryption&connectTimeoutMS=5000";
key_vault_coll = "__keyVault"
key_vault_db = "encryption"
key_vault_namespace = f"{key_vault_db}.{key_vault_coll}"
encrypted_db_name = "medicalRecords"
encrypted_coll_name = "patients"
client = MongoClient(connection_string)
client_encryption = ClientEncryption(
kms_providers,
key_vault_namespace,
client,
CodecOptions(uuid_representation=STANDARD),
)
"""
The code illustrates the particulars of the DEK key.
"""
data_key_id_1 = client_encryption.create_data_key(provider, key_alt_names=["dataKey1"])
data_key_id_2 = client_encryption.create_data_key(provider, key_alt_names=["dataKey2"])
"""
This code illustrates the encrypted field and its associated DEK keys, along with information regarding queryable details.
"""
encrypted_fields_map = {
f"{encrypted_db_name}.{encrypted_coll_name}": {
"fields": [
{
"keyId": data_key_id_1,
"path": "patientId",
"bsonType": "int",
"queries": {"queryType": "equality"},
},
{
"keyId": data_key_id_2,
"path": "medications",
"bsonType": "array",
},
],
},
}
auto_encryption = AutoEncryptionOpts(
kms_providers,
key_vault_namespace,
encrypted_fields_map=encrypted_fields_map,
bypass_query_analysis=True,
# auto encryption option -> crypt_shared_lib_path="",
)
secure_client = MongoClient(connection_string, auto_encryption_opts=auto_encryption)
"""
The code establishes encrypted databases and collections.
"""
encrypted_db = secure_client[encrypted_db_name]
encrypted_db.create_collection(encrypted_coll_name)
print("Created encrypted collection!")
mydbops@ip-x-x-x-x:~/Encryption$
Execute create_collection.py file using python3 command
mydbops@ip-x-x-x-x:~/Encryption$ python3 create_collection.py (This command creates DEK keys in the encryption.__keyVault collection)
mongo-encryption [direct: primary] test> use encryption
switched to db encryption
mongo-encryption [direct: primary] encryption> show collections
__keyVault
mongo-encryption [direct: primary] encryption> db.getCollection("__keyVault").find()
[
{
_id: new UUID("5e6d8206-d39c-41b4-aa86-990de9abdb35"),
keyAltNames: [ 'dataKey1' ],
keyMaterial: Binary.createFromBase64("MyZjqQnCvwQeISbyM3BxRoBeSYCewk4peHJzM7lCIQKrTtZl4KPeef1x/FndALA5wsOcNfUNnFqmRSCznQeg2tA+TO0e/T+XB+sL46OO//OVT2cAD4PchxHCSvijhTmManDVkeXku8m/XjFaxRQ3PUfLzziVOKLacsh8Q16C6OL572837HbDpenc5ou3cilNOufsSIPdSDFmHskpjVZx1g==", 0),
creationDate: ISODate("2023-09-13T07:48:42.091Z"),
updateDate: ISODate("2023-09-13T07:48:42.091Z"),
status: 0,
masterKey: { provider: 'local' }
},
{
_id: new UUID("e3074ff9-d75a-4fe7-9903-ac24e4192769"),
keyAltNames: [ 'dataKey2' ],
keyMaterial: Binary.createFromBase64("qApIwTN4Nmmyi3BdAjQCxPGGs0qSkN+wHsm7c5VCtHvEYxQKwWHtquDYF01izpQsgRdb8ykq0xr2lxSox1OjR9xOinJCRFXmvAL7OPL0mFOHkOijBL4bRasL+KntxdJrn0dsCmtREdcgFNUGntsFWUIEJAWlL1pQPjEiEnc3BAwAOAIOBb5rWLjjcY7CVk9cVwwE0Q5RxNCJsSfRqn6VMw==", 0),
creationDate: ISODate("2023-09-13T07:48:42.105Z"),
updateDate: ISODate("2023-09-13T07:48:42.105Z"),
status: 0,
masterKey: { provider: 'local' }
}
]
mongo-encryption [direct: primary] encryption>
bypass_query_analysis: true parameter helps to disable the auto encryption.
Note: MongoDB community version supports only explicit Encryption
Insert data in the encrypted collection
mydbops@ip-x-x-x-x:~/Encryption$ cat insert_data.py
import os
from pymongo.mongo_client import MongoClient
from pymongo.encryption import ClientEncryption,Algorithm
from bson.binary import STANDARD
from pymongo.encryption_options import AutoEncryptionOpts
from bson.codec_options import CodecOptions
path = "./master-key.txt"
with open(path, "rb") as f:
local_master_key = f.read()
kms_providers = {
"local": {
"key": local_master_key
},
}
key_vault_namespace = "encryption.__keyVault"
key_vault_db_name, key_vault_coll_name = key_vault_namespace.split(".", 1)
connection_string = "mongodb://mongousername:mongopassword@x.x.x.x:27017,x.x.x.x:27018/?authSource=admin&replicaSet=mongo-encryption&connectTimeoutMS=5000"
"""
The code outlines the details of the DEK key.
"""
client = MongoClient(connection_string)
key_vault = client[key_vault_db_name][key_vault_coll_name]
data_key_id_1 = key_vault.find_one({"keyAltNames": "dataKey1"})["_id"]
data_key_id_2 = key_vault.find_one({"keyAltNames": "dataKey2"})["_id"]
opts = AutoEncryptionOpts(
kms_providers,
key_vault.full_name,
bypass_query_analysis=True,
key_vault_client=client
)
"""
This code presents the information for the encrypted database and collection.
"""
encrypted_client = MongoClient(connection_string, auto_encryption_opts=opts)
db = encrypted_client.medicalRecords
coll = db.patients
client_encryption = ClientEncryption(
kms_providers, key_vault_namespace, client, client.codec_options
)
"""
This code portrays the actual documents alongside their corresponding DEK keys.
"""
data_arr = [
{"patientId":client_encryption.encrypt(12345678, Algorithm.INDEXED, data_key_id_1, contention_factor=1),"medications":client_encryption.encrypt(["Atorvastatin", "Levothyroxine"], Algorithm.UNINDEXED, data_key_id_2),"num":0 }, ##DOC1
{"patientId":client_encryption.encrypt(1234567890, Algorithm.INDEXED, data_key_id_1, contention_factor=1),"medications":client_encryption.encrypt(["Atorvastatin", "Levothyroxine"], Algorithm.UNINDEXED, data_key_id_2),"num":1 }, ##DOC2
{"patientId":client_encryption.encrypt(1237890, Algorithm.INDEXED, data_key_id_1, contention_factor=1),"medications":client_encryption.encrypt(["Atorvastatin", "Levothyroxine"], Algorithm.UNINDEXED, data_key_id_2),"num":2 } ##DOC3
]
"""
This code depicts the insertion of documents.##
"""
coll.insert_many(data_arr)
mydbops@ip-x-x-x-x:~/Encryption$
Execute insert_data.py file using python3 command
mydbops@ip-x-x-x-x:~/Encryption$ python3 insert_data.py (Insert document in the encryption enabled collection)
mongo-encryption [direct: primary] test> use medicalRecords
switched to db medicalRecords
mongo-encryption [direct: primary] medicalRecords> show collections
patients // Actual encrypted collection
enxcol_.patients.ecoc //metadata collection1
enxcol_.patients.esc //metadata collection2
mongo-encryption [direct: primary] medicalRecords> db.patients.find()
[
{
_id: ObjectId("65016d4e94de829f2de741ec"),
firstName: 'Jon',
patientId: Binary.createFromBase64("Dl5tggbTnEG0qoaZDemr2zUQpX6FjYfad0LGCUjNPJeDUTQCmTlsIolQNeV1zhxb6qEe6P/ON4+v3atQ5rQsxxsLAtsc9q8QMGLb29WAfcsEWOsjccaSCF", 6),
medications: Binary.createFromBase64("EOMHT/nXWk/nmQOsJOQZJ2kE5DBFUXdgg9J9VWyO12hSSo+kuv/f16GyC1ROYWTzZ8fgFOqHkV/J", 6),
num: 0,
__safeContent__: [
Binary.createFromBase64("7SmfEqJHn9j0oATqlV6Z0hicQ+AenFLwKTtJ/FInGjk=", 0)
]
},
{
_id: ObjectId("65016f6cb46a8582ac44f764"),
patientId: Binary.createFromBase64("Dl5tggbTnEG0qoaZDemr2zUQHlXFn5fZTvoDSIy7bP5Go3PYMb//huABR+djZcPlCouTPjr/lWFXpwjIw8A7vrpac0fQ6R759Efns1T7IWMCx4lGBHRV15zB00BUinF/", 6),
medications: Binary.createFromBase64("EOMHT/nXWk/nmQOsJOQZJ2kE6C0enU0KN4V/LP73VhfEv/JNNyKiI37Yb39H2Reb93tiRkYW8Na5dn3/2yWeNdd7P6jQ7kcWBMK7cRmFzew58gotlE/ccp7Nj1jPb7+aCaU4LqVEA7dq0KYOIbMyrqb/", 6),
num: 1,
__safeContent__: [
Binary.createFromBase64("4OOKGFLyZ3HTjlJG5aISzeIy9Rgq/5VhV6cIyMPAO74=", 0)
]
},
{
_id: ObjectId("65016f6cb46a8582ac44f765"),
patientId: Binary.createFromBase64("Dl5tggbTnEG0qoaZDemr2zUQqMvPG6lk6kV/AoyboMnJ/875++axOZrXvOzSxjGj5O0+/MZ6iR7UN0sN7osn8gq5BYB18Gk7N7r/1Q0r4STxmUGZpam7eHYgI", 6),
medications: Binary.createFromBase64("EOMHT/nXWk/nmQOsJOQZJ2kENdZSPhsXigBmKWsvxOKgIUwp2jtRduHbIWHFTFghoLg7eUBVzpaiUF+7rW", 6),
num: 2,
__safeContent__: [
Binary.createFromBase64("ySp8TYlV68gmD72PGh4x2qm8eSfLwyhgRy2ihygMGHw=", 0)
]
}
]
mongo-encryption [direct: primary] medicalRecords>
Metadata Collections
Queryable Encryption generates two internal metadata collections within the same database as the collection containing a Queryable Encryption encrypted field.
patients // Actual encrypted collection
enxcol_.patients.ecoc //metadata collection1
enxcol_.patients.esc //metadata collection2
enxcol_.patients.ecoc (Encryption Context Cache)
The .ecoc collection stores encryption context information, which includes a cache of cryptographic tokens, cryptographic tags, and related metadata.
Cryptographic Tokens: Encrypted representations of specific field values within documents.
Cryptographic Tags: Metadata associated with encrypted fields, providing information about how to decrypt the data.
Context Information: Details about the encryption context, such as which data encryption key (DEK) is associated with each cryptographic token.
Cache: This collection acts as a cache to optimize query performance by storing frequently used encryption context information. As a result, the size of the metadata collections continues to grow. If the size of the metadata collection surpasses 1 GB, we need to execute the compaction command against the metadata collection.
Temporary Storage: While the data itself is stored in the main collections, .ecoc provides temporary storage for cryptographic metadata used in query processing.
enxcol_.patients.esc (Encryption System Catalog)
The .esc collection serves as the encryption system catalog, storing high-level information about the encryption setup and configuration.
DEK Information: Details about the Data Encryption Keys (DEKs), including their identifiers and associated metadata.
Key Management: Information about key management, including which Customer Master Key (CMK) is used to protect the DEKs.
Algorithms: Information about encryption algorithms and parameters used in the system.
Key Rotation: Details about key rotation policies and schedules.
Configuration: Configuration settings and metadata related to the overall encryption system.
These collections are managed by MongoDB internally to support queryable encryption operations. They help the database server efficiently process queries involving encrypted data by providing the necessary context and metadata to perform decryption and filtering without exposing sensitive information.
These collections are typically not directly accessed or modified by users or applications but are an integral part of the queryable encryption infrastructure in MongoDB.
Retrieve Document with Encrypted Fields
mydbops@ip-172-31-34-30:~/Encryption$ cat query.py
import os
from pymongo.mongo_client import MongoClient
from pymongo.encryption import ClientEncryption,Algorithm,QueryType
from bson.binary import STANDARD
from pymongo.encryption_options import AutoEncryptionOpts
from bson.codec_options import CodecOptions
path = "./master-key.txt"
with open(path, "rb") as f:
local_master_key = f.read()
kms_providers = {
"local": {
"key": local_master_key
},
}
key_vault_namespace = "encryption.__keyVault"
key_vault_db_name, key_vault_coll_name = key_vault_namespace.split(".", 1)
connection_string = "mongodb://mongousername:mongopassword@x.x.x.x:27017,x.x.x.x:27018/?authSource=admin&replicaSet=mongo-encryption&connectTimeoutMS=5000"
client = MongoClient(connection_string)
key_vault = client[key_vault_db_name][key_vault_coll_name]
"""
The code outlines the details of the DEK key.
"""
data_key_id_1 = key_vault.find_one({"keyAltNames": "dataKey1"})["_id"]
data_key_id_2 = key_vault.find_one({"keyAltNames": "dataKey2"})["_id"]
opts = AutoEncryptionOpts(
kms_providers,
key_vault.full_name,
bypass_query_analysis=True,
key_vault_client=client
)
"""
This code presents the information for the encrypted database and collection.
"""
encrypted_client = MongoClient(connection_string, auto_encryption_opts=opts)
db = encrypted_client.medicalRecords
coll = db.patients
client_encryption = ClientEncryption(
kms_providers, key_vault_namespace, client, client.codec_options
)
"""
This code illustrates the find query executed on the encrypted collections.
"""
find_payload = client_encryption.encrypt(
1234567890,
Algorithm.INDEXED,
data_key_id_1,
query_type=QueryType.EQUALITY,
contention_factor=1,
)
doc = coll.find_one({"encryptedIndexed": find_payload})
print(doc)
mydbops@ip-x-x-x-x:~/Encryption$
Execute index_dataretrival.js file using python3 command
root@ip-x-x-x-x:/home/mydbops/Encryption# python3 query.py
## Code retrieve the specific documents based on the find query.
{'_id': ObjectId('65016f6cb46a8582ac44f764'), 'patientId': 1234567890, 'medications': ['Atorvastatin', 'Levothyroxine'], 'num': 1, '__safeContent__': [b'\xe0\xe3\x8a\x18R\xf2gq\xd3\x8eRF\xe5\xa2\x12\xcd\xe22\xf5\x18*\xff\x95aW\xa7\x08\xc8\xc3\xc0;\xbe']}
root@ip--x-x-x-x:/home/mydbops/Encryption#
Update document in the encrypted collection
1. Query the document before updating the patientId value
root@ip-x-x-x-x:~/Encryption# python3 query.py
{'_id': ObjectId('65016f6cb46a8582ac44f765'), 'patientId': 1237890, 'medications': ['Atorvastatin', 'Levothyroxine'], 'num': 2, '__safeContent__': [b"\xc9*|M\x89U\xeb\xc8&\x0f\xbd\x8f\x1a\x1e1\xda\xa9\xbcy'\xcb\xc3(`G-\xa2\x87(\x0c\x18|"]}
root@ip-x-x-x-x:~/Encryption#
2. Update above-mentioned document’s patientId field value1237890 as 1234
root@ip-x-x-x-x:~/Encryption# cat update.py
import os
from pymongo.mongo_client import MongoClient
from pymongo.encryption import ClientEncryption,Algorithm,QueryType
from bson.binary import STANDARD
from pymongo.encryption_options import AutoEncryptionOpts
from bson.codec_options import CodecOptions
path = "./master-key.txt"
with open(path, "rb") as f:
local_master_key = f.read()
kms_providers = {
"local": {
"key": local_master_key
},
}
key_vault_namespace = "encryption.__keyVault"
key_vault_db_name, key_vault_coll_name = key_vault_namespace.split(".", 1)
connection_string = "mongodb://mongousername:mongopassword@x.x.x.x:27017,x.x.x.x:27018/?authSource=admin&replicaSet=mongo-encryption&connectTimeoutMS=5000"
client = MongoClient(connection_string)
key_vault = client[key_vault_db_name][key_vault_coll_name]
##The code outlines the details of the DEK key.
data_key_id_1 = key_vault.find_one({"keyAltNames": "dataKey1"})["_id"]
data_key_id_2 = key_vault.find_one({"keyAltNames": "dataKey2"})["_id"]
opts = AutoEncryptionOpts(
kms_providers,
key_vault.full_name,
bypass_query_analysis=True,
key_vault_client=client
)
"""
This code presents the information for the encrypted database and collection.
"""
encrypted_client = MongoClient(connection_string, auto_encryption_opts=opts)
db = encrypted_client.medicalRecords
coll = db.patients
client_encryption = ClientEncryption(
kms_providers, key_vault_namespace, client, client.codec_options
)
## This code describes the match filter used in the update query.
find_payload = client_encryption.encrypt(
1237890,
Algorithm.INDEXED,
data_key_id_1,
query_type=QueryType.EQUALITY,
contention_factor=1,
)
"""
This code updates the original document using the specified match filter in the update query.
"""
doc = coll.update_one({"encryptedIndexed": find_payload},{"$set":{"patientId":client_encryption.encrypt(1234, Algorithm.INDEXED, data_key_id_1, contention_factor=1)}})
print("updated")
root@ip-x-x-x-x:~/Encryption#
3. Execute update.py file using python3 command
root@ip-x-x-x-x:~/Encryption# python update.py
updated
root@ip-x-x-x-x:~/Encryption#
4. View the updated data
root@ip-x-x-x-x:~/Encryption# python query.py
{'_id': ObjectId('65016f6cb46a8582ac44f765'), 'patientId': 1234, 'medications':
['Atorvastatin', 'Levothyroxine'], 'num': 2, '__safeContent__': [b'\xbe\x13ah\xbc\xd6\x96\xd0\xc5`[$p{\xf4\xa6\x8b\xe7\xab\xee\x80\xf2~\x8a\xd9)\xf4\xcbi\xc2\xb2,']}
root@ip-x-x-x-x:~/Encryption#
Delete document in the encrypted collection
1. Query the document before deleting the patientId field value 1234 document
root@ip-x-x-x-x:~/Encryption# python3 query.py
{'_id': ObjectId('65016f6cb46a8582ac44f765'), 'patientId': 1234, 'medications': ['Atorvastatin', 'Levothyroxine'], 'num': 2, '__safeContent__': [b'\xbe\x13ah\xbc\xd6\x96\xd0\xc5`[$p{\xf4\xa6\x8b\xe7\xab\xee\x80\xf2~\x8a\xd9)\xf4\xcbi\xc2\xb2,']}
root@ip-x-x-x-x:~/Encryption#
2. Delete the above-mentioned document
root@ip-172-31-43-25:~/Encryption# cat delete.py
import os
from pymongo.mongo_client import MongoClient
from pymongo.encryption import ClientEncryption,Algorithm,QueryType
from bson.binary import STANDARD
from pymongo.encryption_options import AutoEncryptionOpts
from bson.codec_options import CodecOptions
path = "./master-key.txt"
with open(path, "rb") as f:
local_master_key = f.read()
kms_providers = {
"local": {
"key": local_master_key
},
}
key_vault_namespace = "encryption.__keyVault"
key_vault_db_name, key_vault_coll_name =
key_vault_namespace.split(".", 1)
connection_string = "mongodb://mongousername:mongopassword@x.x.x.x:27017,x.x.x.x:27018/?authSource=admin&replicaSet=mongo-encryption&connectTimeoutMS=5000"
client = MongoClient(connection_string)
key_vault = client[key_vault_db_name][key_vault_coll_name]
##The code outlines the details of the DEK key.
data_key_id_1 = key_vault.find_one({"keyAltNames": "dataKey1"})["_id"]
data_key_id_2 = key_vault.find_one({"keyAltNames": "dataKey2"})["_id"]
opts = AutoEncryptionOpts(
kms_providers,
key_vault.full_name,
bypass_query_analysis=True,
key_vault_client=client
)
"""
This code presents the information for the encrypted database and collection.
"""
encrypted_client = MongoClient(connection_string, auto_encryption_opts=opts)
db = encrypted_client.medicalRecords
coll = db.patients
client_encryption = ClientEncryption(
kms_providers, key_vault_namespace, client, client.codec_options
)
## This code describes the match filter used in the delete query.
find_payload = client_encryption.encrypt(
1234,
Algorithm.INDEXED,
data_key_id_1,
query_type=QueryType.EQUALITY,
contention_factor=1,
)
"""
This code deletes the original document using the specified match filter in the delete query.
"""
doc = coll.delete_one({"encryptedIndexed": find_payload})
print("deleted.")
root@ip-x-x-x-x:~/Encryption#
3. Execute delete.py file using python3 command
root@ip-x-x-x-x:~/Encryption# python delete.py
deleted.
root@ip-x-x-x-x:~/Encryption#
4. Check the document is deleted or not
root@ip-172-31-43-25:~/Encryption# cat query.py
import os
from pymongo.mongo_client import MongoClient
from pymongo.encryption import ClientEncryption,Algorithm,QueryType
from bson.binary import STANDARD
from pymongo.encryption_options import AutoEncryptionOpts
from bson.codec_options import CodecOptions
path = "./master-key.txt"
with open(path, "rb") as f:
local_master_key = f.read()
kms_providers = {
"local": {
"key": local_master_key
},
}
key_vault_namespace = "encryption.__keyVault"
key_vault_db_name, key_vault_coll_name = key_vault_namespace.split(".", 1)
connection_string =
"mongodb://mongousername:mongopassword@x.x.x.x:27017,x.x.x.x:27018/?authSource=admin&replicaSet=mongo-encryption&connectTimeoutMS=5000"
client = MongoClient(connection_string)
key_vault = client[key_vault_db_name][key_vault_coll_name]
##The code outlines the details of the DEK key.
data_key_id_1 = key_vault.find_one({"keyAltNames": "dataKey1"})["_id"]
data_key_id_2 = key_vault.find_one({"keyAltNames": "dataKey2"})["_id"]
opts = AutoEncryptionOpts(
kms_providers,
key_vault.full_name,
bypass_query_analysis=True,
key_vault_client=client
)
"""
This code presents the information for the encrypted database and collection.
"""
encrypted_client = MongoClient(connection_string, auto_encryption_opts=opts)
db = encrypted_client.medicalRecords
coll = db.patients
client_encryption = ClientEncryption(
kms_providers, key_vault_namespace, client, client.codec_options
)
"""
This code illustrates the find query executed on the encrypted collections.
"""
find_payload = client_encryption.encrypt(
1234,
Algorithm.INDEXED,
data_key_id_1,
query_type=QueryType.EQUALITY,
contention_factor=1,
)
doc = coll.find_one({"encryptedIndexed": find_payload})
print(doc)
root@ip-172-31-43-25:~/Encryption#
5. Execute query.py file using python3 command
root@ip-x-x-x-x:~/Encryption# python query.py
None
root@ip-x-x-x-x:~/Encryption#
Limitations
Important key limitations of Queryable Encryption:
Queryable Encryption is incompatible with MongoDB Atlas Search.
Manual compaction is required when the meta data collection exceed 1 GB of size
Queryable Encryption is not applicable for MongoDB standalone servers
Shard key must not be a encrypted fields
Queryable Encryption does not support batch operation
We can't rename the collection with encrypted fields
Explicit queryable encryption in MongoDB 7.0 is a groundbreaking feature that allows you to secure your data while maintaining the power of query capabilities. By following best practices and understanding the intricacies of key management, encryption algorithms, and key rotation, you can confidently implement explicit queryable encryption to safeguard your sensitive information in MongoDB.
In the upcoming sections, we'll explore each aspect of explicit queryable encryption in detail, providing step-by-step instructions and real-world examples to guide you through the implementation process.
Stay tuned for an in-depth exploration of this important security feature in MongoDB 7.0.
.png)
.png)
.svg)
