MongoDB 7.0, the latest milestone in the world of data management and query optimization, is here to redefine your data handling experience. This groundbreaking update comes packed with an array of exciting features that promise to redefine your experience with this popular NoSQL database system. MongoDB 7.0 isn't just a database update; it's a transformation in how you interact with your data. In this blog post, we'll embark on a journey through MongoDB's revamped Query Features, which are sure to leave both developers and data enthusiasts captivated. Overview of the significant update

• Compound Wildcard Indexes: MongoDB now supports the creation of wildcard indexes on individual fields or sets of fields. A compound index can consist of multiple index terms, with one being a wildcard term and others being additional index terms.

• Bitwise Operators: While bitwise operators have been available for find and update queries, MongoDB 7.0 extends their usage to aggregations. This enhancement allows you to perform common bitwise operations directly within database aggregations, resulting in improved query performance, especially for application-focused and operational analytics.

• Percentile Operators: The new percentile operators return an array of scalar values corresponding to specified percentile values. You can employ the $percentile operator as an accumulator within the $group stage or as an aggregation expression.

• Role-based Data Access with $$USER_ROLES: MongoDB introduces the use of variables, capable of holding various BSON data types. To access the value of a variable, prepend the variable name with double dollar signs ($$), such as $$<variable>. If the variable references an object, you can access specific fields within the object using dot notation, for instance, $$<variable>.<field>.

Compound Wildcard Indexes

Benefits

MongoDB introduced wildcard indexing starting from version 4.2, but it had a limitation where you couldn't create compound wildcard indexes. However, with the release of MongoDB version 7.0, you can now create compound wildcard indexes. This enhancement is particularly valuable for optimizing collections containing nested or sub-document structures. Below, we've provided an example along with a use case illustrating the benefits of wildcard indexing.

Example

Let's explore how Wildcard Indexes in MongoDB 7.0 can bring efficiency and flexibility to your data management, using a practical example. Imagine you're overseeing a customer database for an eCommerce platform. Each customer has a variety of attributes stored under different categories. Your goal is to enable users to search for customers based on specific regions and any attribute, regardless of its category.

Here's an example of a customer document structure:

 
{

  "_id": ObjectId("abcdef123456"),

  "Region": "North America",

  "Attributes": {

    "Personal": {

      "Name": "Alice",

      "Age": 30

    },

    "Purchase": {

      "TotalSpent": 1500,

      "LastPurchase": "2023-07-15"

    }

  }

}
	

To achieve dynamic searching across all attribute categories while prioritizing the region, you can create a Wildcard Index as follows:

 
db.customer_attributes.createIndex({"Region": 1, "Attributes.$**": 1});
	

In this example, Region: 1 ensures efficient retrieval based on the region. Meanwhile, Attributes.$**: 1 introduces the wildcard, enabling searches across all attributes and categories within the Attributes field.

The index can support the following queries:

 
db.customer_attributes.find({ "Region": "North America", "Attributes.Personal.Name": "Alice" })

db.customer_attributes.find({ "Region": "North America", "Attributes.Purchase.TotalSpent": "1500" })
	

This wildcard index allows for flexible querying of sub-document structures, making it easier to search for and retrieve data from complex nested documents.

Wildcard Index Restrictions

Wildcard indexes in MongoDB offer great flexibility but come with certain restrictions and limitations. Here are some key limitations for wildcard indexes:

Compound Wildcard Index Restrictions:

• Single Wildcard Term: A compound wildcard index can only have one wildcard term. You cannot have multiple wildcard terms in a single compound index. For example, you cannot specify an index like this:

 
db.customer_attributes.createIndex({ Region: 1, "Attributes.$**": 1, "Object.$**": 1 });
	

• In this example, there are two wildcard terms (Attributes.$ and Object.$), which are not allowed in a compound wildcard index.
• Single Key Terms: The non-wildcard terms in a compound wildcard index must be single key terms. Multikey index terms are not permitted. This means that you cannot create compound wildcard indexes with arrays.

• Limited Use of wildcardProjection: The wildcardProjection option is only valid when the wildcard field is $**. You cannot use wildcardProjection when you specify a field path for the wildcard index term. In other words, you can project specific fields when the wildcard is applied at the top level using $**, but you cannot do so when applying it to a specific field path.

Incompatible Index Properties:

• TTL (Time to Live): You cannot specify the TTL property for a wildcard index. TTL indexes are used for automatically removing documents from a collection after a certain amount of time, and this feature is not compatible with wildcard indexing.
• Unique: Wildcard indexes cannot be marked as unique. Unique indexes ensure that no two documents in a collection share the same value for the indexed field(s). Wildcard indexing does not support this property.

Incompatible Index Types:

• 2d (Geospatial): Wildcard syntax ($.**) cannot be used to create 2d geospatial indexes. These indexes are designed for geospatial data, and wildcard indexing is not suitable for this purpose.
• 2dsphere (Geospatial): Similarly, you cannot create 2dsphere geospatial indexes using wildcard syntax. 2dsphere indexes are used for more complex geospatial data types, and they require specific geospatial queries.
• Hashed: Hashed indexes, which are used for hashed values, are not compatible with wildcard syntax either. Wildcard indexing focuses on wildcard matching within documents, and hashing is not applicable in this context.

We need to consider these limitations when making decisions about wildcard compound indexing strategies. Wildcard indexing is particularly useful for flexible querying of nested or dynamic fields but has its own set of restrictions.To gain more insights, read our recent blog post on Wildcard Compound Indexes.Unlocking the Potential of Bitwise Operators in Your PipelineEnhancing Queries with Bitwise Operators In versions of MongoDB prior to 7.0, bitwise operators were limited to use in find and update queries. However, with the introduction of MongoDB 7.0, you can now perform bitwise operations directly within aggregation queries. This enhancement greatly improves query performance, particularly for application-driven and operational analytics, as it allows for more flexible and efficient data manipulation and analysis within the database itself.Real-world Application: Filtering Based on Binary Data FlagsImagine a common scenario in which you need to filter documents based on specific binary data flags. Consider the case of a permission system where user flags are stored as binary values. With MongoDB 7.0, this task becomes remarkably efficient:

 
{

  $match: {

    $expr: {

      $eq: [{$bitAnd: ["$userFlags", 4]}, 4]

    }

  }

}
	

This filter uses the $bitAnd operator to check whether the fourth bit of the userFlags field is set. This type of operation is essential in various applications, and MongoDB's integration of Bitwise Operators within aggregations simplifies and accelerates the process.

Bitwise Operators at Your Fingertips

MongoDB introduces several powerful Bitwise Operators:

$bitAnd (aggregation)

• $bitAnd: Performs a bitwise AND operation on an array of int or long values.

 
db.switches.find()

[

    { _id: 0, a: NumberInt(0), b: NumberInt(127) },

    { _id: 1, a: NumberInt(2), b: NumberInt(3) },

    { _id: 2, a: NumberInt(3), b: NumberInt(5) }

] 

Bitwise AND with Two Integers : 

The following aggregation uses the $bitAnd operator in the $project stage:

db.switches.aggregate( [{$project: {result: {$bitAnd: [ "$a", "$b" ] } } } ])

The operation returns the following results:

  [

    { _id: 0, result: 0 }

    { _id: 1, result: 2 }

    { _id: 2, result: 1 }

  ]
	

$bitNot (aggregation)

• $bitNot: Executes a bitwise NOT operation on a single argument or an array containing a single int or long value.

 
db.switches.find()[

    { _id: 0, a: NumberInt(0), b: NumberInt(127) },

    { _id: 1, a: NumberInt(2), b: NumberInt(3) },

    { _id: 2, a: NumberInt(3), b: NumberInt(5) }

] 

The following aggregation uses the $bitNot operator in the $project stage:

db.switches.aggregate( [{$project: {result: {$bitNot: "$a"} } } ])

The operation returns the following results:

  [

    { _id: 0, result: -1 },

    { _id: 1, result: -3 },

    { _id: 2, result: -4 }

  ]
	

$bitOr (aggregation)  

• $bitOr: Computes a bitwise OR operation on an array of int or long values.

 
db.switches.find()[

    { _id: 0, a: NumberInt(0), b: NumberInt(127) },

    { _id: 1, a: NumberInt(2), b: NumberInt(3) },

    { _id: 2, a: NumberInt(3), b: NumberInt(5) }

] 

Bitwise OR with Two Integers

The following aggregation uses the $bitOr operator in the $project stage:

db.switches.aggregate( [{$project: {result: {$bitOr: [ "$a", "$b" ] } } } ])

The operation returns the following results:

  [

    { _id: 0, result: 127 },

    { _id: 1, result: 3 },

    { _id: 2, result: 7 }

  ]

	

$bitXor (aggregation)

• $bitXor: Provides the result of a bitwise XOR (exclusive OR) operation on an array of int and long values.

 
db.switches.find()

[

    { _id: 0, a: NumberInt(0), b: NumberInt(127) },

    { _id: 1, a: NumberInt(2), b: NumberInt(3) },

    { _id: 2, a: NumberInt(3), b: NumberInt(5) }

] 

Bitwise Xor with Two Integers : 

The following aggregation uses the $bitXor operator in the $project stage:

db.switches.aggregate( [{$project: {result: {$bitXor: ["$a", "$b"] } } } ])

The operation returns the following results:

  [

    { _id: 0, result: 127 },

    { _id: 1, result: 1 },

    { _id: 2, result: 6 }

  ]
	

Exploring $bitwise in MongoDB: Simple Use Cases and Examples

The $bit operator in MongoDB is designed to work with integer fields, specifically either 32-bit integers or 64-bit integers. It's particularly useful when dealing with binary flags or permissions stored as integers. Here's a simple and clear use case with an example:

Use Case: User Permissions

Suppose you have a system where user permissions are represented as an integer, and each bit in that integer represents a specific permission.

• Bit 1 (the rightmost bit) represents Read permission.
• Bit 2 represents Write permission.
• Bit 3 represents Delete permission.

Example:

Let's say you have a user with permissions value 6, which corresponds to Write (bit 2) and Delete (bit 3) permissions. You want to check if the user has the Write permission.

Here's how you can do it using the $bitwise operator:

 
db.users.aggregate([{

    $match: {

      _id: 123,         // Replace with the user's ID

      permissions: {

        $bitwise: {

          and: [2]    // Checking for the "Write" permission (bit 2)

        }

      }

    }

  }

]

)
	

Explanation:

• The $match stage filters for the user with _id 123.
• The permissions field contains the user's permissions value (6).
• $bitwise performs a bitwise AND operation with 2, which represents the Write permission (bit 2).
• If the result is not 0, it means the user has the Write permission.
• So, in this example, if the result of the aggregation is not an empty document, it indicates that the user with _id 123 has the Write permission.

In summary, the $bitwise operator is useful for checking and manipulating permissions or flags represented as binary values in MongoDB, making it easier to manage access control and permissions for users or entities in your application.

Simplifying Data Analysis with Percentile Operators

In MongoDB 7.0, the $percentile operator is a newly introduced feature. It allows you to obtain an array of scalar values representing specific percentile values from your data. You can utilize $percentile as an aggregation operator within the $group stage or as an expression in your aggregation pipeline. This feature enables you to easily calculate and retrieve percentiles for your data during MongoDB queries.

Behavior and use cases of $percentile

The $percentile operator in MongoDB 7.0 can be employed in the following stages of your aggregation pipeline:

• $group Stages as an Accumulator: You can use $percentile within the $group stage as an accumulator. This allows you to compute and aggregate percentile values for groups of documents based on certain criteria.
• $setWindowFields Stages as an Accumulator: The $percentile operator can also be utilized within $setWindowFields stages as an accumulator. This stage enables you to define window functions, and $percentile can help you calculate percentiles over defined windows.
• $project Stages as an Aggregation Expression: In the $project stage, you can use $percentile as an aggregation expression. This allows you to include percentile calculations in the projection of your output documents, making it convenient to obtain percentile values for specific fields or data.

Considerations for Percentile Calculations

• In $group stages, it employs an approximate calculation method for percentiles.

• In $project stages, it may use the discrete calculation method even if the approximate method is specified.

• In $setWindowFields stages, the calculation method used depends on the workload.

• It's important to note that the computed percentile values may vary, even with the same dataset, due to the use of approximate algorithms. Duplicate samples can create ambiguity, especially if there are many duplicates. In such cases, the percentile values may not accurately represent the sample distribution. Additionally, for p = 0.0, $percentile returns the minimum value, and for p = 1.0, it returns the maximum value.

Command Fields, $percentile takes the following fields:

‍

 
Example : 

db.testScores.find()

[

   { studentId: "2345", test01: 62, test02: 81, test03: 80 },

   { studentId: "2356", test01: 60, test02: 83, test03: 79 },

   { studentId: "2358", test01: 67, test02: 82, test03: 78 },

   { studentId: "2367", test01: 64, test02: 72, test03: 77 },

   { studentId: "2369", test01: 60, test02: 53, test03: 72 }

]

Calculate a Single Value as an Accumulator : 

Create an accumulator that calculates a single percentile value:

db.testScores.aggregate( [

   {

      $group: {

         _id: null,

         test01_percentiles: {

            $percentile: {

               input: "$test01",

               p: [ 0.85 ],

               method: 'approximate'

            }

         },

      }

   }

] )

Output:

{ id: null, test01percentiles: [ 64 ] }
	

• The $group stage groups all the documents in the collection because the _id field is set to null, so it considers all documents as a single group.
• The percentile calculation is performed on the test01 field of all documents.
• Since the p array in this example contains only one value, the $percentile operator calculates a single percentile term. In this case, it calculates the 95th percentile value, which is 64 based on the provided data.

Benefits at a GlanceMongoDB 7.0's Approximate Percentile Operator brings a host of advantages:

• Enhanced Efficiency: By performing percentile calculations directly on the database server, you eliminate the need for cumbersome post-processing steps, saving time and resources.

• Simplified Queries: Complex percentile calculations are condensed into a concise operator within your aggregation pipelines, making your queries more readable and manageable.

• Instant Insights: With efficient calculations at your fingertips, you can swiftly uncover important data trends, allowing for quicker decision-making.

Enhancing Data Security: Implementing Role-Based Access in MongoDB 7.0MongoDB 7.0 introduces a groundbreaking feature – the $$USER_ROLES variable. This dynamic variable offers a seamless way to provide role-based access to data right within your queries, streamlining security and data privacy practices. Let's dive into a detailed example to see how this feature works.

The Scenario: A Sales ApplicationImagine you're developing a sales application where different users have distinct roles. Everyone can access basic account data, but certain sensitive details, such as contract information, should only be visible to specific managers. MongoDB's $$USER_ROLES variable makes it remarkably simple to implement this security feature.The Implementation: Role-Based Data FilteringHere's a step-by-step breakdown of how you can use $$USER_ROLES to achieve role-based data access in MongoDB 7.0:Define Your User RolesFirst, define the various user roles within your application. In our example, we have two roles: regular users and managers.Create a PipelineCreate a pipeline that sets up your role-based data filtering logic. In this pipeline, we use the $cond operator to conditionally include or exclude data based on the user's role.

 
pipeline = [

  {

     $set: {

        "contractDetails": {

           $cond: {

              if: { $in: [ 

                      "manager",

                      '$$USER_ROLES.role'

                      ] },

              then: "$contractDetails",

              else: "$$REMOVE"

           }

        }

     }

  }

]
	

In this example, if the user's role matches manager, the pipeline retains the contractDetails field; otherwise, it removes it from the results.

Create a View

Now, create a MongoDB view that uses the pipeline to filter the data. This view ensures that users can only see the data that aligns with their role privileges.

 
db.createView('accounts_view', 'accounts', pipeline ) 
	

The Outcome: Role-Based Data Access

With this setup, when a user accesses the accounts_view, they will only see contractDetails if they have the manager role. Regular users will see all other account data but won't have access to sensitive contract information.

Advantages of Using $$USER_ROLES

• Enhanced Security: Data access is precisely controlled, bolstering overall system security.
• Simplified Architecture: You no longer need to maintain separate views for each role, reducing complexity.
• Improved Data Privacy: Sensitive data remains hidden from unauthorized users, complying with modern data privacy requirements.

MongoDB 7.0's $$USER_ROLES variable empowers developers to implement role-based data access with ease, improving security and enhancing user experiences. By embracing this feature, you're not just protecting your data; you're also streamlining your application's architecture.

A Revolutionary Advancement in Developer Experience

MongoDB 7.0's query features are more than tools; they're a transformation in your developer journey. These features empower you to sculpt your application's data interaction efficiently, right from the start.

From Compound Wildcard Indexes to new aggregation pipeline operators, MongoDB 7.0 streamlines queries. With less focus on optimization, you can invest time in what truly matters: crafting extraordinary applications. It's a leap towards cleaner, faster, and more intuitive queries. Embrace MongoDB 7.0 and elevate your development experience.

Reference

Stay connected for more valuable MongoDB insights.

Also read: A Deep Dive into MongoDB's Queryable Encryption