Welcome to the MongoDB 7.0 revolution! In this blog, we'll explore one of its groundbreaking features: compound wildcard indexes. MongoDB's flexibility meets unstructured data, making wildcard indexes invaluable. With version 7.0, compound wildcard indexes offer even more possibilities. Join us as we dive into the details and discover how this feature can optimize your database queries and adapt to ever-changing data structures.

Compound Wildcard Index in MongoDB 7.0

The wildcard index is a powerful tool that allows us to create an index encompassing all fields within a collection's documents. MongoDB's unique support for unstructured data, in contrast to the predefined schemas of SQL databases, renders wildcard indexes particularly beneficial. And with the introduction of MongoDB version 7.0, a new frontier opens: the world of compound wildcard indexes. These indexes blend a wildcard term with one or more additional index terms, enhancing the database's indexing capabilities.

Suitable Situation

Consider this scenario: your application frequently queries an embedded document field that boasts inconsistent subfields. In such a case, the creation of a wildcard index becomes a prudent choice. This wildcard index will extend its support to queries on all subfields within the embedded document.

Example

 
{
  "productId": 1,
  "productDetails": {
    "name": "smartwatch",
    "color": "black",
    "available": 6,
    "instock": true,
 },
   "createdDate": "2023-09-05T14:30:00Z"
   "UpdatedDate": "2023-09-05T14:30:00Z"
	

• In a scenario where your application frequently queries both productId and the various attributes within productDetails, creating a compound wildcard index for the productDetails field, in addition to a regular index for 'productId,' can significantly enhance query optimization by leveraging these indexes.
• Moreover, when the specific fields requiring indexing are uncertain or liable to change due to evolving business use cases, utilizing compound wildcard indexes may be a suitable approach.

Index creation command

‍To bring this concept to life, let's explore how to create a compound wildcard index on the 'productId' and 'productDetails' subdocument within the 'product' collection, based on the schema we just discussed. The command for this index creation looks like this:

 
db.product.createIndex(
   { productId: 1, "productDetails.$**": 1 },
   {
      name: "productId_productDetails_index"
   }
)
	

• Here, the wildcard term, productDetails.$**, effectively encompasses all sub-fields within the 'productDetails' field.
• It's important to note that the other index term, 'productId,' is not a wildcard specification; it's a standard field specification, providing precise indexing for that specific field.

Filter Fields with a wildcardProjection in MongoDB 7.0

‍In the MongoDB 7.0 landscape, wildcardProjection emerges as a powerful tool when crafting an index on an individual sub-field. It grants us the ability to precisely include or exclude specific subfields, offering fine-grained control over the indexing process.

‍Include Specific Fields from a Compound Wildcard Index

‍Let's start with an example. Imagine that the wildcard index term, $**, initially specifies every field within the collection. However, using wildcardProjection, we can narrow down the index to cover only the fields we specify. Take, for instance, a scenario involving a product collection, where we wish to index only the productDetails.name and productDetails.color fields. The indexing command would look like this:

 
db.product.createIndex(
  { productId: 1, "$**": 1 },
  { "wildcardProjection": {
        "productDetails.name": 1,
        "productDetails.color": 1
     },
     name: "productId_productDetails_index"
  }
)
	

It's worth noting that the wildcardProjection option can be utilized exclusively when the wildcard term itself is $**.

Exclude Specific Fields from a Compound Wildcard Index

On the flip side, it's essential to understand that MongoDB 7.0 does not support the exclusion of a specific field from a compound wildcard index. However, you do have the flexibility to choose whether to exclude any field or subfield within a wildcard index. For a more comprehensive understanding of this exclusion process, refer to the section on Exclude Specific Fields from a Wildcard Index.

Compound Wildcard Considerations in MongoDB 7.0

In our exploration of compound wildcard indexes in MongoDB 7.0, there are a few essential considerations to keep in mind:

• Sparse Indexes: Compound wildcard indexes are inherently sparse, meaning they selectively index documents based on specific criteria.
• Inclusion Criteria: Documents are included in these indexes if they lack the wildcard field but possess at least one of the compound fields. This selective inclusion ensures efficiency where it matters.
• Sorting Flexibility: MongoDB provides the flexibility to sort index fields, including wildcard fields, in ascending (1) or descending (-1) order, empowering you to optimize queries to your specific needs.

Compound Wildcard Index Restrictions in MongoDB 7.0

When working with compound wildcard indexes in MongoDB 7.0, it's essential to be aware of certain restrictions that govern their usage:

• A compound wildcard index can only have a single wildcard term. For example, if we consider a document in the product collection, as shown below, we cannot create a compound index with two or more wildcard terms.

 
{
  "productId": 1,
  "productDetails": {
    "name": "smartwatch",
    "color": "black",
    "available": 6,
    "instock": true,
 },
   "createdDate": "2023-09-05T14:30:00Z"
   "UpdatedDate": "2023-09-05T14:30:00Z"

  "AdditionalDetails": {
    "type": "digital",
    "strip": "leather",
    "replacement": true
    }
   }    
	

Index Example:

 
{ productId: 1, "productDetails.$**": 1, "AdditionalDetails.$**": 1 }
	

• Here, the wildcard term, productDetails.$**, effectively encompasses all sub-fields within the 'productDetails' field.
• It's important to note that the other index term, 'productId,' is not a wildcard specification; it's a standard field specification, providing precise indexing for that specific field.

Filter Fields with a wildcardProjection in MongoDB 7.0

In the MongoDB 7.0 landscape, wildcardProjection emerges as a powerful tool when crafting an index on an individual sub-field. It grants us the ability to precisely include or exclude specific subfields, offering fine-grained control over the indexing process.

Include Specific Fields from a Compound Wildcard Index

Let's start with an example. Imagine that the wildcard index term, $**, initially specifies every field within the collection. However, using wildcardProjection, we can narrow down the index to cover only the fields we specify. Take, for instance, a scenario involving a product collection, where we wish to index only the productDetails.name and productDetails.color fields. The indexing command would look like this:

 
db.product.createIndex(
  { productId: 1, "$**": 1 },
  { "wildcardProjection": {
        "productDetails.name": 1,
        "productDetails.color": 1
     },
     name: "productId_productDetails_index"
  }
)
	

It's worth noting that the wildcardProjection option can be utilized exclusively when the wildcard term itself is $**.

Exclude Specific Fields from a Compound Wildcard Index

On the flip side, it's essential to understand that MongoDB 7.0 does not support the exclusion of a specific field from a compound wildcard index. However, you do have the flexibility to choose whether to exclude any field or subfield within a wildcard index. For a more comprehensive understanding of this exclusion process, refer to the section on Exclude Specific Fields from a Wildcard Index.

Compound Wildcard Considerations in MongoDB 7.0

In our exploration of compound wildcard indexes in MongoDB 7.0, there are a few essential considerations to keep in mind:

• Sparse Indexes: Compound wildcard indexes are inherently sparse, meaning they selectively index documents based on specific criteria.
• Inclusion Criteria: Documents are included in these indexes if they lack the wildcard field but possess at least one of the compound fields. This selective inclusion ensures efficiency where it matters.
• Sorting Flexibility: MongoDB provides the flexibility to sort index fields, including wildcard fields, in ascending (1) or descending (-1) order, empowering you to optimize queries to your specific needs.

Compound Wildcard Index Restrictions in MongoDB 7.0

When working with compound wildcard indexes in MongoDB 7.0, it's essential to be aware of certain restrictions that govern their usage:

• A compound wildcard index can only have a single wildcard term. For example, if we consider a document in the product collection, as shown below, we cannot create a compound index with two or more wildcard terms.

 
{
  "productId": 1,
  "productDetails": {
    "name": "smartwatch",
    "color": "black",
    "available": 6,
    "instock": true,
 },
   "createdDate": "2023-09-05T14:30:00Z"
   "UpdatedDate": "2023-09-05T14:30:00Z"

  "AdditionalDetails": {
    "type": "digital",
    "strip": "leather",
    "replacement": true
    }
   }       
	

Index Example:

 
{ productId: 1, "productDetails.$**": 1, "AdditionalDetails.$**": 1 }
	

• Non-wildcard terms within a compound wildcard index must be single key terms. Multikey index terms are not supported.
• Compound wildcard indexes do not allow the specification of TTL (Time To Live) and Unique index properties. These properties are not applicable in this context.
• Certain specialized index types, such as 2d (Geospatial), 2dsphere (Geospatial), and Hashed, cannot be created for compound wildcard indexes.
• By default, the id field is omitted from compound wildcard indexes. If you need to include it, you must explicitly do so using wildcardProjection.

Behavior of Compound wildcard Index in Array and Embedded documentIn MongoDB 7.0, understanding how wildcard indexes behave when indexing fields that involve objects or arrays is crucial for efficient data organization:For Embedded documents:

• When indexing embedded documents, the wildcard index delves inside the object, indexing its individual contents.
• It continues to index any nested objects it encounters within the embedded document.

For Array Fields:

• In the case of array fields, the wildcard index systematically traverses each element within the array.
• When an element within the array is an object, the wildcard index goes further inside, indexing the contents of that object.
• However, if an element within the array is another array, directly nested within the parent array, the wildcard index does not descend further into the nested array. Instead, it indexes the nested array as a whole entity.

To offer a visual aid in understanding this behavior, we've provided a flowchart that illustrates the process.

Backward compatibleEnsuring a smooth transition when downgrading from MongoDB 7.x to the previous major version, 6.x, requires some important steps:

• Drop Compound Wildcard Index: Prior to the downgrade, it is essential to drop any existing compound wildcard indexes. Failure to do so can result in MongoDB failing to start after the downgrade.
• Official Support for Downgrade: Downgrading from MongoDB 7.x to 6.x is a sensitive operation and should only be attempted with official support from the MongoDB team.

How to find the Compound Wildcard index in a collection? To identify and locate compound wildcard indexes within your databases and collections, we provide a useful JavaScript function:

 
searchCompoundWildcardIndexes = function () {
	var output = [];
	db.getMongo().getDBNames().forEach(function (dbname) {
		if (dbname != "admin" && dbname != "config" && dbname != "local") {
			db.getSiblingDB(dbname).getCollectionInfos().forEach(function (collInfo) {
				var cname = collInfo.name;
				var collType = collInfo.type;
				if (typeof collType == 'undefined' || collType == "collection") {
					if (cname != "system.profile" && cname != "system.js" && cname != "system.namespaces" && cname != "system.indexes" && cname != "system.views") {
						var wcIndex = [];
						db.getSiblingDB(dbname)[cname].getIndexes().forEach(function (i) {
							if (Object.keys(i.key).length > 1) {
								Object.keys(i.key).toString().search(/\$\*\*/) > -1 ? wcIndex.push(i) : false;
							}
						})
						if(wcIndex.length >0) {
							output.push({
								dbName: dbname,
								collection: cname,
								compoundWildcardIndexes: wcIndex
							})
						}
					}
				}
			})
		}
	})
	if (output.length > 0) {
		printjson(output); 
	} else {
		print("There is no wildCard Indexes!!")
	}
}
	

Execute the command searchCompoundWildcardIndexes() to uncover any existing compound wildcard indexes in your MongoDB environment.

 
searchCompoundWildcardIndexes()
	

Dropping Index

We can only drop the compound wildcard index using the index name. The command below is used to drop the index

 
db.product.dropIndex( "productId_productDetails_index" ) 
	

MongoDB 7.0 introduces game-changing compound indexes, revolutionizing applications handling embedded documents. This feature streamlines evolving dataset indexing while ensuring consistently optimized, high-performing queries.

Empowering developers, it enables compound indexes with field and wildcard combinations, eliminating the need for numerous separate indexes. The wildcardProjection option adds precision, reducing storage usage and accelerating query execution. However like any tool it's important to understand the considerations and limitations associated with compound wildcard indexes. Successfully utilizing this feature requires finding a balance between flexibility and specificity to achieve performance.

In conclusion, compound wildcard indexes exemplify MongoDBs commitment to adaptability and performance. As datasets become more complex and diverse, features like these will be essential, in ensuring that databases can keep up with the evolving needs of applications.

If you're eager to explore more about MongoDB 7.0 and its exciting features, don't miss out on our previous blogs:

• MongoDB 7.0 What's New - Part 1
• MongoDB 7.0 What's New - Part 2

Explore Mydbops Blogs for a wealth of MongoDB-related content and resources.