MongoDB 7.0 Release: What's New - Part 2

Sep 20, 2023

Mins to Read

All

Welcome to the second part of our exploration into the latest enhancements and features introduced in MongoDB version 7.0. In this installment, we will continue our journey through the key modifications and additions that this update brings to the MongoDB ecosystem. If you missed Part 1, you can catch up on it here.

General Changes

Cache Refresh Time Fields

Starting with MongoDB 7.0, log messages for slow queries also include the catalogCacheIndexLookupDurationMillis field that indicates the time that the operation spent fetching information from the index cache.

This release also renames the shardVersionRefreshMillis field to placementVersionRefreshMillis.

Concurrent Storage Engine Transactions

Starting with MongoDB 7.0, an algorithm that dynamically adjusts the number of concurrent storage engine transactions is enabled by default.

wiredTiger.concurrentTransactions

A document that returns information about

The number of concurrent read transactions allowed into the WiredTiger storage engine.
The number of concurrent write transactions allowed into the WiredTiger storage engine.
Any adjustments that the system has made to the number of allowed concurrent transactions.

Starting with MongoDB 7.0, wiredTigerConcurrentReadTransactions and wiredTigerConcurrentWriteTransactions parameters are available for all (WiredTiger and In-Memory) storage engines. In earlier versions, this parameter was available for the WiredTiger storage engine only.

You can only set this parameter during run time using the setParameter database command

 
db.adminCommand( { setParameter: 1, wiredTigerConcurrentReadTransactions:  } )

db.adminCommand( { setParameter: 1, wiredTigerConcurrentWriteTransactions:  } )

CurrentOp Metrics

Starting with MongoDB 7.0, the currentOp command and the db.currentOp() method include these new fields

currentOp.admissionPriority

The value represents the precedence of an operation as it seeks to obtain a ticket for executing a storage engine action, exclusively for internal purposes.

Possible values are: low, normal, and immediate. Only operations with a low value are reported.

currentOp.collUuid

The UUID of the sampled collection. This field only appears on documents related to query sampling.

currentOp.startTime

The time at which query sampling began. This field only appears on documents related to query sampling.

currentOp.samplesPerSecond

The maximum number of queries to samples per second. On a sharded cluster, this is reported on mongos instead of mongod. On a replica set, this is reported on mongod. This field only appears on documents related to query sampling.

currentOp.sampledReadsCount

The number of sampled read queries. This field only appears on documents related to query sampling.

currentOp.sampledWritesCount

The number of sampled write queries. This field only appears on documents related to query sampling.

currentOp.sampledReadsBytes

The size of the sampled read queries, in bytes.On a replica set, this is reported on every mongod.On a sharded cluster, this only reported on mongod with --shardsvr. This field only appears on documents related to query sampling.

currentOp.sampledWritesBytes

The size of the sampled write queries, in bytes. On a replica set, this is reported on every mongod.On a sharded cluster, this only reported on mongod with --shardsvr. This field only appears on documents related to query sampling.

$currentOp (aggregation) Metrics

Starting with MongoDB 7.0, the currentOp aggregation stage includes these new fields:

Compound Wildcard Indexes

You can create compound wildcard indexes. A compound wildcard index has one wildcard term and one or more additional index terms.

Use compound wildcard indexes to support queries on known patterns and to limit the total number of indexes in a collection.

Use the below commands to create the compound wildcard indexes at runtime.

 
db.runCommand(
   {
       createIndexes: "collegeDetails",
       indexes: [{
             key: {studentId: 1, "$**": 1},
             name: "college_customFields",
             wildcardProjection: { "customFields.addr": 1, "customFields.name": 1}
 }]})

The wildcard index term, $**, specifies every field in the collection. The wildcardProjection limits the index to the specified fields, customFields.addr and customFields.name.
You can only use a wildcardProjection when the wildcard term is $**.

Use the below commands to create the compound wildcard indexes.

 
db.salesData.createIndex({ tenantId: 1, "customFields.$**": 1 },{name: "tenant_customFields_shellHelper",background:true})

Limitations of WildCard index

A compound wildcard index can only have one wildcard term. For example, you cannot specify the following index:

 
{ userID: 1, "object1.$**": 1, "object2.$**": 1 }

The non-wildcard terms in a compound wildcard index must be single key terms. Multikey index terms are not permitted.
The wildcardProjection option is only valid when the wildcard field is $**. You cannot use wildcardProjection when you specify a field path for the wildcard index term.
You cannot include the same field in the wildcard fields and the regular fields. You can use a wildcardProjection to exclude fields from the wildcard pattern.

Large Change Stream Events

If you have change stream events larger than 16 MB, you can use the new $changeStreamSplitLargeEvent stage to split the events into smaller fragments.

Before you decide to use $changeStreamSplitLargeEvent, you should first try to reduce the change event size. For example:

Don't request documents pre- or post-images unless your application requires them. This generates fullDocument and fullDocumentBeforeChange fields in more cases, which are typically the largest objects in a change event.
Use a $project stage to include only the fields necessary for your application. This reduces the change in event size and avoids the additional time to split large events into fragments. This allows more change events to be returned in each batch.

You can only have one $changeStreamSplitLargeEvent stage in your pipeline, and it must be the last stage. You can only use $changeStreamSplitLargeEvent in a $changeStream pipeline.

 
myChangeStreamCursor = db.myCollection.watch(
   [ { $changeStreamSplitLargeEvent: {} } ],
   { fullDocument: "required", fullDocumentBeforeChange: "required" }
)

The following new metrics report information about large change stream events:

serverStatus Output Change

serverStatus includes the following new fields in its output.

plancache

A document that reports query plan cache statistics.

planCache.classic.hits

Number of classic execution engine query plans found in the query cache and reused to avoid the query planning phase.

planCache.classic.misses

Number of classic execution engine query plans which were not found in the query cache and went through the query planning phase.

planCache.sbe.hits

Number of slot-based execution engine query plans found in the query cache and reused to avoid the query planning phase.

planCache.sbe.misses

The Number of slot-based execution engine query plans that were not found in the query cache and went through the query planning phase.

queryAnalyzer Metrics

Slot-Based Query Execution Engine

The slot-based query execution engine improves performance for a wider range of find and aggregation queries.

Slow query log messages now include a queryFramework field that indicates which query engine completed the query.

queryFramework: classic indicates that the classic engine completed the query.
queryFramework: sbe indicates that the slot-based query execution engine completed the query.

User Roles System Variable

Starting with MongoDB 7.0, you can use the new USER_ROLES system variable to return the roles of the current user.

To use a system variable, add $$ to the start of the variable name. Specify the USER_ROLES system variable as $$USER_ROLES.

 
db.budget.find( {$expr: {$not: {$eq: [ { $setIntersection: [ "$allowedRoles", "$$USER_ROLES.role" ] }, [] ]}}} )

New Sharding Statistics for Chunk Migrations

MongoDB includes the following new sharding statistics for chunk migrations:

shardingStatistics.countDonorMoveChunkCommitted

Total number of chunk migrations committed on the shard, of which the current node is a member.

The chunk migration is performed by moveChunk and moveRange commands in a range migration procedure. Only available on a shard.

shardingStatistics.countDonorMoveChunkAborted

Total number of chunk migrations aborted on the shard, of which the current node is a member.

The chunk migration is performed by moveChunk and moveRange commands in a range migration procedure. Only available on a shard.

shardingStatistics.totalDonorMoveChunkTimeMillis

Cumulative time in milliseconds to move chunks from the current shard to another shard. For each chunk migration, the time starts when a moveRange or moveChunk command starts, and ends when the chunk is moved to another shard in a range migration procedure. Only available on a shard.

shardingStatistics.countBytesClonedOnRecipient

Cumulative number of bytes cloned on the current member node during a range migration procedure, where the current member node acted as the primary node for the recipient shard.

For details about data synchronization, see Replica Set Data Synchronization. Only available on a shard.

shardingStatistics.countDocsClonedOnCatchUpOnRecipient

Cumulative number of documents cloned on the current member node during the catch-up phase of a range migration procedure, where the current member node acted as the primary node for the recipient shard.Only available on a shard.

shardingStatistics.countBytesClonedOnCatchUpOnRecipient

Cumulative number of bytes cloned on the current member node during the catch-up phase of a range migration procedure, where the current member node acted as the primary node for the recipient shard.Only available on a shard.

New Slow Query Log Message

Starting with MongoDB 7.0, the totalOplogSlotDurationMicros in the slow query log message shows the time between a write operation getting a commit timestamp to commit the storage engine writes and actually committing. mongod supports parallel writes. However, it commits write operations with commit timestamps in any order.

New Parameters

analyzeShardKey-related Parameters

MongoDB 7.0 adds the following parameters related to the analyzeShardKey command:

autoMergerIntervalSecs Parameter

MongoDB 7.0 adds the autoMergerIntervalSecs parameter which, when AutoMerger is enabled, specifies the amount of time between automerging rounds, in seconds. autoMergerIntervalSecs can only be set on config servers of sharded clusters.

autoMergerThrottlingMS Parameter

MongoDB 7.0 adds the autoMergerThrottlingMS which, when AutoMerger is enabled, specifies the minimum amount time between merges initiated by the AutoMerger on the same collection, in milliseconds. autoMergerThrottlingMS can only be set on config servers of sharded clusters.

balancerMigrationsThrottlingMs Parameter

MongoDB 7.0 adds the balancerMigrationsThrottlingMs parameter which allows you to throttle the balancing rate.

gEnableDetailedConnectionHealthMetricLogLines Parameter

MongoDB 7.0 adds the gEnableDetailedConnectionHealthMetricLogLines parameter which lets you specify whether or not a set of log messages related to cluster connection health metrics appears in the log.

oidcIdentityProviders Parameter

MongoDB 7.0 adds the oidcIdentityProviders parameter which allows you to specify identity provider (IDP) configurations when using OpenID Connect authentication.

2.12 configureQueryAnalysis-related Parameters

In MongoDB 7.0, several new parameters are introduced in connection with the configureQueryAnalysis command.

MongoDB 7.0 What's New Part 2 by Shiney J

‍

MongoDB 7.0 brings a host of exciting updates that cater to various aspects of database management and performance optimization. From improved query metrics to enhanced sharding statistics and dynamic transaction handling, MongoDB 7.0 empowers users with powerful tools to manage their databases efficiently.

We hope this overview has provided you with a clearer understanding of the advancements in MongoDB 7.0 and how they can elevate your database operations.

Thank you for joining us in exploring the MongoDB 7.0 updates.

Explore Mydbops Blogs for a wealth of MongoDB-related content and resources.

Slot-Based Execution Engine

MongoDB 7.0 updates

No items found.