Welcome to the second part of our exploration into the latest enhancements and features introduced in MongoDB version 7.0. In this installment, we will continue our journey through the key modifications and additions that this update brings to the MongoDB ecosystem. If you missed Part 1, you can catch up on it here.
General Changes
Cache Refresh Time Fields
Starting with MongoDB 7.0, log messages for slow queries also include the catalogCacheIndexLookupDurationMillis field that indicates the time that the operation spent fetching information from the index cache.
This release also renames the shardVersionRefreshMillis field to placementVersionRefreshMillis.
Concurrent Storage Engine Transactions
Starting with MongoDB 7.0, an algorithm that dynamically adjusts the number of concurrent storage engine transactions is enabled by default.
wiredTiger.concurrentTransactions
A document that returns information about
- The number of concurrent read transactions allowed into the WiredTiger storage engine.
- The number of concurrent write transactions allowed into the WiredTiger storage engine.
- Any adjustments that the system has made to the number of allowed concurrent transactions.
Starting with MongoDB 7.0, wiredTigerConcurrentReadTransactions and wiredTigerConcurrentWriteTransactions parameters are available for all (WiredTiger and In-Memory) storage engines. In earlier versions, this parameter was available for the WiredTiger storage engine only.
You can only set this parameter during run time using the setParameter database command
CurrentOp Metrics
Starting with MongoDB 7.0, the currentOp command and the db.currentOp() method include these new fields
currentOp.admissionPriority
The value represents the precedence of an operation as it seeks to obtain a ticket for executing a storage engine action, exclusively for internal purposes.
Possible values are: low, normal, and immediate. Only operations with a low value are reported.
currentOp.collUuid
The UUID of the sampled collection. This field only appears on documents related to query sampling.
currentOp.startTime
The time at which query sampling began. This field only appears on documents related to query sampling.
currentOp.samplesPerSecond
The maximum number of queries to samples per second. On a sharded cluster, this is reported on mongos instead of mongod. On a replica set, this is reported on mongod. This field only appears on documents related to query sampling.
currentOp.sampledReadsCount
The number of sampled read queries. This field only appears on documents related to query sampling.
currentOp.sampledWritesCount
The number of sampled write queries. This field only appears on documents related to query sampling.
currentOp.sampledReadsBytes
The size of the sampled read queries, in bytes.On a replica set, this is reported on every mongod.On a sharded cluster, this only reported on mongod with --shardsvr. This field only appears on documents related to query sampling.
currentOp.sampledWritesBytes
The size of the sampled write queries, in bytes. On a replica set, this is reported on every mongod.On a sharded cluster, this only reported on mongod with --shardsvr. This field only appears on documents related to query sampling.
$currentOp (aggregation) Metrics
Starting with MongoDB 7.0, the currentOp aggregation stage includes these new fields:
- $currentOp.collUuid
- $currentOp.startTime
- $currentOp.samplesPerSecond
- $currentOp.sampledReadsCount
- $currentOp.sampledWritesCount
- $currentOp.sampledReadsBytes
- $currentOp.sampledWritesBytes
Compound Wildcard Indexes
You can create compound wildcard indexes. A compound wildcard index has one wildcard term and one or more additional index terms.
Use compound wildcard indexes to support queries on known patterns and to limit the total number of indexes in a collection.
Use the below commands to create the compound wildcard indexes at runtime.
- The wildcard index term, $**, specifies every field in the collection. The wildcardProjection limits the index to the specified fields, customFields.addr and customFields.name.
- You can only use a wildcardProjection when the wildcard term is $**.
Use the below commands to create the compound wildcard indexes.
Limitations of WildCard index
- A compound wildcard index can only have one wildcard term. For example, you cannot specify the following index:
- The non-wildcard terms in a compound wildcard index must be single key terms. Multikey index terms are not permitted.
- The wildcardProjection option is only valid when the wildcard field is $**. You cannot use wildcardProjection when you specify a field path for the wildcard index term.
- You cannot include the same field in the wildcard fields and the regular fields. You can use a wildcardProjection to exclude fields from the wildcard pattern.
Large Change Stream Events
If you have change stream events larger than 16 MB, you can use the new $changeStreamSplitLargeEvent stage to split the events into smaller fragments.
Before you decide to use $changeStreamSplitLargeEvent, you should first try to reduce the change event size. For example:
- Don't request documents pre- or post-images unless your application requires them. This generates fullDocument and fullDocumentBeforeChange fields in more cases, which are typically the largest objects in a change event.
- Use a $project stage to include only the fields necessary for your application. This reduces the change in event size and avoids the additional time to split large events into fragments. This allows more change events to be returned in each batch.
You can only have one $changeStreamSplitLargeEvent stage in your pipeline, and it must be the last stage. You can only use $changeStreamSplitLargeEvent in a $changeStream pipeline.
The following new metrics report information about large change stream events:
serverStatus Output Change
serverStatus includes the following new fields in its output.
plancache
A document that reports query plan cache statistics.
planCache.classic.hits
Number of classic execution engine query plans found in the query cache and reused to avoid the query planning phase.
planCache.classic.misses
Number of classic execution engine query plans which were not found in the query cache and went through the query planning phase.
planCache.sbe.hits
Number of slot-based execution engine query plans found in the query cache and reused to avoid the query planning phase.
planCache.sbe.misses
The Number of slot-based execution engine query plans that were not found in the query cache and went through the query planning phase.
queryAnalyzer Metrics
- queryAnalyzers.activeCollections
- queryAnalyzers.totalCollections
- queryAnalyzers.totalSampledReadsCount
- queryAnalyzers.totalSampledWritesCount
- queryAnalyzers.totalSampledReadsBytes
- queryAnalyzers.totalSampledWritesBytes
Slot-Based Query Execution Engine
The slot-based query execution engine improves performance for a wider range of find and aggregation queries.
Slow query log messages now include a queryFramework field that indicates which query engine completed the query.
- queryFramework: classic indicates that the classic engine completed the query.
- queryFramework: sbe indicates that the slot-based query execution engine completed the query.
User Roles System Variable
Starting with MongoDB 7.0, you can use the new USER_ROLES system variable to return the roles of the current user.
To use a system variable, add $$ to the start of the variable name. Specify the USER_ROLES system variable as $$USER_ROLES.
New Sharding Statistics for Chunk Migrations
MongoDB includes the following new sharding statistics for chunk migrations:
shardingStatistics.countDonorMoveChunkCommitted
Total number of chunk migrations committed on the shard, of which the current node is a member.
The chunk migration is performed by moveChunk and moveRange commands in a range migration procedure. Only available on a shard.
shardingStatistics.countDonorMoveChunkAborted
Total number of chunk migrations aborted on the shard, of which the current node is a member.
The chunk migration is performed by moveChunk and moveRange commands in a range migration procedure. Only available on a shard.
shardingStatistics.totalDonorMoveChunkTimeMillis
Cumulative time in milliseconds to move chunks from the current shard to another shard. For each chunk migration, the time starts when a moveRange or moveChunk command starts, and ends when the chunk is moved to another shard in a range migration procedure. Only available on a shard.
shardingStatistics.countBytesClonedOnRecipient
Cumulative number of bytes cloned on the current member node during a range migration procedure, where the current member node acted as the primary node for the recipient shard.
For details about data synchronization, see Replica Set Data Synchronization. Only available on a shard.
shardingStatistics.countDocsClonedOnCatchUpOnRecipient
Cumulative number of documents cloned on the current member node during the catch-up phase of a range migration procedure, where the current member node acted as the primary node for the recipient shard.Only available on a shard.
shardingStatistics.countBytesClonedOnCatchUpOnRecipient
Cumulative number of bytes cloned on the current member node during the catch-up phase of a range migration procedure, where the current member node acted as the primary node for the recipient shard.Only available on a shard.
New Slow Query Log Message
Starting with MongoDB 7.0, the totalOplogSlotDurationMicros in the slow query log message shows the time between a write operation getting a commit timestamp to commit the storage engine writes and actually committing. mongod supports parallel writes. However, it commits write operations with commit timestamps in any order.
New Parameters
analyzeShardKey-related Parameters
MongoDB 7.0 adds the following parameters related to the analyzeShardKey command:
- analyzeShardKeyCharacteristicsDefaultSampleSize
- analyzeShardKeyNumMostCommonValues
- analyzeShardKeyNumRanges
- analyzeShardKeyMonotonicityCorrelationCoefficientThreshold
autoMergerIntervalSecs Parameter
MongoDB 7.0 adds the autoMergerIntervalSecs parameter which, when AutoMerger is enabled, specifies the amount of time between automerging rounds, in seconds. autoMergerIntervalSecs can only be set on config servers of sharded clusters.
autoMergerThrottlingMS Parameter
MongoDB 7.0 adds the autoMergerThrottlingMS which, when AutoMerger is enabled, specifies the minimum amount time between merges initiated by the AutoMerger on the same collection, in milliseconds. autoMergerThrottlingMS can only be set on config servers of sharded clusters.
balancerMigrationsThrottlingMs Parameter
MongoDB 7.0 adds the balancerMigrationsThrottlingMs parameter which allows you to throttle the balancing rate.
gEnableDetailedConnectionHealthMetricLogLines Parameter
MongoDB 7.0 adds the gEnableDetailedConnectionHealthMetricLogLines parameter which lets you specify whether or not a set of log messages related to cluster connection health metrics appears in the log.
oidcIdentityProviders Parameter
MongoDB 7.0 adds the oidcIdentityProviders parameter which allows you to specify identity provider (IDP) configurations when using OpenID Connect authentication.
2.12 configureQueryAnalysis-related Parameters
In MongoDB 7.0, several new parameters are introduced in connection with the configureQueryAnalysis command.
- queryAnalysisSamplerConfigurationRefreshSecs
- queryAnalysisWriterIntervalSecs
- queryAnalysisWriterMaxMemoryUsageBytes
- queryAnalysisWriterMaxBatchSize
- queryAnalysisSampleExpirationSecs
MongoDB 7.0 brings a host of exciting updates that cater to various aspects of database management and performance optimization. From improved query metrics to enhanced sharding statistics and dynamic transaction handling, MongoDB 7.0 empowers users with powerful tools to manage their databases efficiently.
We hope this overview has provided you with a clearer understanding of the advancements in MongoDB 7.0 and how they can elevate your database operations.
Thank you for joining us in exploring the MongoDB 7.0 updates.
Explore Mydbops Blogs for a wealth of MongoDB-related content and resources.