Maximize MongoDB Log Understanding: Hatchet's Log Analytical Arsenal

Aug 21, 2023

Mins to Read

All

Hatchet is a powerful MongoDB JSON log analyzer designed to streamline the process of analyzing MongoDB logs. Ever wished you could decode the mysteries of MongoDB logs without getting tangled up? Say hello to Hatchet – your MongoDB log analysis superhero!

It provides advanced features for log processing, aggregation and storage of the processed data. The web interface of Hatchet is highly interactive and user-friendly, providing a seamless experience for searching logs and navigating through reports and charts. The intuitive design and easy-to-use interface make it simple for users to find what they need when they need it. Hatchet provides fast access to data and a high level of performance, making it the ideal solution for logs analysis and management.

Ready to dive into the world of smarter log analysis? Let's venture into the world of Hatchet and master the art of MongoDB log analysis!

Installation Process of Hatchet

Building Hatchet

To get started, clone the repository and execute the build.sh script. Please note that you'll need gcc to support CGO during this process.

 
git clone --depth 1 git@github.com:simagix/hatchet.git
cd hatchet ; ./build.sh

Upon completion, you'll find the executable hatchet file in the dist/ directory. Your Hatchet installation is now set up.

Quick Start

To process the log file mongod.log and launch a web server that listens on port 3721, use the following command:

 
./dist/hatchet -web mongod.log

Open your web browser and navigate to http://localhost:3721/ to explore the generated reports and charts.

Alternatively, you can opt for in-memory mode without storing data persistently. For instance:

 
./dist/hatchet -url in-memory mongod.log

3. You can concurrently use Hatchet on different ports as well.

 
./dist/hatchet -web -port 3700 mongod.log

or

./dist/hatchet -url -port 3700 in-memory mongod.log

Additional Features

Beyond its adept file-reading capabilities, Hatchet unveils a world of extra functionalities that take your analysis a step further. These capabilities include reading from S3, web servers, and even MongoDB Atlas. Let's explore these enhancements:

Web Servers

The tool supports reading from web servers using both the http:// and https:// protocols. The -user flag is optional when using basic authentication.

 
hatchet [-user {username}:{password}] https://{hostname}/{log name}

MongoDB Atlas

To download logs directly from MongoDB Atlas, you will need to use the -user and -digest flags and provide the necessary information for both. These flags are used to authenticate and authorize your access to the database.

 
hatchet -user {pub key}:{private key} -digest https://cloud.mongodb.com/api/atlas/v1.0/groups/{group ID}/clusters/{hostname}/logs/mongodb.gz

AWS S3

Hatchet is equipped with the capability to fetch files from AWS S3. During the download process, Hatchet seamlessly extracts the Region and Credentials details from the configuration files situated at ${HOME}/.aws. This eliminates the need for manual input of this information whenever you utilize Hatchet to download files from AWS S3.

 
hatchet -s3 [--endpoint-url {test endpoint}] {bucket}/{key name}

Logs Obfuscation

Hatchet can be used to obfuscate logs. It inherently conceals the values of identified patterns within the attr field. These patterns encompass sensitive information like sessionId, port numbers and namespaces data, all done automatically.

 
hatchet -obfuscate {log file} > {output file}

The Versatile and Flexible Log Analyzer

Inside the realm of the Hatchet log analyzer lies an array of diverse report types, each catering to distinct needs:

‍

Audit

This page offers insights into Driver Compatibility, Exceptions, Failed Operations, Operations Stats, Stats by Namespaces, and intelligent insights concerning upgrades, along with automatic suggestions.

The Hatchet log analyzer provides intelligent recommendations by meticulously analyzing the shared MongoDB logs.

Driver Compatibility
The table below presents information about the driver, version, IP address, and corresponding recommendations for ensuring driver compatibility.

Exceptions
Various types of errors detected within the log file are logged in the exception table, along with their respective levels of severity.

MongoDB log analyzer - Hatchet tool - Exceptions 1

By clicking the search button within the table, you can access the error logs associated with it, stored in the exception table. Furthermore, you have the option to apply filters based on the component and severity.

Stats By Namespaces
The below table group the data based on the namespaces

Clicking the search button within the table for the specified namespace allows you to access the logs associated with that particular namespace. Additionally, you have the option to apply filters based on the component and severity, tailored to your specific use case.

Operation Stats
It categorizes operations and provides a count for each type of operation.

Clicking on the graph icon in the aforementioned table enables the generation of graphs depicting operations over the entire time range of the provided MongoDB logs. Furthermore, it is possible to apply time range filters to the graphs.

Stats

This page offers a concise overview of slow operations, including query patterns, operation counts, Index types and durations. The data within the table can also be sorted as needed.

TopN

This page displays the top 23 slowest operation logs.

Charts

Various graph types can be generated using the provided MongoDB logs.

In summary, Hatchet was crafted to elevate the log analysis capabilities of Keyhole, alongside the utilization of mtools for collecting connection and IP details.

Hatchet delivers an unmatched log processing experience, catering to both visual and hands-on users. With its clear reports, concise charts, comprehensive Stats page, and multiple avenues for accessing processed logs, it ensures a comprehensive log analysis journey.

For further exploration: