Struggling with slow databases as your data keeps growing? Regular databases can get overloaded when lots of users try to write information at once. This can make your application sluggish and risk losing data. But there's a solution! TiDB is a powerful database that can handle massive amounts of data. We will explain how TiDB's special write process keeps your data safe and fast, even as your application grows!

But that's not all! TiDB also offers powerful scheduling capabilities to automate tasks and optimize performance. To learn more about TiDB Scheduling, check out our dedicated blog post here: TiDB Scheduling: The Secret Weapon for Peak Database Performance

TiDB

TiDB is a distributed SQL database that has gained popularity for its horizontal scalability, strong consistency, and compatibility with the MySQL protocol. One of the key aspects that set TiDB apart is its unique write path, which is designed to handle high concurrency and ensure data consistency across multiple nodes. In this blog, we'll take a detailed look at the write path in TiDB, exploring its components, functionality, and how it manages to provide a seamless experience for handling large-scale data.

The Architecture of TiDB

Before diving into the write path, it's essential to understand the overall architecture of TiDB. TiDB consists of three main components:

• TiDB Server: The stateless SQL layer that processes user queries, connects to the storage layer, and returns results to the client.
• Placement Driver (PD): The cluster manager that handles metadata, scheduling, and load balancing.
• TiKV: The distributed transactional key-value storage layer where the actual data resides.

The Write Path Explained

The write path in TiDB is the journey that data takes from the moment it's sent by the client until it's safely stored in the TiKV nodes. Here's a step-by-step breakdown:

• Client Request: It all begins when a client sends a write request (such as an INSERT, UPDATE, or DELETE statement) to the TiDB server.
• SQL Processing: The TiDB server parses and executes the SQL statement. It then converts the statement into a series of prewrite operations for the underlying key-value pairs using the DistSQL API and KV API
  • DistSQL API: Breaks down the execution plan into smaller tasks that can be distributed to multiple TiKV nodes. Each TiKV node processes its portion of the data and returns the results.
  • KV API (Key-Value API): Provides the interface for interacting with the underlying TiKV(key-value) storage layer
• Transaction Handling: TiDB uses a distributed transaction model based on Google's Percolator. Each write operation is associated with a transaction, which is assigned a unique timestamp (TS) from the PD.
• Prewrite Phase: In this phase, TiDB sends the pre-write requests to the region corresponding TiKV nodes. TiKV locks the keys to be written and writes the values to a temporary buffer, ensuring that other transactions can't modify these keys until the transaction is committed.
• Getting Commit Timestamp: Once the pre-write phase is successful, TiDB requests a new timestamp from the PD to serve as the commit TS for the transaction.
• Commit Phase: TiDB then sends commit requests to the TiKV nodes, instructing them to make the prewritten data permanent. The keys are unlocked, and the data becomes visible to other transactions.
• Handling Conflicts and Retries: If there are conflicts (e.g., two transactions trying to modify the same key), TiDB may need to retry the transaction with a new timestamp.
• Acknowledgment: Once the commit phase is successful, TiDB sends an acknowledgment to the client, indicating that the write operation is complete.

Key Features of the Write Path:

• Distributed Transactions: TiDB's use of distributed transactions ensures strong consistency across multiple nodes, even in the event of network partitions or node failures.
• Optimistic Concurrency Control: TiDB uses optimistic concurrency control to reduce lock contention, improving performance for workloads with high concurrency.
• Horizontal Scalability: The write path is designed to scale horizontally, allowing you to add more TiKV nodes to handle increased load without significant changes to the application.

The write path in TiDB is a testament to its robust architecture, designed to handle large-scale, distributed workloads with ease. By understanding the intricacies of the write path, developers and architects can better leverage TiDB's capabilities to build scalable and resilient applications.

Mydbops offers comprehensive TiDB Consulting and Remote DBA services to help you leverage the full potential of TiDB. Our team of Certified TiDB Professionals possesses in-depth knowledge of TiDB's architecture and the intricacies of write path. Contact Us Today !

{{cta}}