Why Not 2PC

For OLTP workloads, shared-nothing systems conventionally depend on the 2PC protocol for preserving the atomicity and serializability of distributed transactions. Although these systems offer superior performance for local transactions (i.e., transactions whose data are hosted on a single node), overhead of processing distributed transactions (i.e., transactions whose data are spread over multiple nodes) can degrade the system performance significantly. The key contributor to the degradation is the 2PC protocol. To commit a distributed transaction, the 2PC protocol requires multiple network round-trips between all participant machines which typically take a much longer time than the local transaction processing.

The LEAP Protocol

Towards non-2PC distributed transaction processing in a shared-nothing architecture, we propose the LEAP (Localizing Executions via Aggressive Placement of data) protocol as a solution. LEAP converts a distributed transaction into a local transaction to eliminate the expensive distributed commit procedure. To localize the execution of a distributed transaction, LEAP aggressively places all cross-node data needed for the transaction on a single node (i.e., the node issuing the transaction). By doing so, the transaction executor would physically hold all the data it uses during the transaction execution and can easily commit or abort the transaction without worrying about the distributed consensus as in the 2PC protocol. LEAP is designed in anticipation of the wide availability of modern fast networks (e.g., 10 Gigabit Ethernet, InfiniBand) to provide latency and bandwidth guarantees.

LEAP-Based OLTP Engine

We developed a LEAP-based OLTP engine, namely L-Store (a.k.a. LEAP-Store). The figure below shows the architecture of L-Store which consists of N nodes and one distributed in-memory storage.

architecture

There are four main functional components in each node as follows.

  • Application Layer: This layer provides the interface to interact with the client applications.
  • Storage Engine: L-Store employs an in-memory storage. To facilitate efficient data access, lightweight indexes (e.g., hash index) are utilized to facilitate speedy retrieval of the data tables (for actual data records), the owner tables (for data owner information) and the locks.
  • Transaction Engine: This engine handles transaction execution. It adopts multi-threading to process transactions in order to increase the degree of parallelism. It interacts with the storage engine to fetch/write the data and the node management component to detect the data locks and send the ownership transfer request.
  • Node Management: This layer manages the entire node including controlling the locks and running the LEAP protocol. It also serves as the agent for inter-node communication. Its implementation applies the Actor model to interact with the messages for ownership transfer.