VBS is a file system federation, which is specially designed for supporting different file systems by leveraging both performance and utilization. Supported file systems include legacy systems(POSIX), HDFS/GFS(read-only), Key-value stores(append-only) and mission critical storage(memory-resident).
Due to various storage requirements, including:
It is imperative to provide a uniform platform for supporting different file systems by leveraging both performance and utilization.
We provide a file system federation, called Virtual Block Service (VBS for short), for supporting different file systems. The system architecture is shown as follows:
Virtual Block (VB) is an abstraction of a contiguous space in the underline storage device for storing data.
The features of VBS are shown as follows:
In order to provide the high availability of the system, replicas of VBs are required. In this way, we need to handle the CAP problem. The CAP theorem can be described as follows:
For replicated services, when the network is P, we can only get C or A, but not both Reads -> inconsistent data Writes -> conflicts
To tackle the CAP problem, we introduce a systematic way for handling network partition with well-defined semantics:
Prerequisite: clients can be restarted at any node. Invariants: reads and writes are always performed at the primary copy. Writes will then be propagated to replicas When a read/write is timed out S1: Detect whether the node hosting primary copy is partitioned by ZooCollection S2:Yes. Wait for system recover a new primary copy and retry S3: No. Report I am partitioned. Kill self and restart