Documentation

L-Store Overview

L-Store (a.k.a. LEAP-Store) is a distributed in-memory database with non-2PC transaction management. It can be deployed and run on a shared-nothing cluster. For fault-tolerant purpose, L-Store exploits any distributed in-memory storage as the underlying persistent storage.

Downloading L-Store

Get L-Store from the downloads page of the project website.

Building L-Store

L-Store is implemented using Scala and compiled using SBT. After the source code is cloned, run the following command to generate the L-Store jar. 

$ sbt assembly

The generated lstore-assembly-xxx.jar is placed in ./target/scala-2.11/, where xxx indicates the version.

Note that the compilation only depends on the availability of Java and SBT. Although L-Store is coded in Scala, it is unnecessary to install Scala in your system for the purpose of compilation.

Using L-Store via Scripts

Launching the Master

Choose one node as the L-Store master and run the start_master.sh script with any optional parameters.

sh start_master.sh --master-port 4320

Once the master is up, it waits for the workers.

Launching the Workers

Open another terminal and run the launch_slaves.sh script with any optional parameters. 

sh launch_slaves.sh --port 4321

Then the nodes registered in the slaves file will be initiated as the L-Store workers. Note that each entry in the slaves file contains two attributes: host name of the worker node and number of task slots provided by that node. For example, the entry below indicates the node at awan-1-43-0.awan.comp.nus.edu.sg is a worker with 16 task slots.

awan-1-43-0.awan.comp.nus.edu.sg 16

Auxiliary Scripts

  • test_nodes.sh: Test the connectivity of nodes registered in the slaves file. It is useful for diagnosing the health of nodes in the cluster.

  • mkdir_slaves.sh: Create the specified directory in all the worker nodes registered in the slaves file. It is recommended to run this script for the first time of nodes being registered.

  • kill_slaves.sh: Send the kill signal to all the worker nodes registered in the slaves file. This is only for trouble-shooting or interrupting the computation.

Configuration

User must properly configure the above scripts according to his/her running environment before using them. The following parameters need to be set by the user.

  • LSTORE_VER: The version of L-Store.
  • MASTER: The host name or IP of the master.
  • MASTER_INPUT_DIR: The directory of input data.
  • MASTER_OUTPUT_DIR: The output directory of the master.
  • EXEC_CMD (in exec_task.sh): The execution command which takes a single parameter (i.e., the task identifier) for the computation.
  • JAR_DIR: The directory containing the L-Store jar.
  • HOSTS: The file specifying the registered worker nodes, i.e., the slaves file.
  • SLAVE_LOG: The absolute path of log file of each worker. Note that each log file is overwritten by the latest running.
  • REQUIRED_DIR: The directory that needs to be created in the registered worker nodes.

Using L-Store Directly

This section describes the detailed operations behind the start_master.sh and launch_slaves.sh scripts. This is mainly for the purpose of development and debug. For normal use of L-Store, we suggest user to leverage the scripts as mentioned above.

Launching the Master

Launch the L-Store master by running the command below. 

$ java -jar lstore-assembly-xxx.jar --master -i /path/to/input/dir -s sh[]/path/to/script.sh[]@t@

After the master is launched, it waits for the workers. No data processing is actually performed unless at least one worker is launched.

A script template is required by the -s parameter when launching the master. It describes how each task is launched for execution. Since the command line parser of L-Store (using args4j) does not allow any white space within a parameter value, a symbolic placeholder of white space (by default, "[]") is used to represent a white space. Moreover, a placeholder of task identifier (by default, "@t@") is used to specify the appearance of the task identifier (e.g., file name) in the task launching script.

During runtime, the master compiles the script template by replacing the placeholder of task identifier with the actual task identifier to generate the task launching script. The script is then sent to the worker and executed by one of its task slots.

Launching the Worker

Launch the L-Store worker by running the command below.

$ java -jar lstore-assembly-xxx.jar -m specify.master.hostname -t 4

This initiates a worker with 4 task slots for task processing. After the worker is launched, it immediately starts to fetch tasks from the master to process.

Parameters

The --help parameter shows the information of the required/optional parameters. For example, run L-Store with the --help. 

$ java -jar lstore-assembly-xxx.jar --help

Description of the parameters is displayed as follows. 

  --forever                   : run without stopping (for master only) [def:false]      
  --master                    : run as the master [def:false]
  --master-port               : master port number [def:5350]
  --port                      : slave port number [def:5351]
  -? (--help)                 : print this help message
  -d (--output-dir)           : output directory [def:output]
  -i (--input-dir)            : directory of input files [required for master]
  -m (--master-host)          : master hostname [def:none]
  -o (--output-file)          : output filename [def:none]
  -s (--script)               : script template [def:"echo[]@t@"]
  -t (--task-slots)           : number of task slots [def:1]

Note that --forever, --master-port, -d, -o, -s are for master only, and -port, -m, -t are for worker only.

By default, a worker uses the same path of input directory as for the master. Alternatively, a worker can use a different path of input directory by setting the -i parameter to the corresponding path within its launching command.

Hidden Parameters

The following parameters are hidden in the --help printing. They are provided for ad-hoc purposes. 

  --task-id-placeholder      : placeholder of task identifier [def:"@t@"]
  --space-placeholder        : placeholder of white space [def:"[]"]
  --shutdown-delay           : delay (in seconds) before shutdown [def:1]

Note that all these hidden parameters are for master only.