Standalone

For applications that need the performance characteristics and data model that Hypertable has to offer, but don't require horizontal scalability, Hypertable can be setup to run on a single machine over its native filesystem.  For better performance characteristics, this machine can be configured with an SSD drive.  This document describes how to get Hypertable up and running on a single standalone machine.

Table of Contents

Prerequisites

Before you get started with the installation, there are some general system requirements that need to be satisfied before proceeding.  These requirements are described as follows:

  • root access - If you plan to install Hypertable in the standard location (/opt/hypertable) you will need root access to create and populate that directory.  You can either carry out the steps below while logged in as the root user or you need to have sudo privileges.  If you do not have root access, you can install the .tar.bz2 package anywhere you like.
     
  • open file limit - Most operating systems have a limit on the total number of files that a process can have open at any one time.  If you plan to load a large amount of data into Hypertable, you will likely need to increase this limit.  See Open File Limit for details on how to increase this limit.
     
  • firewall - The Hypertable processes use TCP and UDP to communicate with one another and with client applications.  Firewalls can block this traffic and prevent Hypertable from operating properly.  Any firewall that blocks traffic to or from specific ports should be disabled or the appropriate ports should be opened up to allow Hypertable communication.  See Hypertable Firewall Requirements for instructions on how to do this.
     
  • VERSION - This document assumes that the shell variable $VERSION contains the version number of Hypertable being installed (e.g. 0.9.5.5).

Step 1 - Install Hypertable Package

Hypertable can be installed via binary packages which can be downloaded as described on the Hypertable Download page. The packages come bundled with nearly all of the dependent shared libraries. The nice thing about this approach is that just two packages are required for linux, a 64-bit linux package and a 32-bit linux package. The only requirement is that your system be built with glibc 2.4+ (released on March 6th 2006) which provides stack smashing protection.    Hypertable comes with a program launch script, ht, that sets up LD_LIBRARY_PATH (or DYLD_LIBRARY_PATH) to point to the lib/ directory of the installation so that the dependent libraries can be found by the dynamic linker. 

To begin the package installation, switch to the directory containing the package file and then issue the command listed below for your operating system.

Redhat, CentOS, or SUSE Installation

$ sudo rpm -ivh --replacepkgs --nomd5 package.rpm

Debian or Ubuntu Installation

$ sudo dpkg --install package.deb

Bzipped Archive Installation

$ sudo tar xjvf package.tar.bz2

Mac installation

Double-click the package.dmg file and follow the instructions

The Redhat, Debian, and Mac packages will install Hypertable under a directory by the name of /opt/hypertable/$VERSION by default.  You will need to change the ownership of the installation files and directories to the owner that you plan to launch the services as. For example:

sudo chown -R john:staff /opt/hypertable/$VERSION

Step 2 - FHS-ize Installation

See Filesystem Hierarchy Standard for an introduction to FHS.  Create the directories /etc/opt/hypertable and /var/opt/hypertable on all machines in the cluster and change ownership to the user account under which the binaries will be run. For example:

$ sudo mkdir /etc/opt/hypertable /var/opt/hypertable
$ sudo chown john:staff /etc/opt/hypertable /var/opt/hypertable

Then FHS-ize the installation with the following command:

$ /opt/hypertable/$VERSION/bin/fhsize.sh

Step 3 - Set "current" link

To make the latest version of Hypertable referenceable from a well-known location, we recommend setting a "current" link to point to the latest installation.  After installation, make a symlink from /opt/hypertable/current to point to the latest installed version.

$ cd /opt/hypertable
$ ln -s $VERSION current

Step 4 - Setup data volume

Creating a data directory is only required if you did not FHS-ize your installation as described in Step 2.

To configure the Hypertable installation to write to the correct data volume, make sure the volume is formatted and mounted somewhere, for example /data.  Then create a directory on that data volume to serve as the toplevel directory for the Hypertable data files.  For example:

$ sudo mkdir -p /data/hypertable/fs

Then make sure this directory is readable and writeable by the user account from which you will be running the Hypertable service.  For example:

$ sudo chown -R john:staff /data/hypertable

Finally, create a symbolic link called "fs" inside the Hypertable installation pointing to the directory you just created on the data volume.  For example:

$ cd /opt/hypertable/current
$ ln -s /data/hypertable/fs

Step 5 - Edit Hypertable Configuration File

Open the file /opt/hypertable/current/conf/hypertable.cfg file in an editor and modify the Hyperspace.Replica.Host property to contain the name of your local machine instead of localhost.  For example:

Hyperspace.Replica.Host=yourmachine

This modification will allow other machines in your network to access this local Hypertable instance by simply copying this modified configuration file into the conf/ directory of their Hypertable installation:

othermachine$ scp yourmachine:/opt/hypertable/current/conf/hypertable.cfg /opt/hypertable/current/conf
othermachine$ /opt/hypertable/current/bin/ht shell

Step 6 - Install Notification Script

The Hypertable Master will invoke a notification script (conf/notification-hook.sh) to inform the Hypertable administrator of certain events such as machine failure or any problems that may have been encountered during machine failure recovery.  The script accepts two arguments, a subject string and a message body string.  The prefix of the subject line string can be examined to determine the type of notification, "NOTICE" indicating a notification of abnormal condition, and "ERROR" indicating a hard error that requires intervention.  The following is an example notification script (/opt/hypertable/current/conf/notification-hook.sh) that can be used to email notificaiton to a list of administrators:

#!/usr/bin/env bash

recipients="root"
subject=$1
message=$2
echo -e $message | mail -s "$subject" ${recipients}

Modify the recipients variable to contain the the list of recipients to whom notificaiton messages are to be sent.  Verify that the script works properly by testing it manually:

/opt/hypertable/current/conf/notification-hook.sh "Test Message" "This is a test."

Step 7 - Start Hypertable

Run the following command to launch Hypertable.

$ /opt/hypertable/current/bin/start-all-servers.sh local

It will generate output that looks like the following

DFS broker: available file descriptors: 256
Started DFS Broker (local)
Started Hyperspace
Started Hypertable.Master
Started Hypertable.RangeServer
Started ThriftBroker

Step 8 - Verify Installation

Create a table.

echo "USE '/'; CREATE TABLE foo ( c1, c2 ); GET LISTING;" \
    | /opt/hypertable/current/bin/ht shell --batch

The output of this command should look like:

foo
sys (namespace)
tmp (namespace)

Load some data.

echo "USE '/'; INSERT INTO foo VALUES('001', 'c1', 'very'), \
    ('000', 'c1', 'Hypertable'), ('001', 'c2', 'easy'), ('000', 'c2', 'is');" \
    | /opt/hypertable/current/bin/ht shell --batch

Dump the table.

echo "USE '/'; SELECT * FROM foo;" \
    | /opt/hypertable/current/bin/ht shell --batch

The output of this command should look like:

000	c1	Hypertable
000	c2	is
001	c1	very
001	c2	easy

Step 9 - Stop Hypertable

To stop Hypertable, run the following command:

$ /opt/hypertable/current/bin/stop-servers.sh

To wipe Hypertable clean, removing all tables, run the following command (which will stop Hypertable if it is not already stopped):

$ /opt/hypertable/current/bin/clean-database.sh

What Next?

Congratulations!  Now that you have successfully installed Hypertable, we recommend that you walk through the HQL Tutorial to get familiar with using the system.