htck

The htck program is used to check the integrity (metadata consistency) of a Hypertable database. If an integrity problem is detected, the tool can also be used to repair it.

Preparing to Run

You must perform the following steps prior to running htck:

Stop all client activity (esp. insert activity).
Run ht wait-until-quiesced to wait for Hypertable to get into an idle state. For example:
```
$ ht wait-until-quiesced
16d27cbad8dd528dd6745fef491dfb4f metadata.tsv
16d27cbad8dd528dd6745fef491dfb4f metadata.tsv
```
As a side-effect, the ht wait-until-quiesced command will generate a metadata.tsv file in the current working directory that contains a dump of the sys/METADATA table after the system enters the idle state. This file will be used as input to the htck program.
Stop Hypertable
```
$ ht cluster stop
```

Once Hypertable has stopped completely, the next step is to restart Hyperspace and start an FS broker on the machine from which you plan to run htck. For example:

$ ht cluster start_hyperspace
$ ht start-fsbroker hadoop  # or local, ...

Checking Integrity

To check for integrity problems, run htck as follows, using the metadata.tsv file that was generated by the ht wait-until-quiesced command.

$ ht htck check metadata.tsv

If htck detects no integrity problems, it will generate output that looks something like the following (entry counts may differ):

METADATA entries: 100977
RSML range entities: 100977
RSML dangling entities: 0
METADATA overlaps:  0
METADATA gaps:  0
METADATA incomplete:  0
RSML dangling fix-ups: 0
METADATA shrunk ranges:  0
Unassigned locations: 0
Unassigned start row: 0
Unassigned entities: 0
Load acknowledge fix-ups: 0

If htck does detect an integrity problem, it will produce output that describes the problems that it encountered. For example:

METADATA entries: 100976
RSML range entities: 100977
RSML dangling entities: 1
Error: GAP in metadata range found
  last end row:  2/81:ur7H6vYc7tekOLBCte+aPsGA==_388659_2097371132442
  next start row: 2/81:ur7HdsMGe7Z/wApbmVezq28g_502360009_2604183617513
METADATA overlaps:  0
METADATA gaps:  1
METADATA incomplete:  0
RSML dangling fix-ups: 0
METADATA shrunk ranges:  0
RSML dangling entries:
  2/81[ur7H6vYc7tekOLBCte+aPsGA==_388659_2097371132442..ur7HdsMGe7Z/wApbmVezq28g_502360009_2604183617513] at rs211
Unassigned locations: 0
Unassigned start row: 0
Unassigned entities: 0
Load acknowledge fix-ups: 0

Repairing Integrity

To repair integrity problems that were discovered with htck check, you can run htck again, but replace the word check with repair. For example:

$ ht htck repair metadata.tsv

The above command will generate a file metadata.repair.tsv in the current working directory. This file will need to be loaded via LOAD DATA INFILE as soon as the system is brought back up. For example:

$ ht cluster start

After a few seconds, the system should have loaded the sys/METADATA ranges, at which point you can load the metadata.repair.tsv file (e.g. in another terminal window):

$ echo "USE sys; LOAD DATA INFILE 'metadata.repair.tsv' INTO TABLE METADATA;" | ht shell --batch

At this point, once the system comes up fully, it should be in a consistent state. To double-check, rerun the ht wait-until-quiesced command followed by htck check and verify that no integrity problems are detected.