Manages live range map and set of log names that can be safely removed. More...

#include <TableInfoMap.h>

Collaboration diagram for Hypertable::TableInfoMap:

Public Member Functions
	TableInfoMap ()
	Constructor. More...

	TableInfoMap (HyperspaceTableCachePtr schema_cache)
	Constructor with HyperspaceTableCache. More...

virtual	~TableInfoMap ()
	Destructor. More...

bool	lookup (const String &table_id, TableInfoPtr &info)
	Returns the TableInfo object for a given table. More...

void	get (const String &table_id, TableInfoPtr &info)
	Gets the TableInfo object for a table, creating one if not found. More...

void	promote_staged_range (const TableIdentifier &table, RangePtr &range, const char *transfer_log)
	Adds a staged range. More...

bool	remove (const String &table_id, TableInfoPtr &info)
	Removes a table from the map. More...

void	get_all (std::vector< TableInfoPtr > &tv)
	Gets all TableInfo objects in map. More...

void	get_ranges (Ranges &ranges, StringSet *remove_ok_logs=0)
	Gets set of live RangeData objects and corresponding transfer logs that can be safely removed. More...

void	clear ()
	Clears the map. More...

bool	empty ()
	Determines if map is empty. More...

void	merge (TableInfoMap *other)
	Merges in another map. More...

void	merge (TableInfoMap *other, vector< MetaLog::EntityPtr > &entities, StringSet &transfer_logs)
	Merges in another map and updates RMSL with merged entities. More...

Private Types
typedef std::map< String, TableInfoPtr >	InfoMap
	table_id-to-TableInfoPtr map type More...

Private Member Functions
void	merge_unlocked (TableInfoMap *other)
	Merges in another map (without locking mutex). More...

Private Attributes
std::mutex	m_mutex
	Mutex for serializing access to members More...

HyperspaceTableCachePtr	m_schema_cache
	Hyperspace table cache More...

InfoMap	m_map
	table_id-to-TableInfoPtr map More...

Detailed Description

Manages live range map and set of log names that can be safely removed.

This class is used to maintain an active set of ranges, organized by table ID. It maps table IDs to TableInfo objects holding a set of active Range objects. It also manages a set of transfer log names that can be safely removed from the system. The maintenance scheduler is responsible for removing commit log fragments (including linked transfer logs). It does this by obtaining the set of live ranges, computing some statistics for each range and then removing the log fragments that contain no data that hasn't been compacted. However, loading a range is a two step process that introduces a race condition: 1) the range's transfer log is linked into the commit log and 2) the range is persisted to the RSML. If the maintenance scheduler does its commit log purging work between steps #1 and #2, it' possible for a newly added transfer log to get removed before the Range has been fully loaded. To avoid this race condition we've introduced MetaLogEntityRemoveOkLogs which contains the set of transfer logs that have been linked in and which may be safely removed. The MetaLogEntityRemoveOkLogs entity is modified and persisted atomically to the RSML with new Range entities by the promote_staged_range() method. The get_ranges() method returns a consistent snapshot of the current set of live ranges and the set of transfer logs in MetaLogEntityRemoveOkLogs.

Definition at line 67 of file TableInfoMap.h.

Member Typedef Documentation

typedef std::map<String, TableInfoPtr> Hypertable::TableInfoMap::InfoMap

private

table_id-to-TableInfoPtr map type

Definition at line 210 of file TableInfoMap.h.

Constructor & Destructor Documentation

Hypertable::TableInfoMap::TableInfoMap ( )

inline

Constructor.

Definition at line 71 of file TableInfoMap.h.

Hypertable::TableInfoMap::TableInfoMap ( HyperspaceTableCachePtr schema_cache )

inline

Constructor with HyperspaceTableCache.

Objects constructed with this constructor will use the provided schema cache to lookup schemas instead of reading them from Hyperspace.

Parameters

schema_cache Table schema cache

Definition at line 79 of file TableInfoMap.h.

TableInfoMap::~TableInfoMap ( )

virtual

Destructor.

Definition at line 42 of file TableInfoMap.cc.

Member Function Documentation

void TableInfoMap::clear ( )

Clears the map.

Definition at line 191 of file TableInfoMap.cc.

bool TableInfoMap::empty ( )

Determines if map is empty.

Returns: true if map is empty, false otherwise

Definition at line 196 of file TableInfoMap.cc.

void TableInfoMap::get	(	const String &	table_id,
		TableInfoPtr &	info
	)

Gets the TableInfo object for a table, creating one if not found.

Parameters

table_id	Table identifier string
info	Output parameter to hold TableInfo object

Exceptions

Exception with code set to Error::RANGESERVER_SCHEMA_PARSE_ERROR

Definition at line 56 of file TableInfoMap.cc.

void TableInfoMap::get_all ( std::vector< TableInfoPtr > & tv )

Gets all TableInfo objects in map.

Parameters

tv	Output vector to hold TableInfo objects

Definition at line 163 of file TableInfoMap.cc.

void TableInfoMap::get_ranges	(	Ranges &	ranges,
		StringSet *	remove_ok_logs = `0`
	)

Gets set of live RangeData objects and corresponding transfer logs that can be safely removed.

This method is used to fetch all of the live range objects in the map. The maintenance scheduler is responsible for removing commit log fragments (including linked transfer logs). It does this by obtaining the set of live ranges, computing some statistics for each range and then removing the log fragments that contain no data that hasn't been compacted. However, loading a range is a two step process that introduces a race condition: 1) the range's transfer log is linked into the commit log and 2) the range is persisted to the RSML. If the maintenance scheduler does its commit log purging work between steps #1 and #2, it' possible for a newly added transfer log to get removed before the Range has been fully loaded. To avoid this race condition we've introduced MetaLogEntityRemoveOkLogs which contains the set of transfer logs that have been linked in and which may be safely removed. The MetaLogEntityRemoveOkLogs entity is modified and persisted atomically to the RSML with new Range entities by the promote_staged_range() method. This method returns a consistent snapshot of the current set of live ranges and the set of transfer logs in MetaLogEntityRemoveOkLogs.

Parameters

ranges	Output parameter to hold RangeData objects
remove_ok_logs	Pointer to string set to hold logs that can be removed

See also: promote_staged_range

Definition at line 170 of file TableInfoMap.cc.

bool TableInfoMap::lookup	(	const String &	table_id,
		TableInfoPtr &	info
	)

Returns the TableInfo object for a given table.

Parameters

table_id	Table identifier string
info	Output parameter to hold TableInfo object

Returns: true if mapping found for table_id, false otherwise.

Definition at line 47 of file TableInfoMap.cc.

void TableInfoMap::merge ( TableInfoMap * other )

Merges in another map.

This method locks m_mutex and calls merge_unlocked(). It should only be called for ranges that have already been persisted into the RSML from a previous call to promote_staged_range() such as during the loading of ranges from the RSML at system startup.

Parameters

other Map to merge in

Definition at line 202 of file TableInfoMap.cc.

void TableInfoMap::merge	(	TableInfoMap *	other,
		vector< MetaLog::EntityPtr > &	entities,
		StringSet &	transfer_logs
	)

Merges in another map and updates RMSL with merged entities.

This method atomically adds transfer_logs to Global::remove_ok_logs, persists entities and Global::remove_ok_logs to the RSML, and merges in another map. entities and transfer_logs should correspond to the ranges in other that are to be merged in. This method performs these steps atomically (with m_mutex locked) to avoid the race condition described in get_ranges()

Parameters

other	Map to merge in
entities	Vector of range entities
transfer_logs	Set of transfer logs

See also: get_ranges

Definition at line 207 of file TableInfoMap.cc.

void TableInfoMap::merge_unlocked ( TableInfoMap * other )

private

Merges in another map (without locking mutex).

For each TableInfo object in other, if there is no corresponding TableInfo object in this object's map, the TableInfo object is directly inserted into this object's map (m_map). If there is already a TableInfo object in this object's map, then all of the RangeData objects are fetched from the other TableInfo object and inserted into this object's TableInfo object. At the end of the method other is cleared.

Parameters

other Map to merge in

Definition at line 221 of file TableInfoMap.cc.

void TableInfoMap::promote_staged_range	(	const TableIdentifier &	table,
		RangePtr &	range,
		const char *	transfer_log
	)

Adds a staged range.

When staging is complete for a range, the Range object is added with a call to this method. Since a range's transfer log cannot be removed until the range has been added, this method adds the range and adds the range's tranfer log to the the global MetaLogEntityRemoveOkLogs object (Global::remove_ok_logs) in one atomic transaction. This method performs the following actions with m_mutex locked:

Replays transfer_log with a call to Range::replay_transfer_log
Links transfer_log into the appropriate commit log
Adds transfer_log to Global::remove_ok_logs
Persists Global::remove_ok_logs and the range's metalog entity to the RSML
Calls TableInfo::promote_staged_range() to make the range object live

Parameters

table	Table identifier
range	Range object
transfer_log	Transfer log for `range`

See also: stage_range

Definition at line 102 of file TableInfoMap.cc.

bool TableInfoMap::remove	(	const String &	table_id,
		TableInfoPtr &	info
	)

Removes a table from the map.

Parameters

table_id	Table identifier string
info	Output parameter to hold removed TableInfo object

Returns: true if table exists in map and is removed, false otherwise.

Definition at line 152 of file TableInfoMap.cc.

Member Data Documentation

InfoMap Hypertable::TableInfoMap::m_map

private

table_id-to-TableInfoPtr map

Definition at line 219 of file TableInfoMap.h.

std::mutex Hypertable::TableInfoMap::m_mutex

private

Mutex for serializing access to members

Definition at line 213 of file TableInfoMap.h.

HyperspaceTableCachePtr Hypertable::TableInfoMap::m_schema_cache

private

Hyperspace table cache

Definition at line 216 of file TableInfoMap.h.

The documentation for this class was generated from the following files:

/home/doug/src/hypertable/src/cc/Hypertable/RangeServer/TableInfoMap.h
/home/doug/src/hypertable/src/cc/Hypertable/RangeServer/TableInfoMap.cc

Public Member Functions

Private Types

Private Member Functions

Private Attributes

Detailed Description

Member Typedef Documentation

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation