Central authority for balance plans. More...

#include <BalancePlanAuthority.h>

Inheritance diagram for Hypertable::BalancePlanAuthority:

Collaboration diagram for Hypertable::BalancePlanAuthority:

Classes
struct	lt_move_spec
	StrictWeakOrdering class for set of RangeMoveSpecs. More...

struct	RecoveryPlans
	Holds plans for each range type. More...

Public Member Functions
	BalancePlanAuthority (ContextPtr context, MetaLog::WriterPtr &mml_writer)
	Constructor. More...

	BalancePlanAuthority (ContextPtr context, MetaLog::WriterPtr &mml_writer, const MetaLog::EntityHeader &header_)
	Constructor with MetaLog header. More...

virtual	~BalancePlanAuthority ()
	Destructor. More...

bool	is_empty ()
	Determines if there are any failover balance plans. More...

void	create_recovery_plan (const String &location, const vector< QualifiedRangeSpec > &root_specs, const vector< RangeState > &root_states, const vector< QualifiedRangeSpec > &metadata_specs, const vector< RangeState > &metadata_states, const vector< QualifiedRangeSpec > &system_specs, const vector< RangeState > &system_states, const vector< QualifiedRangeSpec > &user_specs, const vector< RangeState > &user_states)
	Creates a recovery plan for a failed range server. More...

void	copy_recovery_plan (const String &location, int type, RangeServerRecovery::Plan &out, int &plan_generation)
	Copies part of a recovery plan for a failed range server. More...

void	remove_recovery_plan (const String &location)
	Removes a recovery plan for a failed range server. More...

void	remove_from_receiver_plan (const String &location, int type, const vector< QualifiedRangeSpec > &ranges)
	Removes ranges from a failover plan. More...

void	remove_table_from_receiver_plan (const String &table_id)
	Removes range move specifications for a table. More...

void	change_receiver_plan_location (const String &location, int type, const String &old_destination, const String &new_destination)
	Modifies all failover plans by changing moves to a given destination to a new one. More...

void	get_receiver_plan_locations (const String &location, int type, StringSet &locations)
	Returns the list of receiver location for a recovery plan. More...

bool	recovery_complete (const String &location, int type)
	Checks if recovery plan of given type has been removed. More...

int	get_generation ()
	Returns the current generation number. More...

void	set_generation (int new_generation)
	Sets the generation number (TESTING ONLY). More...

void	clear_current_set ()
	Clears the m_current_set of move specifications (TESTING ONLY). More...

bool	register_balance_plan (BalancePlanPtr &plan, int generation, std::vector< MetaLog::EntityPtr > &entities)
	Registers a new balance plan for load balancing purposes. More...

bool	get_balance_destination (const TableIdentifier &table, const RangeSpec &range, String &location)
	Returns the balance plan destination for a given range. More...

void	balance_move_complete (const TableIdentifier &table, const RangeSpec &range)
	Signals that range move is complete. More...

void	set_mml_writer (MetaLog::WriterPtr &mml_writer)
	Sets the MML writer Sets m_mml_writer to `mml_writer` More...

const String	name () override
	Returns the name of this entity ("BalancePlanAuthority") More...

void	display (std::ostream &os) override
	Writes a human-readable represenation of object to an output stream. More...

void	decode (const uint8_t *bufp, size_t remainp, uint16_t definition_version) override
	Reads serialized encoding of object. More...

Public Member Functions inherited from Hypertable::MetaLog::Entity
	Entity (int32_t type)
	Constructor from entity type. More...

	Entity (const EntityHeader &header_)
	Constructor from entity header. More...

virtual	~Entity ()
	Destructor. More...

void	lock ()
	Locks the entity's mutex. More...

void	unlock ()
	Unlocks the entity's mutex. More...

void	mark_for_removal ()
	Marks entity for removal. More...

bool	marked_for_removal ()
	Checks if entity is marked for removal. More...

int64_t	id () const
	Returns the entity ID. More...

Public Member Functions inherited from Hypertable::Serializable
virtual size_t	encoded_length () const
	Returns serialized object length. More...

virtual void	encode (uint8_t **bufp) const
	Writes serialized representation of object to a buffer. More...

virtual void	decode (const uint8_t *bufp, size_t remainp)
	Reads serialized representation of object from a buffer. More...

Private Types
typedef std::map< String, RecoveryPlans >	RecoveryPlanMap
	Server-to-plan map. More...

typedef std::set < RangeMoveSpecPtr, lt_move_spec >	MoveSetT
	Set of RangeMoveSpecPtr. More...

Private Member Functions
uint8_t	encoding_version () const override
	Returns encoding version. More...

size_t	encoded_length_internal () const override
	Returns internal serialized length. More...

void	encode_internal (uint8_t **bufp) const override
	Writes serialized representation of object to a buffer. More...

void	decode_internal (uint8_t version, const uint8_t *bufp, size_t remainp) override
	Reads serialized representation of object from a buffer. More...

void	decode_old (const uint8_t *bufp, size_t remainp)

RangeServerRecovery::PlanPtr	create_range_plan (const String &location, int type, const vector< QualifiedRangeSpec > &specs, const vector< RangeState > &states)
	Creates a recovery plan for a failed server. More...

void	update_range_plan (RangeServerRecovery::PlanPtr &plan, const String &location, const vector< QualifiedRangeSpec > &new_specs)
	Modifies recovery plan, replacing moves to `location` with a new destination. More...

Private Attributes
ContextPtr	m_context
	Pointer to master context. More...

MetaLog::WriterPtr	m_mml_writer
	Pointer to MML writer. More...

int	m_generation
	Generation number (incremented with each new failover plan) More...

StringSet	m_active
	Cache of active (available) servers. More...

StringSet::iterator	m_active_iter
	Iterator pointing into m_active. More...

RecoveryPlanMap	m_map
	Mapping from failed range server to recovery plan. More...

MoveSetT	m_current_set
	Current set of move specifications for move operations. More...

Additional Inherited Members
Protected Member Functions inherited from Hypertable::MetaLog::Entity
void	encode_entry (uint8_t **bufp)
	Encodes entity header plus serialized state. More...

Protected Attributes inherited from Hypertable::MetaLog::Entity
std::mutex	m_mutex
	Mutex for serializing access to members More...

EntityHeader	header
	Entity header More...

Detailed Description

Central authority for balance plans.

This class maintains all currently active balance plans. A balance plan is a specification for the movement of a set of ranges from their current location to a new location. Balance plans are created in the following circumstances:

Split - When a range splits, a new range is created and is moved to a new range server. A plan is created for the range, indicating where it is to be moved to.
Load balancing - A balance plan is created by the load balancer to record the source and destination locations of all moves in the balance operation.
Failover - When a range server fails, all of its ranges are moved to new locations. A balance plan is created in this situation to record where the ranges are to be moved.

There is one object of this class maintained by the Master and it is persisted to the MML each time its state changes.

Definition at line 67 of file BalancePlanAuthority.h.

Member Typedef Documentation

typedef std::set<RangeMoveSpecPtr, lt_move_spec> Hypertable::BalancePlanAuthority::MoveSetT

private

Set of RangeMoveSpecPtr.

Definition at line 392 of file BalancePlanAuthority.h.

typedef std::map<String, RecoveryPlans> Hypertable::BalancePlanAuthority::RecoveryPlanMap

private

Server-to-plan map.

Definition at line 369 of file BalancePlanAuthority.h.

Constructor & Destructor Documentation

BalancePlanAuthority::BalancePlanAuthority	(	ContextPtr	context,
		MetaLog::WriterPtr &	mml_writer
	)

Constructor.

This constructor constructs an empty object. Persists the object state in the MML.

Parameters

context	Master context
mml_writer	MML writer for persisting state changes

Definition at line 45 of file BalancePlanAuthority.cc.

BalancePlanAuthority::BalancePlanAuthority	(	ContextPtr	context,
		MetaLog::WriterPtr &	mml_writer,
		const MetaLog::EntityHeader &	header_
	)

Constructor with MetaLog header.

This constructor constructs an empty object to be populated by an MML entity.

Parameters

context	Master context
mml_writer	MML writer for persisting state changes
header_	MML entity header

Definition at line 53 of file BalancePlanAuthority.cc.

virtual Hypertable::BalancePlanAuthority::~BalancePlanAuthority ( )

inlinevirtual

Destructor.

Definition at line 89 of file BalancePlanAuthority.h.

Member Function Documentation

void BalancePlanAuthority::balance_move_complete	(	const TableIdentifier &	table,
		const RangeSpec &	range
	)

Signals that range move is complete.

This method removes the range identified by table and range from m_current_set.

Parameters

table	Table identifer of range
range	RangeSpec identifying range

Definition at line 597 of file BalancePlanAuthority.cc.

void BalancePlanAuthority::change_receiver_plan_location	(	const String &	location,
		int	type,
		const String &	old_destination,
		const String &	new_destination
	)

Modifies all failover plans by changing moves to a given destination to a new one.

Parameters

location	Proxy name of server identifying recovery plan ("*" implies all recovery plans)
type	Type of failover plan to modify (see RangeSpec::Type)
old_destination	Existing destination to replace
new_destination	New destination to replace `old_destination`

Definition at line 171 of file BalancePlanAuthority.cc.

void Hypertable::BalancePlanAuthority::clear_current_set ( )

inline

Clears the m_current_set of move specifications (TESTING ONLY).

Definition at line 219 of file BalancePlanAuthority.h.

void BalancePlanAuthority::copy_recovery_plan	(	const String &	location,
		int	type,
		RangeServerRecovery::Plan &	out,
		int &	plan_generation
	)

Copies part of a recovery plan for a failed range server.

Parameters

location	Proxy name of failed server
type	Range type indicating portion of recovery plan to copy (see RangeSpec::Type).
out	Out parameter to hold copy of recovery plan
plan_generation	Out parameter filled in with current generation number

Definition at line 97 of file BalancePlanAuthority.cc.

RangeServerRecovery::PlanPtr BalancePlanAuthority::create_range_plan	(	const String &	location,
		int	type,
		const vector< QualifiedRangeSpec > &	specs,
		const vector< RangeState > &	states
	)

private

Creates a recovery plan for a failed server.

Creates a new recovery plan for a failed server for the ranges of type type. It allocates a RangeServerRecovery::Plan object and adds specs and states to the receiver plan and assigns destinations by walking m_active in round-robin fashion. It also populates the replay plan by reading the list of fragments of the failed servers commit log

/hypertable/servers/location/log/type

and assigns each fragment to a server by walking m_active in round-robin fashion.

Parameters

location	Proxy name of failed server
type	Type of recovery plan (see RangeSpec::Type)
specs	Vector of ranges to be moved
states	Vector of range states (parallels `specs`)

Returns: Recovery plan

Definition at line 453 of file BalancePlanAuthority.cc.

void BalancePlanAuthority::create_recovery_plan	(	const String &	location,
		const vector< QualifiedRangeSpec > &	root_specs,
		const vector< RangeState > &	root_states,
		const vector< QualifiedRangeSpec > &	metadata_specs,
		const vector< RangeState > &	metadata_states,
		const vector< QualifiedRangeSpec > &	system_specs,
		const vector< RangeState > &	system_states,
		const vector< QualifiedRangeSpec > &	user_specs,
		const vector< RangeState > &	user_states
	)

Creates a recovery plan for a failed range server.

When a range server fails, all of its ranges are reassigned to new range servers. Recovery of ranges needs to happen in the following order:

ROOT
METADATA
SYSTEM
USER

This method is called by OperationRecover to create and register a balance plan for a failed range server. This method sets the "removed" bit on the RangeServerConnection object associated with the failed server and persists the modified RangeServerConnection object and this object's state as a single write to the MML.

Parameters

location	Proxy name for failed range server
root_specs	Vector holding ROOT range spec
root_states	Vector holding ROOT range state
metadata_specs	Vector holding METADATA range specs
metadata_states	Vector holding METADATA range states
system_specs	Vector holding SYSTEM range specs
system_states	Vector holding SYSTEM range states
user_specs	Vector holding USER range specs
user_states	Vector holding USER range states

Put the RangeServerConnection object into the 'removed' state and then atomically persist both the RangeServerConnection object and the plan to the MML

Definition at line 342 of file BalancePlanAuthority.cc.

void BalancePlanAuthority::decode	(	const uint8_t **	bufp,
		size_t *	remainp,
		uint16_t	definition_version
	)

overridevirtual

Reads serialized encoding of object.

This method restores the state of the object by decoding a serialized representation of the state from the memory location pointed to by *bufp. See encode() for a description of the serialized format.

Parameters

bufp	Address of source buffer pointer (advanced by call)
remainp	Amount of remaining buffer pointed to by `*bufp` (decremented by call).
definition_version	Version of DefinitionMaster

Reimplemented from Hypertable::MetaLog::Entity.

Definition at line 79 of file BalancePlanAuthority.cc.

void BalancePlanAuthority::decode_internal	(	uint8_t	version,
		const uint8_t **	bufp,
		size_t *	remainp
	)

overrideprivatevirtual

Reads serialized representation of object from a buffer.

This function is to be overridden by derived classes and should decode the object per-se as encoded with encode_internal().

Parameters

version	Encoding version
bufp	Address of destination buffer pointer (advanced by call)
remainp	Address of integer holding amount of serialized encoding remaining

See also: encode_internal() for encoding format

Implements Hypertable::Serializable.

Definition at line 282 of file BalancePlanAuthority.cc.

void BalancePlanAuthority::decode_old	(	const uint8_t **	bufp,
		size_t *	remainp
	)

private

Definition at line 311 of file BalancePlanAuthority.cc.

void BalancePlanAuthority::display ( std::ostream & os )

overridevirtual

Writes a human-readable represenation of object to an output stream.

This method writes a human-readable represenation of the object to the output stream os. The string written starts with the generation number and is followed by all of the failover plans.

Parameters

os	Output stream to write to

Reimplemented from Hypertable::MetaLog::Entity.

Definition at line 61 of file BalancePlanAuthority.cc.

void BalancePlanAuthority::encode_internal ( uint8_t ** bufp ) const

overrideprivatevirtual

Writes serialized representation of object to a buffer.

This function is to be overridden by derived classes and should encode the object per-se.

Parameters

bufp	Address of destination buffer pointer (advanced by call)

Implements Hypertable::Serializable.

Definition at line 261 of file BalancePlanAuthority.cc.

size_t BalancePlanAuthority::encoded_length_internal ( ) const

overrideprivatevirtual

Returns internal serialized length.

This function is to be overridden by derived classes and should return the length of the the serialized object per-se.

Returns: Internal serialized length

See also: encode_internal() for encoding format

Implements Hypertable::Serializable.

Definition at line 243 of file BalancePlanAuthority.cc.

uint8_t BalancePlanAuthority::encoding_version ( ) const

overrideprivatevirtual

Returns encoding version.

Returns: Encoding version

Implements Hypertable::Serializable.

Definition at line 239 of file BalancePlanAuthority.cc.

bool BalancePlanAuthority::get_balance_destination	(	const TableIdentifier &	table,
		const RangeSpec &	range,
		String &	location
	)

Returns the balance plan destination for a given range.

This method looks up the range identified by table and range in m_current_set and returns the destination. If the range does not exist in m_current_set, a new destination is chosen and a new entry is added to m_current_set. If m_current_set was modified, the object state is persisted to the MML

Parameters

table	Table identifier of range to look up
range	RangeSpec of range to lookup
location	Output parameter to hold destination

Returns: false if destination not already assigned and there are no available servers to assign it to, otherwise true

Definition at line 566 of file BalancePlanAuthority.cc.

int Hypertable::BalancePlanAuthority::get_generation ( )

inline

Returns the current generation number.

The generation number (m_generation) starts out at zero and is incremented each time a new plan is created for a failed range server.

Returns: Current generation number.

Definition at line 207 of file BalancePlanAuthority.h.

void BalancePlanAuthority::get_receiver_plan_locations	(	const String &	location,
		int	type,
		StringSet &	locations
	)

Returns the list of receiver location for a recovery plan.

Parameters

location	Proxy name of server identifying recovery plan
type	Type of recovery plan (see RangeSpec::Type)
locations	Output parameter filled in with recceiver locations

Definition at line 212 of file BalancePlanAuthority.cc.

bool BalancePlanAuthority::is_empty ( )

Determines if there are any failover balance plans.

This method determines if there are any registered balance plans due to a range server failure. No failover plans are registered is m_map is empty

Returns: true if balance plans registered, false otherwise

Definition at line 234 of file BalancePlanAuthority.cc.

const String Hypertable::BalancePlanAuthority::name ( )

inlineoverridevirtual

Returns the name of this entity ("BalancePlanAuthority")

Returns: Entity name

Implements Hypertable::MetaLog::Entity.

Definition at line 273 of file BalancePlanAuthority.h.

bool BalancePlanAuthority::recovery_complete	(	const String &	location,
		int	type
	)

Checks if recovery plan of given type has been removed.

Parameters

location	Proxy name of server identifying recovery plan
type	Type of recovery plan (see RangeSpec::Type)

Returns: true if recovery plan for location, type has been removed, false otherwise.

Definition at line 223 of file BalancePlanAuthority.cc.

bool BalancePlanAuthority::register_balance_plan	(	BalancePlanPtr &	plan,
		int	generation,
		std::vector< MetaLog::EntityPtr > &	entities
	)

Registers a new balance plan for load balancing purposes.

This method registers the balance plan plan by adding all of the moves in planto m_current_set. It only registers the plan if generation matches the current generation number. After successfully registering the balance plan, the object state is persisted to the MML along with the entities supplied in entities.

Parameters

plan	Balance plan to register
generation	Required generation number
entities	Additional entities to persist in MML.

Returns: true if balance plan was registered, false if balance plan was not registered due to generation mismatch.

Definition at line 541 of file BalancePlanAuthority.cc.

void BalancePlanAuthority::remove_from_receiver_plan	(	const String &	location,
		int	type,
		const vector< QualifiedRangeSpec > &	ranges
	)

Removes ranges from a failover plan.

This method removes the set of ranges specified by ranges from the failover plan for server location. Once the ranges have been moved, the new state is persisted to the MML.

Parameters

location	Proxy name identifying failover plan
type	Type of failover plan (see RangeSpec::Type)
ranges	Vector of ranges to remove from plan.

Definition at line 129 of file BalancePlanAuthority.cc.

void BalancePlanAuthority::remove_recovery_plan ( const String & location )

Removes a recovery plan for a failed range server.

This method removes the recovery plan associated with server location and persists the new state in the MML. This method is called after recovery is finished for location.

Parameters

location Proxy name of plan to remove

Definition at line 116 of file BalancePlanAuthority.cc.

void BalancePlanAuthority::remove_table_from_receiver_plan ( const String & table_id )

Removes range move specifications for a table.

This method removes all of the move specifications for the table identified by table_id from all of the failover plans.

Parameters

table_id Table identifier of move specs to remove

Definition at line 146 of file BalancePlanAuthority.cc.

void Hypertable::BalancePlanAuthority::set_generation ( int new_generation )

inline

Sets the generation number (TESTING ONLY).

Parameters

new_generation New generation number

Definition at line 212 of file BalancePlanAuthority.h.

void BalancePlanAuthority::set_mml_writer ( MetaLog::WriterPtr & mml_writer )

Sets the MML writer Sets m_mml_writer to mml_writer

Parameters

mml_writer Smart pointer to MML writer

Definition at line 90 of file BalancePlanAuthority.cc.

void BalancePlanAuthority::update_range_plan	(	RangeServerRecovery::PlanPtr &	plan,
		const String &	location,
		const vector< QualifiedRangeSpec > &	new_specs
	)

private

Modifies recovery plan, replacing moves to location with a new destination.

This method modifies the given recovery plan by replacing references to location with other active servers. It is called by create_recovery_plan() to replace the failed server identified by location from existing recovery plans.

Parameters

plan	Recovery plan to modify
location	Proxy name of server being recovered
new_specs	Vector of QualifiedRangeSpec objects to remove from receiver plan

Definition at line 499 of file BalancePlanAuthority.cc.

Member Data Documentation

StringSet Hypertable::BalancePlanAuthority::m_active

private

Cache of active (available) servers.

Definition at line 355 of file BalancePlanAuthority.h.

StringSet::iterator Hypertable::BalancePlanAuthority::m_active_iter

private

Iterator pointing into m_active.

Definition at line 358 of file BalancePlanAuthority.h.

ContextPtr Hypertable::BalancePlanAuthority::m_context

private

Pointer to master context.

Definition at line 346 of file BalancePlanAuthority.h.

MoveSetT Hypertable::BalancePlanAuthority::m_current_set

private

Current set of move specifications for move operations.

Definition at line 395 of file BalancePlanAuthority.h.

int Hypertable::BalancePlanAuthority::m_generation

private

Generation number (incremented with each new failover plan)

Definition at line 352 of file BalancePlanAuthority.h.

RecoveryPlanMap Hypertable::BalancePlanAuthority::m_map

private

Mapping from failed range server to recovery plan.

Definition at line 372 of file BalancePlanAuthority.h.

MetaLog::WriterPtr Hypertable::BalancePlanAuthority::m_mml_writer

private

Pointer to MML writer.

Definition at line 349 of file BalancePlanAuthority.h.

The documentation for this class was generated from the following files:

/home/doug/src/hypertable/src/cc/Hypertable/Master/BalancePlanAuthority.h
/home/doug/src/hypertable/src/cc/Hypertable/Master/BalancePlanAuthority.cc

Classes

Public Member Functions

Private Types

Private Member Functions

Private Attributes

Additional Inherited Members

Detailed Description

Member Typedef Documentation

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation