Tools to help cluster definition file tokenization. More...

Functions
bool	is_identifier_start_character (char c)
	Checks if character is valid bash identifier start character. More...

bool	is_identifier_character (char c)
	Checks if character is valid bash identifier character. More...

bool	is_valid_identifier (const string &name)
	Checks if name is a valid bash identifier. More...

bool	is_number (const string &str)
	Checks if string is an ASCII number. More...

bool	find_token (const string &token, const char base, const char end, size_t *offsetp)
	Finds a string token in a block of code. More...

bool	find_next_token (const char base, size_t offsetp, size_t *lengthp)
	Finds next bash identifier token in a block of text. More...

bool	find_end_char (const char base, const char endp, size_t linep=nullptr)
	Skips to end of block or quoted string in code. More...

bool	skip_control_flow_statement (const char **basep)
	Skips over bash control flow statement. More...

size_t	count_newlines (const char base, const char end)
	Counts number of newlines in text. More...

bool	skip_to_newline (const char **endp)
	Skips to next newline character in text. More...

bool	substitute_variables (const string &input, string &output, map< string, string > &vmap)
	Does variable sustitution in a block of text. More...

Detailed Description

Tools to help cluster definition file tokenization.

Function Documentation

size_t Hypertable::ClusterDefinitionFile::TokenizerTools::count_newlines	(	const char *	base,
		const char *	end
	)

Counts number of newlines in text.

Counts the number of newline characters in the block of text starting at base and ending at end (not inclusive).

Parameters

base	Start of text
end	End of text (not inclusive)

Returns: Number of newline characters in text

Definition at line 262 of file TokenizerTools.cc.

bool Hypertable::ClusterDefinitionFile::TokenizerTools::find_end_char	(	const char *	base,
		const char **	endp,
		size_t *	linep = `nullptr`
	)

Skips to end of block or quoted string in code.

This function expects base to point to one of four characters ('"', '\'', '`', or '{') upon entry. Depending on the starting character, it will skip to the matching end character. If the starting character is '{' it skips over quoted strings and comments and nested blocks to find the matching block close character '}'. The parameter *endp is set to the end of the quoted string or code block, if found. If linep is not null, it is incremented for each newline character encountered.

Parameters

base	Pointer to beginning of quoted string or block
endp	Address of pointer to hold pointer to end of string or block
linep	Address of newline counter

Returns: true if end of string or block was found, false otherwise.

Definition at line 142 of file TokenizerTools.cc.

bool Hypertable::ClusterDefinitionFile::TokenizerTools::find_next_token	(	const char *	base,
		size_t *	offsetp,
		size_t *	lengthp
	)

Finds next bash identifier token in a block of text.

This function finds the first valid bash identifier token in the code block pointed to by base. It skips over comments, quoted strings, and backtick surrounded command substitution text. If the token is found, then *offsetp is set to the offset of the token relative to base and *lengthp is set to the token length.

Parameters

base	Beginning of code block to search
offsetp	Address of offset variable filled in with token offset, if found
lengthp	Address of length variable filled in with token length, if found

Returns: true if token was found, false if not

Definition at line 109 of file TokenizerTools.cc.

bool Hypertable::ClusterDefinitionFile::TokenizerTools::find_token	(	const string &	token,
		const char *	base,
		const char *	end,
		size_t *	offsetp
	)

Finds a string token in a block of code.

This function tries to locate the string token token in the block of code starting at base and ending at end. It skips over comments, quoted strings, and backtick surrounded command substitution text. If the token is found, then *offsetp is set to the offset of the token relative to base.

Parameters

token	String token to search for
base	Beginning of code block to search
end	End of code block to search
offsetp	Address of offset variable filled in with token offset, if found

Returns: true if token was found, false if not

Definition at line 80 of file TokenizerTools.cc.

bool Hypertable::ClusterDefinitionFile::TokenizerTools::is_identifier_character ( char c )

Checks if character is valid bash identifier character.

This function checks to see if the character c is a valid bash identifier character (e.g. alphanumeric or '_').

Parameters

c	Character to check

Returns: true if character is valid bash identifier character false otherwise.

Definition at line 49 of file TokenizerTools.cc.

bool Hypertable::ClusterDefinitionFile::TokenizerTools::is_identifier_start_character ( char c )

Checks if character is valid bash identifier start character.

This function checks to see if the character c is a valid bash identifier starting character (e.g. alphabetic or '_').

Parameters

c	Character to check

Returns: true if character is valid bash identifier start character, false otherwise.

Definition at line 45 of file TokenizerTools.cc.

bool Hypertable::ClusterDefinitionFile::TokenizerTools::is_number ( const string & str )

Checks if string is an ASCII number.

Checks if str is an ASCII representation of a decimal number. Every character in the string must be an ASCII digit character.

Returns: true if string represents an ASCII number, false otherwise.

Definition at line 64 of file TokenizerTools.cc.

bool Hypertable::ClusterDefinitionFile::TokenizerTools::is_valid_identifier ( const string & name )

Checks if name is a valid bash identifier.

This function calls is_identifier_start_character() on the first character of name and is_identifier_character() on all subsequent characters. If they all return true, then the function returns true, otherwise false is returned.

Parameters

name	Name to checks

Returns: true if name is a valid bash identifier, false otherwise.

Definition at line 53 of file TokenizerTools.cc.

bool Hypertable::ClusterDefinitionFile::TokenizerTools::skip_control_flow_statement ( const char ** basep )

Skips over bash control flow statement.

This method skips over a bash control flow statement pointed to by *basep. The following control flow statements are handled:

if ... fi
for ... done
until ... done
while ... done
case ... esac

The function handles nested control flow statements. If the end of the control flow statement is found, *basep is set to point to the character immediately following the control flow statement termination token.

Parameters

basep Address of pointer to beginning of control flow statement

Returns: true if end of control flow statement was found, false otherwise.

Definition at line 213 of file TokenizerTools.cc.

bool Hypertable::ClusterDefinitionFile::TokenizerTools::skip_to_newline ( const char ** endp )

Skips to next newline character in text.

This function advances *endp to the next newline character found or the terminating '\0' character if no newline character is found.

Parameters

endp	Address of text pointer (advanced by call)

Returns: true if newline character was found, false if not.

Definition at line 271 of file TokenizerTools.cc.

bool Hypertable::ClusterDefinitionFile::TokenizerTools::substitute_variables	(	const string &	input,
		string &	output,
		map< string, string > &	vmap
	)

Does variable sustitution in a block of text.

This function copies the input text input to output, performing variable substitution. The function searches the input text for strings of the form ${name} of $name and if name is found in vmap, the string is replaced with the mapped value for name in vmap. If the variable reference is escaped (i.e. $name) then it is skipped.

Parameters

input	Input text
output	Output text with variables substituted
vmap	Variable map mapping name -> sustitution text

Returns: true if any variable sustitutions were performed, false otherwise.

Definition at line 279 of file TokenizerTools.cc.

Functions

Detailed Description

Function Documentation