Tools to help cluster definition file tokenization. More...
Functions | |
bool | is_identifier_start_character (char c) |
Checks if character is valid bash identifier start character. More... | |
bool | is_identifier_character (char c) |
Checks if character is valid bash identifier character. More... | |
bool | is_valid_identifier (const string &name) |
Checks if name is a valid bash identifier. More... | |
bool | is_number (const string &str) |
Checks if string is an ASCII number. More... | |
bool | find_token (const string &token, const char *base, const char *end, size_t *offsetp) |
Finds a string token in a block of code. More... | |
bool | find_next_token (const char *base, size_t *offsetp, size_t *lengthp) |
Finds next bash identifier token in a block of text. More... | |
bool | find_end_char (const char *base, const char **endp, size_t *linep=nullptr) |
Skips to end of block or quoted string in code. More... | |
bool | skip_control_flow_statement (const char **basep) |
Skips over bash control flow statement. More... | |
size_t | count_newlines (const char *base, const char *end) |
Counts number of newlines in text. More... | |
bool | skip_to_newline (const char **endp) |
Skips to next newline character in text. More... | |
bool | substitute_variables (const string &input, string &output, map< string, string > &vmap) |
Does variable sustitution in a block of text. More... | |
Tools to help cluster definition file tokenization.
size_t Hypertable::ClusterDefinitionFile::TokenizerTools::count_newlines | ( | const char * | base, |
const char * | end | ||
) |
Counts number of newlines in text.
Counts the number of newline characters in the block of text starting at base
and ending at end
(not inclusive).
base | Start of text |
end | End of text (not inclusive) |
Definition at line 262 of file TokenizerTools.cc.
bool Hypertable::ClusterDefinitionFile::TokenizerTools::find_end_char | ( | const char * | base, |
const char ** | endp, | ||
size_t * | linep = nullptr |
||
) |
Skips to end of block or quoted string in code.
This function expects base
to point to one of four characters ('"', '\'', '`', or '{') upon entry. Depending on the starting character, it will skip to the matching end character. If the starting character is '{' it skips over quoted strings and comments and nested blocks to find the matching block close character '}'. The parameter *endp
is set to the end of the quoted string or code block, if found. If linep
is not null, it is incremented for each newline character encountered.
base | Pointer to beginning of quoted string or block |
endp | Address of pointer to hold pointer to end of string or block |
linep | Address of newline counter |
Definition at line 142 of file TokenizerTools.cc.
bool Hypertable::ClusterDefinitionFile::TokenizerTools::find_next_token | ( | const char * | base, |
size_t * | offsetp, | ||
size_t * | lengthp | ||
) |
Finds next bash identifier token in a block of text.
This function finds the first valid bash identifier token in the code block pointed to by base
. It skips over comments, quoted strings, and backtick surrounded command substitution text. If the token is found, then *offsetp
is set to the offset of the token relative to base
and *lengthp
is set to the token length.
base | Beginning of code block to search |
offsetp | Address of offset variable filled in with token offset, if found |
lengthp | Address of length variable filled in with token length, if found |
Definition at line 109 of file TokenizerTools.cc.
bool Hypertable::ClusterDefinitionFile::TokenizerTools::find_token | ( | const string & | token, |
const char * | base, | ||
const char * | end, | ||
size_t * | offsetp | ||
) |
Finds a string token in a block of code.
This function tries to locate the string token token
in the block of code starting at base
and ending at end
. It skips over comments, quoted strings, and backtick surrounded command substitution text. If the token is found, then *offsetp
is set to the offset of the token relative to base
.
token | String token to search for |
base | Beginning of code block to search |
end | End of code block to search |
offsetp | Address of offset variable filled in with token offset, if found |
Definition at line 80 of file TokenizerTools.cc.
bool Hypertable::ClusterDefinitionFile::TokenizerTools::is_identifier_character | ( | char | c | ) |
Checks if character is valid bash identifier character.
This function checks to see if the character c
is a valid bash identifier character (e.g. alphanumeric or '_').
c | Character to check |
Definition at line 49 of file TokenizerTools.cc.
bool Hypertable::ClusterDefinitionFile::TokenizerTools::is_identifier_start_character | ( | char | c | ) |
Checks if character is valid bash identifier start character.
This function checks to see if the character c
is a valid bash identifier starting character (e.g. alphabetic or '_').
c | Character to check |
Definition at line 45 of file TokenizerTools.cc.
bool Hypertable::ClusterDefinitionFile::TokenizerTools::is_number | ( | const string & | str | ) |
Checks if string is an ASCII number.
Checks if str
is an ASCII representation of a decimal number. Every character in the string must be an ASCII digit character.
Definition at line 64 of file TokenizerTools.cc.
bool Hypertable::ClusterDefinitionFile::TokenizerTools::is_valid_identifier | ( | const string & | name | ) |
Checks if name is a valid bash identifier.
This function calls is_identifier_start_character() on the first character of name
and is_identifier_character() on all subsequent characters. If they all return true, then the function returns true, otherwise false is returned.
name | Name to checks |
Definition at line 53 of file TokenizerTools.cc.
bool Hypertable::ClusterDefinitionFile::TokenizerTools::skip_control_flow_statement | ( | const char ** | basep | ) |
Skips over bash control flow statement.
This method skips over a bash control flow statement pointed to by *basep
. The following control flow statements are handled:
if ... fi for ... done until ... done while ... done case ... esac
The function handles nested control flow statements. If the end of the control flow statement is found, *basep
is set to point to the character immediately following the control flow statement termination token.
basep | Address of pointer to beginning of control flow statement |
Definition at line 213 of file TokenizerTools.cc.
bool Hypertable::ClusterDefinitionFile::TokenizerTools::skip_to_newline | ( | const char ** | endp | ) |
Skips to next newline character in text.
This function advances *endp
to the next newline character found or the terminating '\0' character if no newline character is found.
endp | Address of text pointer (advanced by call) |
Definition at line 271 of file TokenizerTools.cc.
bool Hypertable::ClusterDefinitionFile::TokenizerTools::substitute_variables | ( | const string & | input, |
string & | output, | ||
map< string, string > & | vmap | ||
) |
Does variable sustitution in a block of text.
This function copies the input text input
to output
, performing variable substitution. The function searches the input text for strings of the form ${name}
of $name
and if name
is found in vmap
, the string is replaced with the mapped value for name
in vmap
. If the variable reference is escaped (i.e. $name
) then it is skipped.
input | Input text |
output | Output text with variables substituted |
vmap | Variable map mapping name -> sustitution text |
Definition at line 279 of file TokenizerTools.cc.