0.9.8.10
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
Classes | Public Types | Public Member Functions | Static Public Member Functions | List of all members
Hypertable::Filesystem Class Referenceabstract

Abstract base class for a filesystem. More...

#include <Filesystem.h>

Inheritance diagram for Hypertable::Filesystem:
Inheritance graph
[legend]

Classes

class  Dirent
 Directory entry. More...
 

Public Types

enum  Flags : uint8_t { Flags::NONE =0, Flags::FLUSH =1, Flags::SYNC =2 }
 Enumeration type for append flags. More...
 
enum  OpenFlags { OPEN_FLAG_DIRECTIO = 0x00000001, OPEN_FLAG_OVERWRITE = 0x00000002, OPEN_FLAG_VERIFY_CHECKSUM = 0x00000004 }
 

Public Member Functions

virtual ~Filesystem ()
 
virtual void open (const String &name, uint32_t flags, DispatchHandler *handler)=0
 Opens a file asynchronously. More...
 
virtual int open (const String &name, uint32_t flags)=0
 Opens a file. More...
 
virtual int open_buffered (const String &name, uint32_t flags, uint32_t buf_size, uint32_t outstanding, uint64_t start_offset=0, uint64_t end_offset=0)=0
 Opens a file in buffered (readahead) mode. More...
 
virtual void decode_response_open (EventPtr &event, int32_t *fd)=0
 Decodes the response from an open request. More...
 
virtual void create (const String &name, uint32_t flags, int32_t bufsz, int32_t replication, int64_t blksz, DispatchHandler *handler)=0
 Creates a file asynchronously. More...
 
virtual int create (const String &name, uint32_t flags, int32_t bufsz, int32_t replication, int64_t blksz)=0
 Creates a file. More...
 
virtual void decode_response_create (EventPtr &event, int32_t *fd)=0
 Decodes the response from a create request. More...
 
virtual void close (int fd, DispatchHandler *handler)=0
 Closes a file asynchronously. More...
 
virtual void close (int fd)=0
 Closes a file. More...
 
virtual void read (int fd, size_t len, DispatchHandler *handler)=0
 Reads data from a file at the current position asynchronously. More...
 
virtual size_t read (int fd, void *dst, size_t len)=0
 Reads data from a file at the current position. More...
 
virtual void decode_response_read (EventPtr &event, const void **buffer, uint64_t *offset, uint32_t *length)=0
 Decodes the response from a read request. More...
 
virtual void append (int fd, StaticBuffer &buffer, Flags flags, DispatchHandler *handler)=0
 Appends data to a file asynchronously. More...
 
virtual size_t append (int fd, StaticBuffer &buffer, Flags flags=Flags::NONE)=0
 Appends data to a file. More...
 
virtual void decode_response_append (EventPtr &event, uint64_t *offset, uint32_t *length)=0
 Decodes the response from an append request. More...
 
virtual void seek (int fd, uint64_t offset, DispatchHandler *handler)=0
 Seeks current file position asynchronously. More...
 
virtual void seek (int fd, uint64_t offset)=0
 Seeks current file position. More...
 
virtual void remove (const String &name, DispatchHandler *handler)=0
 Removes a file asynchronously. More...
 
virtual void remove (const String &name, bool force=true)=0
 Removes a file. More...
 
virtual void length (const String &name, bool accurate, DispatchHandler *handler)=0
 Gets the length of a file asynchronously. More...
 
virtual int64_t length (const String &name, bool accurate=true)=0
 Gets the length of a file. More...
 
virtual int64_t decode_response_length (EventPtr &event)=0
 Decodes the response from a length request. More...
 
virtual void pread (int fd, size_t amount, uint64_t offset, bool verify_checksum, DispatchHandler *handler)=0
 Reads data from a file at the specified position asynchronously. More...
 
virtual size_t pread (int fd, void *dst, size_t len, uint64_t offset, bool verify_checksum=true)=0
 Reads data from a file at the specified position. More...
 
virtual void decode_response_pread (EventPtr &event, const void **buffer, uint64_t *offset, uint32_t *length)=0
 Decodes the response from a pread request. More...
 
virtual void mkdirs (const String &name, DispatchHandler *handler)=0
 Creates a directory asynchronously. More...
 
virtual void mkdirs (const String &name)=0
 Creates a directory. More...
 
virtual void rmdir (const String &name, DispatchHandler *handler)=0
 Recursively removes a directory asynchronously. More...
 
virtual void rmdir (const String &name, bool force=true)=0
 Recursively removes a directory. More...
 
virtual void readdir (const String &name, DispatchHandler *handler)=0
 Obtains a listing of all files in a directory asynchronously. More...
 
virtual void readdir (const String &name, std::vector< Dirent > &listing)=0
 Obtains a listing of all files in a directory. More...
 
virtual void decode_response_readdir (EventPtr &event, std::vector< Dirent > &listing)=0
 Decodes the response from a readdir request. More...
 
virtual void flush (int fd, DispatchHandler *handler)=0
 Flushes a file asynchronously. More...
 
virtual void flush (int fd)=0
 Flushes a file. More...
 
virtual void sync (int fd)=0
 Syncs a file. More...
 
virtual void exists (const String &name, DispatchHandler *handler)=0
 Determines if a file exists asynchronously. More...
 
virtual bool exists (const String &name)=0
 Determines if a file exists. More...
 
virtual bool decode_response_exists (EventPtr &event)=0
 Decodes the response from an exists request. More...
 
virtual void rename (const String &src, const String &dst, DispatchHandler *handler)=0
 Rename a path asynchronously. More...
 
virtual void rename (const String &src, const String &dst)=0
 Rename a path. More...
 
virtual void status (Status &status, Timer *timer=0)=0
 Check status of filesystem. More...
 
virtual void decode_response_status (EventPtr &event, Status &status)=0
 Decodes the response from an status request. More...
 
virtual void debug (int32_t command, StaticBuffer &serialized_parameters)=0
 Invokes debug request asynchronously. More...
 
virtual void debug (int32_t command, StaticBuffer &serialized_parameters, DispatchHandler *handler)=0
 Invokes debug request. More...
 

Static Public Member Functions

static int decode_response (EventPtr &event)
 Decodes the response from an request that only returns an error code. More...
 
static String dirname (String name, char separator= '/')
 A posix-compliant dirname() which strips the last component from a file name. More...
 
static String basename (String name, char separator= '/')
 A posix-compliant basename() which strips directory names from a filename. More...
 

Detailed Description

Abstract base class for a filesystem.

All commands have synchronous and asynchronous versions. Commands that operate on the same file descriptor are serialized by the underlying filesystem. In other words, if you issue three asynchronous commands, they will get carried out and their responses will come back in the same order in which they were issued. Unless other- wise mentioned, the methods could throw Exception.

This abstract base class is overwritten by the various FsBrokers.

Definition at line 72 of file Filesystem.h.

Constructor & Destructor Documentation

virtual Hypertable::Filesystem::~Filesystem ( )
inlinevirtual

Definition at line 117 of file Filesystem.h.

Member Function Documentation

virtual void Hypertable::Filesystem::append ( int  fd,
StaticBuffer buffer,
Flags  flags,
DispatchHandler handler 
)
pure virtual

Appends data to a file asynchronously.

Issues an append request. The caller will get notified of successful completion or error via the given dispatch handler. This command will get serialized along with other commands issued with the same file descriptor.

Parameters
fdThe open file descriptor
bufferThe buffer to append
flagsFlags for this operation: O_FLUSH or 0
handlerThe dispatch handler
virtual size_t Hypertable::Filesystem::append ( int  fd,
StaticBuffer buffer,
Flags  flags = Flags::NONE 
)
pure virtual

Appends data to a file.

Issues an append request and waits for it to complete. This command will get serialized along with other commands issued with the same file descriptor.

Parameters
fdThe open file descriptor
bufferThe buffer to append
flagsFlags for this operation: O_FLUSH or 0
String Filesystem::basename ( String  name,
char  separator = '/' 
)
static

A posix-compliant basename() which strips directory names from a filename.

/usr/bin/sort -> sort

Parameters
nameThe directory name
separatorThe path separator
Returns
The basename

Definition at line 154 of file Filesystem.cc.

virtual void Hypertable::Filesystem::close ( int  fd,
DispatchHandler handler 
)
pure virtual

Closes a file asynchronously.

Issues a close file request. The caller will get notified of successful completion or error via the given dispatch handler. This command will get serialized along with other commands issued with the same file descriptor.

Parameters
fdThe open file descriptor
handlerThe dispatch handler
virtual void Hypertable::Filesystem::close ( int  fd)
pure virtual

Closes a file.

Issues a close command and waits for it to complete. This command will get serialized along with other commands issued with the same file descriptor.

Parameters
fdThe open file descriptor
virtual void Hypertable::Filesystem::create ( const String name,
uint32_t  flags,
int32_t  bufsz,
int32_t  replication,
int64_t  blksz,
DispatchHandler handler 
)
pure virtual

Creates a file asynchronously.

Issues a create file request with various create mode parameters. The caller will get notified of successful completion or error via the given dispatch handler. It is up to the caller to deserialize the returned file descriptor from the MESSAGE event object.

Parameters
nameAbsolute path name of file to open
flagsOpen flags (OPEN_FLAG_DIRECTIO or OPEN_FLAG_OVERWRITE)
bufszBuffer size to use for the underlying FS
replicationReplication factor to use for this file
blkszBlock size to use for the underlying FS
handlerThe dispatch handler which will handle the reply

Implemented in Hypertable::FsBroker::Lib::Client.

virtual int Hypertable::Filesystem::create ( const String name,
uint32_t  flags,
int32_t  bufsz,
int32_t  replication,
int64_t  blksz 
)
pure virtual

Creates a file.

Issues a create file request and waits for completion

Parameters
nameAbsolute path name of file to open
flagsOpen flags (OPEN_FLAG_DIRECTIO or OPEN_FLAG_OVERWRITE)
bufszBuffer size to use for the underlying FS
replicationReplication factor to use for this file
blkszBlock size to use for the underlying FS
Returns
The new file handle

Implemented in Hypertable::FsBroker::Lib::Client.

virtual void Hypertable::Filesystem::debug ( int32_t  command,
StaticBuffer serialized_parameters 
)
pure virtual

Invokes debug request asynchronously.

Parameters
commanddebug command identifier
serialized_parameterscommand specific serialized parameters

Implemented in Hypertable::FsBroker::Lib::Client.

virtual void Hypertable::Filesystem::debug ( int32_t  command,
StaticBuffer serialized_parameters,
DispatchHandler handler 
)
pure virtual

Invokes debug request.

Parameters
commandThe debug command identifier
serialized_parametersThe command specific serialized parameters
handlerThe dispatch/callback handler

Implemented in Hypertable::FsBroker::Lib::Client.

int Filesystem::decode_response ( EventPtr event)
static

Decodes the response from an request that only returns an error code.

Parameters
eventA reference to the response event
Returns
The error code

Definition at line 108 of file Filesystem.cc.

virtual void Hypertable::Filesystem::decode_response_append ( EventPtr event,
uint64_t *  offset,
uint32_t *  length 
)
pure virtual

Decodes the response from an append request.

Parameters
eventA reference to the response event
offsetAddress of offset variable
lengthAddress of length variable

Implemented in Hypertable::FsBroker::Lib::Client.

virtual void Hypertable::Filesystem::decode_response_create ( EventPtr event,
int32_t *  fd 
)
pure virtual

Decodes the response from a create request.

Parameters
eventreference to response event
fdAddress of variable to hold file descriptor

Implemented in Hypertable::FsBroker::Lib::Client.

bool Filesystem::decode_response_exists ( EventPtr event)
pure virtual

Decodes the response from an exists request.

Parameters
eventA reference to the response event
Returns
true if the file exists, false otherwise

Implemented in Hypertable::FsBroker::Lib::Client.

Definition at line 94 of file Filesystem.cc.

virtual int64_t Hypertable::Filesystem::decode_response_length ( EventPtr event)
pure virtual

Decodes the response from a length request.

Parameters
eventReference to response event
Returns
length of the file, in bytes

Implemented in Hypertable::FsBroker::Lib::Client.

virtual void Hypertable::Filesystem::decode_response_open ( EventPtr event,
int32_t *  fd 
)
pure virtual

Decodes the response from an open request.

Parameters
eventreference to response event
fdAddress of variable to hold file descriptor

Implemented in Hypertable::FsBroker::Lib::Client.

virtual void Hypertable::Filesystem::decode_response_pread ( EventPtr event,
const void **  buffer,
uint64_t *  offset,
uint32_t *  length 
)
pure virtual

Decodes the response from a pread request.

Parameters
eventA reference to the response event
bufferAddress of buffer pointer
offsetAddress of offset variable
lengthAddress of length variable

Implemented in Hypertable::FsBroker::Lib::Client.

virtual void Hypertable::Filesystem::decode_response_read ( EventPtr event,
const void **  buffer,
uint64_t *  offset,
uint32_t *  length 
)
pure virtual

Decodes the response from a read request.

Parameters
eventA reference to the response event
bufferAddress of buffer pointer
offsetAddress of offset variable
lengthAddress of length variable

Implemented in Hypertable::FsBroker::Lib::Client.

void Filesystem::decode_response_readdir ( EventPtr event,
std::vector< Dirent > &  listing 
)
pure virtual

Decodes the response from a readdir request.

Parameters
eventA reference to the response event
listingReference to output vector of Dirent objects

Implemented in Hypertable::FsBroker::Lib::Client.

Definition at line 71 of file Filesystem.cc.

virtual void Hypertable::Filesystem::decode_response_status ( EventPtr event,
Status status 
)
pure virtual

Decodes the response from an status request.

Parameters
eventReference to response event
statusReference to status information output parameter

Implemented in Hypertable::FsBroker::Lib::Client.

String Filesystem::dirname ( String  name,
char  separator = '/' 
)
static

A posix-compliant dirname() which strips the last component from a file name.

/usr/bin/ -> /usr stdio.h -> .

Parameters
nameThe directory name
separatorThe path separator
Returns
The stripped directory

Definition at line 127 of file Filesystem.cc.

virtual void Hypertable::Filesystem::exists ( const String name,
DispatchHandler handler 
)
pure virtual

Determines if a file exists asynchronously.

Issues an exists request. The caller will get notified of successful completion or error via the given dispatch handler.

Parameters
nameThe absolute pathname of file
handlerThe dispatch handler

Implemented in Hypertable::FsBroker::Lib::Client.

virtual bool Hypertable::Filesystem::exists ( const String name)
pure virtual

Determines if a file exists.

Parameters
nameThe absolute pathname of the file
Returns
true if the file exists, otherwise false

Implemented in Hypertable::FsBroker::Lib::Client.

virtual void Hypertable::Filesystem::flush ( int  fd,
DispatchHandler handler 
)
pure virtual

Flushes a file asynchronously.

Isues a flush command which causes all buffered writes to get persisted to disk. The caller will get notified of successful completion or error via the given dispatch handler. This command will get serialized along with other commands issued with the same file descriptor.

Parameters
fdThe open file descriptor
handlerThe dispatch handler
virtual void Hypertable::Filesystem::flush ( int  fd)
pure virtual

Flushes a file.

Issues a flush command which causes all buffered writes to get flushed to the underlying filesystem. For "normal" filesystems, such as the local filesystem, this command translates into an fsync() system call. However, for some distributed filesystems such as HDFS, this command causes the filesystem broker to flush buffered writes into the memory of all of the replica servers, but doesn't necessarily push the writes all the way down to the physical storage. This command will get serialized along with other commands issued with the same file descriptor.

Parameters
fdThe open file descriptor
virtual void Hypertable::Filesystem::length ( const String name,
bool  accurate,
DispatchHandler handler 
)
pure virtual

Gets the length of a file asynchronously.

Issues a length request. The caller will get notified of successful completion or error via the given dispatch handler.

Parameters
nameThe absolute pathname of file
accurateWhether the accurate or an estimated file length is required (an hdfs performance optimization)
handlerThe dispatch handler

Implemented in Hypertable::FsBroker::Lib::Client.

virtual int64_t Hypertable::Filesystem::length ( const String name,
bool  accurate = true 
)
pure virtual

Gets the length of a file.

Issues a length request and waits for it to complete.

Parameters
nameThe absolute pathname of file
accurateWhether the accurate or an estimated file length is required (an hdfs performance optimization)

Implemented in Hypertable::FsBroker::Lib::Client.

virtual void Hypertable::Filesystem::mkdirs ( const String name,
DispatchHandler handler 
)
pure virtual

Creates a directory asynchronously.

Issues a mkdirs request which creates a directory, including all its missing parents. The caller will get notified of successful completion or error via the given dispatch handler.

Parameters
nameThe absolute pathname of directory to create
handlerThe dispatch handler

Implemented in Hypertable::FsBroker::Lib::Client.

virtual void Hypertable::Filesystem::mkdirs ( const String name)
pure virtual

Creates a directory.

Issues a mkdirs request which creates a directory, including all its missing parents, and waits for it to complete.

Parameters
nameThe absolute pathname of the directory to create

Implemented in Hypertable::FsBroker::Lib::Client.

virtual void Hypertable::Filesystem::open ( const String name,
uint32_t  flags,
DispatchHandler handler 
)
pure virtual

Opens a file asynchronously.

Issues an open file request. The caller will get notified of successful completion or error via the given dispatch handler. It is up to the caller to deserialize the returned file descriptor from the MESSAGE event object.

Parameters
nameAbsolute path name of file to open
flagsOpen flags (OPEN_FLAG_DIRECTIO or 0)
handlerThe dispatch handler which will handle the reply

Implemented in Hypertable::FsBroker::Lib::Client.

virtual int Hypertable::Filesystem::open ( const String name,
uint32_t  flags 
)
pure virtual

Opens a file.

Issues an open file request and waits for it to complete.

Parameters
nameAbsolute path name of file to open
flagsOpen flags (OPEN_FLAG_DIRECTIO or 0)
Returns
The new file handle

Implemented in Hypertable::FsBroker::Lib::Client.

virtual int Hypertable::Filesystem::open_buffered ( const String name,
uint32_t  flags,
uint32_t  buf_size,
uint32_t  outstanding,
uint64_t  start_offset = 0,
uint64_t  end_offset = 0 
)
pure virtual

Opens a file in buffered (readahead) mode.

Issues an open file request and waits for it to complete. Turns on readahead mode so that data is prefetched.

Parameters
nameAbsolute path name of file to open
flagsOpen flags (OPEN_FLAG_DIRECTIO or 0)
buf_sizeRead ahead buffer size
outstandingMaximum number of outstanding reads
start_offsetStarting read offset
end_offsetEnding read offset
Returns
The new file handle

Implemented in Hypertable::FsBroker::Lib::Client.

virtual void Hypertable::Filesystem::pread ( int  fd,
size_t  amount,
uint64_t  offset,
bool  verify_checksum,
DispatchHandler handler 
)
pure virtual

Reads data from a file at the specified position asynchronously.

Issues a pread request. The caller will get notified of successful completion or error via the given dispatch handler. It's up to the caller to deserialize the returned data in the MESSAGE event object. EOF is indicated by a short read.

Parameters
fdThe open file descriptor
offsetThe starting offset of read
amountThe amount of data to read (in bytes)
verify_checksumTells filesystem to perform checksum verification
handlerThe dispatch handler
virtual size_t Hypertable::Filesystem::pread ( int  fd,
void *  dst,
size_t  len,
uint64_t  offset,
bool  verify_checksum = true 
)
pure virtual

Reads data from a file at the specified position.

Issues a pread request and waits for it to complete, returning the read data. EOF is indicated by a short read. This command will get serialized along with other commands issued with the same file descriptor.

Parameters
fdThe open file descriptor
dstThe destination buffer for read data
lenThe amount of data to read
offsetstarting The offset of read
verify_checksumTells filesystem to perform checksum verification
Returns
The amount of data read (in bytes)
virtual void Hypertable::Filesystem::read ( int  fd,
size_t  len,
DispatchHandler handler 
)
pure virtual

Reads data from a file at the current position asynchronously.

Issues a read request. The caller will get notified of successful completion or error via the given dispatch handler. It's up to the caller to deserialize the returned data in the MESSAGE event object. EOF is indicated by a short read. This command will get serialized along with other commands issued with the same file descriptor.

Parameters
fdThe open file descriptor
lenAmount of data to read
handlerThe dispatch handler
virtual size_t Hypertable::Filesystem::read ( int  fd,
void *  dst,
size_t  len 
)
pure virtual

Reads data from a file at the current position.

Issues a read request and waits for it to complete, returning the read data. EOF is indicated by a short read. This command will get serialized along with other commands issued with the same file descriptor.

Parameters
fdThe open file descriptor
dstThe destination buffer for read data
lenThe amount of data to read
Returns
The amount read (in bytes)
virtual void Hypertable::Filesystem::readdir ( const String name,
DispatchHandler handler 
)
pure virtual

Obtains a listing of all files in a directory asynchronously.

Issues a readdir request. The caller will get notified of successful completion or error via the given dispatch handler.

Parameters
nameThe absolute pathname of directory
handlerThe dispatch handler

Implemented in Hypertable::FsBroker::Lib::Client.

virtual void Hypertable::Filesystem::readdir ( const String name,
std::vector< Dirent > &  listing 
)
pure virtual

Obtains a listing of all files in a directory.

Issues a readdir request and waits for it to complete.

Parameters
nameAbsolute pathname of directory
listingReference to output vector of Dirent objects for each entry

Implemented in Hypertable::FsBroker::Lib::Client.

virtual void Hypertable::Filesystem::remove ( const String name,
DispatchHandler handler 
)
pure virtual

Removes a file asynchronously.

Issues a remove request. The caller will get notified of successful completion or error via the given dispatch handler.

Parameters
nameThe absolute pathname of file to delete
handlerThe dispatch handler

Implemented in Hypertable::FsBroker::Lib::Client.

virtual void Hypertable::Filesystem::remove ( const String name,
bool  force = true 
)
pure virtual

Removes a file.

Issues a remove request and waits for it to complete.

Parameters
nameThe absolute pathname of file to delete
forceIf true then ignore non-existence error

Implemented in Hypertable::FsBroker::Lib::Client.

virtual void Hypertable::Filesystem::rename ( const String src,
const String dst,
DispatchHandler handler 
)
pure virtual

Rename a path asynchronously.

Parameters
srcThe source path
dstThe destination path
handlerThe dispatch/callback handler

Implemented in Hypertable::FsBroker::Lib::Client.

virtual void Hypertable::Filesystem::rename ( const String src,
const String dst 
)
pure virtual

Rename a path.

Parameters
srcThe source path
dstThe destination path

Implemented in Hypertable::FsBroker::Lib::Client.

virtual void Hypertable::Filesystem::rmdir ( const String name,
DispatchHandler handler 
)
pure virtual

Recursively removes a directory asynchronously.

Issues a rmdir request. The caller will get notified of successful completion or error via the given dispatch handler.

Parameters
nameThe absolute pathname of directory to remove
handlerThe dispatch handler

Implemented in Hypertable::FsBroker::Lib::Client.

virtual void Hypertable::Filesystem::rmdir ( const String name,
bool  force = true 
)
pure virtual

Recursively removes a directory.

Issues a rmdir request and waits for it to complete.

Parameters
nameThe absolute pathname of directory to remove
forceIf true then don't throw an error if file does not exist

Implemented in Hypertable::FsBroker::Lib::Client.

virtual void Hypertable::Filesystem::seek ( int  fd,
uint64_t  offset,
DispatchHandler handler 
)
pure virtual

Seeks current file position asynchronously.

Issues a seek request. The caller will get notified of successful completion or error via the given dispatch handler. This command will get serialized along with other commands issued with the same file descriptor.

Parameters
fdThe open file descriptor
offsetThe absolute offset to seek to
handlerThe dispatch handler
virtual void Hypertable::Filesystem::seek ( int  fd,
uint64_t  offset 
)
pure virtual

Seeks current file position.

Issues a seek request and waits for it to complete. This command will get serialized along with other commands issued with the same file descriptor.

Parameters
fdThe open file descriptor
offsetThe absolute offset to seek to
virtual void Hypertable::Filesystem::status ( Status status,
Timer timer = 0 
)
pure virtual

Check status of filesystem.

Parameters
statusStatus output
timerDeadline timer
Returns
Nagios-style status code

Implemented in Hypertable::FsBroker::Lib::Client.

virtual void Hypertable::Filesystem::sync ( int  fd)
pure virtual

Syncs a file.

Issues a sync command which causes the filesystem to persist all buffered updates to the physical storage. It is equivalent to the fsync() Unix system call. This command will get serialized along with other commands issued with the same file descriptor.

Parameters
fdThe open file descriptor

The documentation for this class was generated from the following files: