Arbiter  0.0.0
Public Member Functions | Static Public Member Functions | List of all members
arbiter::Arbiter Class Reference

The primary interface for storage abstraction. More...

#include <arbiter/arbiter.hpp>

Public Member Functions

 Arbiter ()
 
 Arbiter (const Json::Value &json)
 Construct an Arbiter with driver configurations.
 
bool hasDriver (std::string path) const
 
void addDriver (std::string type, std::unique_ptr< Driver > driver)
 Add a custom driver for the supplied type. More...
 
std::string get (std::string path) const
 
std::unique_ptr< std::string > tryGet (std::string path) const
 
std::vector< char > getBinary (std::string path) const
 
std::unique_ptr< std::vector< char > > tryGetBinary (std::string path) const
 
std::size_t getSize (std::string path) const
 
std::unique_ptr< std::size_t > tryGetSize (std::string path) const
 
void put (std::string path, const std::string &data) const
 
void put (std::string path, const std::vector< char > &data) const
 
std::string get (std::string path, http::Headers headers, http::Query query=http::Query()) const
 
std::unique_ptr< std::string > tryGet (std::string path, http::Headers headers, http::Query query=http::Query()) const
 
std::vector< char > getBinary (std::string path, http::Headers headers, http::Query query=http::Query()) const
 
std::unique_ptr< std::vector< char > > tryGetBinary (std::string path, http::Headers headers, http::Query query=http::Query()) const
 
void put (std::string path, const std::string &data, http::Headers headers, http::Query query=http::Query()) const
 
void put (std::string path, const std::vector< char > &data, http::Headers headers, http::Query query=http::Query()) const
 
void copy (std::string src, std::string dst, bool verbose=false) const
 
void copyFile (std::string file, std::string to, bool verbose=false) const
 
bool isRemote (std::string path) const
 
bool isLocal (std::string path) const
 
bool exists (std::string path) const
 
bool isHttpDerived (std::string path) const
 
std::vector< std::string > resolve (std::string path, bool verbose=false) const
 Resolve a possibly globbed path. More...
 
Endpoint getEndpoint (std::string root) const
 Get a reusable Endpoint for this root directory.
 
const DrivergetDriver (std::string path) const
 
std::unique_ptr< fs::LocalHandlegetLocalHandle (std::string path, const Endpoint &tempEndpoint) const
 Get a fs::LocalHandle to a possibly remote file. More...
 
std::unique_ptr< fs::LocalHandlegetLocalHandle (std::string path, std::string tempPath="") const
 Get a fs::LocalHandle to a possibly remote file. More...
 
http::Pool & httpPool ()
 

Static Public Member Functions

static std::string getType (std::string path)
 
static std::string stripType (std::string path)
 
static std::string getExtension (std::string path)
 

Detailed Description

The primary interface for storage abstraction.

The Arbiter is the primary layer of abstraction for all supported Driver instances, allowing remote paths to be treated as a simple filesystem or key-value store. Routing to specialized drivers is based on a substring comparison of an incoming path's delimited type, for example http://arbiter.io/data.txt would route to the HTTP driver and s3://my-bucket/data.txt would route to the S3 driver. Paths containing no prefixed type information are routed to the filesystem driver.

All Arbiter operations are thread-safe except unless otherwise noted.

Constructor & Destructor Documentation

arbiter::Arbiter::Arbiter ( )

Construct a basic Arbiter with only drivers the don't require external configuration parameters.

Member Function Documentation

void arbiter::Arbiter::addDriver ( std::string  type,
std::unique_ptr< Driver driver 
)

Add a custom driver for the supplied type.

After this operation completes, future requests into arbiter beginning with the prefix type followed by the delimiter :// will be routed to the supplied driver. If a Driver of type type already exists, the supplied driver will replace it.

This operation will throw ArbiterError if driver is empty.

Note
This operation is not thread-safe.
void arbiter::Arbiter::copy ( std::string  src,
std::string  dst,
bool  verbose = false 
) const

Copy data from src to dst. src will be resolved with Arbiter::resolve prior to the copy, so globbed directories are supported. If src ends with a slash, it will be resolved with a recursive glob, in which case any nested directory structure will be recreated in dst.

If dst is a filesystem path, fs::mkdirp will be called prior to the start of copying. If src is a recursive glob, fs::mkdirp will be repeatedly called during copying to ensure that any nested directories are reproduced.

void arbiter::Arbiter::copyFile ( std::string  file,
std::string  to,
bool  verbose = false 
) const

Copy the single file file to the destination to. If to ends with a / or '\' character, then file will be copied into the directory to with the basename of file. If does not end with a slash character, then to will be interpreted as a file path.

If to is a local filesystem path, then fs::mkdirp will be called prior to copying.

bool arbiter::Arbiter::exists ( std::string  path) const

Returns true if this path exists. Equivalent to:

tryGetSize(path).get() != nullptr
Note
This means that an existing file of size zero will return true.
std::string arbiter::Arbiter::get ( std::string  path) const

Get data or throw if inaccessible.

std::string arbiter::Arbiter::get ( std::string  path,
http::Headers  headers,
http::Query  query = http::Query() 
) const

Get data with additional HTTP-specific parameters. Throws if isHttpDerived is false for this path.

std::vector< char > arbiter::Arbiter::getBinary ( std::string  path) const

Get data in binary form or throw if inaccessible.

std::vector< char > arbiter::Arbiter::getBinary ( std::string  path,
http::Headers  headers,
http::Query  query = http::Query() 
) const

Get data in binary form with additional HTTP-specific parameters. Throws if isHttpDerived is false for this path.

const Driver & arbiter::Arbiter::getDriver ( std::string  path) const

Returns the Driver, if one can be found, for the given path. The driver type is determined from the path substring prior to the delimiter ://. If this delimiter does not exist, then the filesystem driver is returned. If the delimiter exists but a corresponding driver type is not found, ArbiterError is thrown.

Optionally, filesystem paths may be explicitly prefixed with file://.

std::string arbiter::Arbiter::getExtension ( std::string  path)
static

Get the characters following the final instance of '.', or an empty string if there are no '.' characters.

std::unique_ptr< fs::LocalHandle > arbiter::Arbiter::getLocalHandle ( std::string  path,
const Endpoint tempEndpoint 
) const

Get a fs::LocalHandle to a possibly remote file.

If path is remote (see Arbiter::isRemote), this operation will fetch the file contents and write them to the local filesystem in the directory represented by tmpEndpoint. The contents of path are not copied if path is already local.

There are no filename guarantees if path is remote. Use LocalHandle::localPath to determine this.

Note
This operation will throw an ArbiterError if the path is remote and tmpEndpoint is also remote.
Parameters
pathPossibly remote path to fetch.
tempEndpointIf is remote, the local copy will be created at this Endpoint.
Returns
A fs::LocalHandle for local access to the resulting file.
std::unique_ptr< fs::LocalHandle > arbiter::Arbiter::getLocalHandle ( std::string  path,
std::string  tempPath = "" 
) const

Get a fs::LocalHandle to a possibly remote file.

If tempPath is not specified, the environment will be searched for a temporary location.

std::size_t arbiter::Arbiter::getSize ( std::string  path) const

Get file size in bytes or throw if inaccessible.

std::string arbiter::Arbiter::getType ( std::string  path)
static

If no delimiter of "://" is found, returns "file". Otherwise, returns the substring prior to but not including this delimiter.

bool arbiter::Arbiter::hasDriver ( std::string  path) const

True if a Driver has been registered for this file type.

http::Pool& arbiter::Arbiter::httpPool ( )
inline

Fetch the common HTTP pool, which may be useful when dynamically constructing adding a Driver via Arbiter::addDriver.

bool arbiter::Arbiter::isHttpDerived ( std::string  path) const

Returns true if the protocol for this driver is build on HTTP, like the S3 and Dropbox drivers are. If this returns true, http::Headers and http::Query parameter methods may be used for this path.

bool arbiter::Arbiter::isLocal ( std::string  path) const

Returns true if this path is on the local filesystem, or false if it is remote.

bool arbiter::Arbiter::isRemote ( std::string  path) const

Returns true if this path is a remote path, or false if it is on the local filesystem.

void arbiter::Arbiter::put ( std::string  path,
const std::string &  data 
) const

Write data to path.

void arbiter::Arbiter::put ( std::string  path,
const std::vector< char > &  data 
) const

Write data to path.

void arbiter::Arbiter::put ( std::string  path,
const std::string &  data,
http::Headers  headers,
http::Query  query = http::Query() 
) const

Write data to path with additional HTTP-specific parameters. Throws if isHttpDerived is false for this path.

void arbiter::Arbiter::put ( std::string  path,
const std::vector< char > &  data,
http::Headers  headers,
http::Query  query = http::Query() 
) const

Write data to path with additional HTTP-specific parameters. Throws if isHttpDerived is false for this path.

std::vector< std::string > arbiter::Arbiter::resolve ( std::string  path,
bool  verbose = false 
) const

Resolve a possibly globbed path.

If path ends with *, then this operation will attempt to non-recursively glob all files matching the given wildcard. Directories matching the wildcard and their contents are not returned.

Examples:

// Returns all files in `~/data`. A directory `~/data/dir/` and its
// contents would not be returned since this is non-recursive.
a.resolve("~\data\*");
// Returns all files matching `prefix-*` in bucket `my-bucket`. An
// object `s3://my-bucket/prefix-123/hello.txt` would not be returned,
// but object `s3://my-bucket/prefix-456.txt` would.
a.resolve("s3://my-bucket/prefix-*");
Note
Throws ArbiterError if the selected driver does not support globbing, for example the HTTP driver.
Parameters
pathPath to search, may be a directory or file. Globbing will only occur in the case of a directory.
verboseIf true, driver-specific status information may be printed to STDOUT during globbing.
Returns
If path ends with *, the results are the contents of the resulting non-recursive resolution of this path. Otherwise, the results are a vector of size one containing only path itself, unaltered.
std::string arbiter::Arbiter::stripType ( std::string  path)
static

Strip the type and delimiter ://, if they exist.

std::unique_ptr< std::string > arbiter::Arbiter::tryGet ( std::string  path) const

Get data if accessible.

std::unique_ptr< std::string > arbiter::Arbiter::tryGet ( std::string  path,
http::Headers  headers,
http::Query  query = http::Query() 
) const

Get data with additional HTTP-specific parameters. Throws if isHttpDerived is false for this path.

std::unique_ptr< std::vector< char > > arbiter::Arbiter::tryGetBinary ( std::string  path) const

Get data in binary form if accessible.

std::unique_ptr< std::vector< char > > arbiter::Arbiter::tryGetBinary ( std::string  path,
http::Headers  headers,
http::Query  query = http::Query() 
) const

Get data in binary form with additional HTTP-specific parameters. Throws if isHttpDerived is false for this path.

std::unique_ptr< std::size_t > arbiter::Arbiter::tryGetSize ( std::string  path) const

Get file size in bytes if accessible.


The documentation for this class was generated from the following files: