Class cloud_file (o2scl_hdf)¶

class cloud_file¶

Read a file and download from a URL if necessary.

Idea for Future:: Convert to use boost::filesystem?

Idea for Future:: Automatically handle compressed files? This turns out to be complicated, because does the user specify the compressed hash, the decompressed hash, or both? I suppose it’s best that the user specifies both, and you only download if you need to, or uncompress if you need to, but this will require nearly a full reworking of the get_file_hash function (or a completely new function). For the moment, compression has to be handled by the user.

Note

This class requires POSIX I/O calls and a system call which uses mkdir -p, thus will probably only work on unix-like systems.

Note

This class uses system calls to curl or wget which must be installed separatley.

Warning

This class has several potential security issues and should not be used without due care.

Specify hash type

int hash_type¶: Current hash type (default sha256)

static const int sha256 = 0¶

static const int md5 = 1¶

static const int md5sum = 2¶

cloud_file()¶

int hdf5_open(hdf_file &hf, std::string file, std::string url, std::string dir = "")¶: Open an HDF file named file in directory dir downloading from URL url if necessary.

int hdf5_open_hash(hdf_file &hf, std::string file, std::string url, std::string hash = "", std::string dir = "")¶: Open an HDF file named file in directory dir with hash hash, downloading from URL url if necessary.

int get_file(std::string file, std::string url, std::string dir = "")¶: Get file named file in directory dir from url url.

int get_file_hash(std::string file, std::string url, std::string hash = "", std::string dir = "")¶

Get file named file in directory dir in subdirectory subdir from url url.

This function begins with the directory dir. If dir is not present and cannot be created, the user is prompted for the correct data directory. This function then searches for file file in the directory. If it is found, it is compared with the specified hash. If the hash matches, then the full filename is returned. If the hash does not match or if the file is not found, then this function uses curl (or wget if curl was unsuccessful) to download the file from url. The file is then compared with the hash again, and the full filename is returned if the hash matches. Otherwise the error handler is called.

Public Members

bool allow_wget¶: If true, allow the use of wget to download the file (default true)

bool allow_curl¶: If true, allow the use of curl to download the file (default true)

int verbose¶: Verbosity parameter (default 1)

bool throw_on_fail¶: If true, throw an exception on failure (default true)