disco.ddfs
– Client interface for Disco Distributed Filesystem¶
See also: Disco Distributed Filesystem.
Note
Parameters below which are indicated as tags can be specified as a tag:// URL, or the name of the tag.
-
class
disco.ddfs.
DDFS
(master=None, proxy=None, settings=None)¶ Opens and encapsulates a connection to a DDFS master.
Parameters: master – address of the master, for instance disco://localhost
.-
attrs
(tag, token=None)¶ Get a list of the attributes of the tag
tag
and their values.
-
blobs
(tag, ignore_missing=True, token=None)¶ Walks the tag graph starting at tag.
Yields only the terminal nodes of the graph (blobs).
Parameters: ignore_missing (bool) – Whether or not missing tags will raise a disco.error.CommError
.
-
chunk
(tag, urls, replicas=None, forceon=[], retries=10, delayed=False, update=False, token=None, chunk_size=67108864, max_record_size=1048576, **kwargs)¶ Chunks the contents of urls, pushes the chunks to ddfs and tags them with tag.
-
delattr
(tag, attr, token=None)¶ Delete the attribute
attr
of the tagtag
.
-
delete
(tag, token=None)¶ Delete
tag
.
-
exists
(tag)¶ Returns whether or not
tag
exists.
-
get
(tag, token=None)¶ Return the tag object stored at
tag
.
-
getattr
(tag, attr, token=None)¶ Return the value of the attribute
attr
of the tagtag
.
-
list
(prefix='')¶ Return a list of all tags starting with
prefix
.
-
pull
(tag, blobfilter=<function <lambda>>, token=None)¶ Iterate over the blobs in a
tag
after optionally applying ablobfilter
over the blob names.
-
push
(tag, files, replicas=None, forceon=[], retries=10, delayed=False, update=False, token=None)¶ Pushes a bunch of files to ddfs and tags them with tag.
Parameters: files (a list of paths
,(path, name)
-tuples, or(fileobject, name)
-tuples.) – the files to push as blobs to DDFS. If names are provided, they will be used as prefixes by DDFS for the blobnames. Names may only contain chars inr'[^A-Za-z0-9_\-@:]'
.
-
put
(tag, urls, token=None)¶ Put the list of
urls
to the tagtag
.Warning
Generally speaking, concurrent applications should use
DDFS.tag()
instead.
-
setattr
(tag, attr, val, token=None)¶ Set the value of the attribute
attr
of the tagtag
.
-
tag
(tag, urls, token=None, **kwargs)¶ Append the list of
urls
to thetag
.
-
urls
(tag, token=None)¶ Return the urls in the
tag
.
-
walk
(tag, ignore_missing=True, tagpath=(), token=None)¶ Walks the tag graph starting at tag.
Yields a 3-tuple (tagpath, tags, blobs).
Parameters: ignore_missing (bool) – Whether or not missing tags will raise a disco.error.CommError
.
-