Storage

class Storage(url=None, auth=None)[source]

The parent Service class implements authentication and exponential backoff/retry. Override the url parameter to use a different instance of the backing service.

copy_from_bucket(src_bucket_name, src, dest, user_ns=None, storage_type='data')[source]

Copy a file from a google cloud storage bucket to your descartes data bucket. This requires that the dlstorage service account have access to your 3rd party bucket. :param str src_bucket_name: The name of the bucket from which you want to copy data. :param str src: The path to the file you want to copy inside the src_bucket_name` bucket. :param str dest: The new path of the file inside the descarteslabs data bucket.

delete(key, storage_type='data')[source]

Delete the data stored at location key with storage type storage_type

Parameters:
  • key (str) – A unique string mapped to an existing storage blob
  • storage_type (str) – A type of data storage. Possible values: data”, “tmp”, “result”. Default: “data”.
get(key, storage_type='data')[source]

Retrieve data stored at location key, with storage type storage_type

Parameters:
  • key (str) – A unique string mapped to an existing storage blob
  • storage_type (str) – A type of data storage. Possible values: “data”, “tmp”, “result”. Default: “data”.
Returns:

The string that was stored at key

get_signed_url(key, storage_type='data')[source]
get_upload_url(key, storage_type='data', **kwargs)[source]
iter_list(prefix=None, storage_type='data')[source]

Yield keys that have been stored, with an optional prefix and storage_type. :param str prefix: A prefix match of keys returned. :param str storage_type: A type of data storage. Possible values: “data”, “tmp”, “result”. Default: “data”.

Returns:Yields keys in an iterable.
list(prefix=None, storage_type='data')[source]

List keys that have been stored, with an optional prefix and storage_type. :param str prefix: A prefix match of keys returned. :param str storage_type: A type of data storage. Possible values: “data”, “tmp”, “result”. Default: “data”.

Returns:A list of keys.
set(key, value, storage_type='data')[source]

Store string value at location key, with storage type storage_type

Parameters:
  • key (str) – A unique string mapped to an existing storage blob
  • value (str) – bytes to be stored at location key
  • storage_type (str) – A type of data storage. Possible values: “data”, “tmp”, “result”. Default: “data”.
TIMEOUT = (9.5, 120)

Data Storage Service

cached(storage_client, minimum_runtime=0.0)[source]