Storage

class Storage(url=None, auth=None, retries=None)[source]

The Storage API provides a mechanism to store arbitrary data and later retrieve it using simple key-value pair semantics.

Parameters:
  • url (str) – A HTTP URL pointing to a version of the storage service (defaults to current version)
  • auth (Auth) – A custom user authentication (defaults to the user authenticated locally by token information on disk or by environment variables)
  • retries (urllib3.util.retry.Retry) – A custom retry configuration used for all API requests (defaults to a reasonable amount of retries)
delete(key, storage_type='data')[source]

Deletes the data stored at location key with storage type storage_type.

Parameters:
  • key (str) – A key identifying an existing storage blob
  • storage_type (str) – A type of data storage. Possible values: "data", "tmp", "result". Default: "data".
Raises:

descarteslabs.client.exceptions.NotFoundError – if no data exists for the given key and storage type

exists(key, storage_type='data')[source]

Determine if there is data stored at location key with storage type storage_type.

Parameters:
  • key (str) – A key identifying a storage blob
  • storage_type (str) – A type of data storage. Possible values: "data", "tmp", "result". Default: "data".
Returns:

whether there is data at location key

Return type:

bool

get(key, storage_type='data')[source]

Retrieves data stored at location key with storage type storage_type.

Parameters:
  • key (str) – A key identifying a storage blob
  • storage_type (str) – A type of data storage. Possible values: "data", "tmp", "result". Default: "data".
Returns:

the data stored at key as a string

Return type:

str

Raises:

descarteslabs.client.exceptions.NotFoundError – if no data exists for the given key and storage type

get_file(key, file_obj, storage_type='data')[source]

Retrieves data stored at location key with storage type storage_type and write it to a file.

Parameters:
  • key (str) – A key identifying a storage blob
  • file_obj (str|file-like) – File-like object or name of file to which retrieved data will be written. If this is a file-like object it must accept bytes (for example, a file opened in binary mode such as with open(filename, 'wb')).
  • storage_type (str) – A type of data storage. Possible values: "data", "tmp", "result". Default: "data".
Raises:

descarteslabs.client.exceptions.NotFoundError – if no data exists for the given key and storage type

get_signed_url(key, storage_type='data')[source]

Gets a temporary signed URL for downloading data stored at a key.

Parameters:
  • key (str) – A key a storage blob
  • storage_type (str) – A type of data storage. Possible values: "data", "tmp", "result". Default: "data".
Returns:

A temporary signed URL used to download the data

Return type:

str

get_upload_url(key, storage_type='data', **kwargs)[source]

Gets a temporary signed URL for uploading data to a key.

Parameters:
  • key (str) – A key a storage blob
  • storage_type (str) – A type of data storage. Possible values: "data", "tmp", "result". Default: "data".
Returns:

A temporary signed URL used to upload data to

Return type:

str

iter_list(prefix=None, storage_type='data')[source]

Yields keys stored under the given storage_type, optionally with a required prefix.

Parameters:
  • prefix (str) – Only include keys with this prefix if given
  • storage_type (str) – A type of data storage. Possible values: "data", "tmp", "result". Default: "data".
Returns:

An iterator over all matching keys

Return type:

generator(str)

list(prefix=None, storage_type='data')[source]

Lists keys stored under the given storage_type, optionally with a required prefix. Includes up to 1000 results, use Storage.iter_list() to iterate over more results.

Parameters:
  • prefix (str) – Only include keys with this prefix if given
  • storage_type (str) – A type of data storage. Possible values: "data", "tmp", "result". Default: "data".
Returns:

A list of matching keys

Return type:

list(str)

set(key, value, storage_type='data')[source]

Stores a value at location key with storage type storage_type.

Parameters:
  • key (str) – A key identifying a storage blob
  • value (str|file-like) – A string value or file-like object to be stored at location key
  • storage_type (str) – A type of data storage. Possible values: "data", "tmp", "result". Default: "data".
set_file(key, file_obj, storage_type='data')[source]

Stores data from a file or file-like object at location key with storage type storage_type.

Parameters:
  • key (str) – A key identifying a storage blob
  • value (str|file-like) – A file name or a file-like object with data to be stored at location key
  • storage_type (str) – A type of data storage. Possible values: "data", "tmp", "result". Default: "data".
TIMEOUT = (9.5, 120)