Storage

class Storage(url=None, auth=None, retries=None)

The Storage API provides a mechanism to store arbitrary data and later retrieve it using simple key-value pair semantics.

Parameters:

• url (str) – URL for the storage service. Only change this if you are being asked to use a non-default Descartes Labs catalog. If not set, then descarteslabs.config.get_settings().STORAGE_URL will be used.
• auth (Auth) – A custom user authentication (defaults to the user authenticated locally by token information on disk or by environment variables)
• retries (urllib3.util.retry.Retry) – A custom retry configuration used for all API requests (defaults to a reasonable amount of retries)

Attributes:

TIMEOUT

Methods:

delete(key[, storage_type])	Deletes the data stored at location key with storage type storage_type.
exists(key[, storage_type])	Determine if there is data stored at location key with storage type storage_type.
get(key[, storage_type])	Retrieves data stored at location key with storage type storage_type.
get_file(key, file_obj[, storage_type])	Retrieves data stored at location key with storage type storage_type and write it to a file.
get_signed_url(key[, storage_type])	Gets a temporary signed URL for downloading data stored at a key.
get_upload_url(key[, storage_type])	Gets a temporary signed URL for uploading data to a key.
iter_list([prefix, storage_type])	Yields keys stored under the given storage_type, optionally with a required prefix.
list([prefix, storage_type])	Lists keys stored under the given storage_type, optionally with a required prefix.
set(key, value[, storage_type])	Stores a value at location key with storage type storage_type.
set_file(key, file_obj[, storage_type])	Stores data from a file or file-like object at location key with storage type storage_type.

delete(key, storage_type='data')

Deletes the data stored at location key with storage type storage_type.

Parameters:

• key (str) – A key identifying an existing storage blob
• storage_type (str) – A type of data storage. Possible values: "data", "tmp", "result". Default: "data".

Raises:

descarteslabs.exceptions.NotFoundError – if no data exists for the given key and storage type

exists(key, storage_type='data')

Determine if there is data stored at location key with storage type storage_type.

Parameters:

• key (str) – A key identifying a storage blob
• storage_type (str) – A type of data storage. Possible values: "data", "tmp", "result". Default: "data".

Returns:

whether there is data at location key

Return type:

bool

get(key, storage_type='data')

Retrieves data stored at location key with storage type storage_type.

Parameters:

• key (str) – A key identifying a storage blob
• storage_type (str) – A type of data storage. Possible values: "data", "tmp", "result". Default: "data".

Returns:

the data stored at key as a string

Return type:

str

Raises:

descarteslabs.exceptions.NotFoundError – if no data exists for the given key and storage type

get_file(key, file_obj, storage_type='data')

Retrieves data stored at location key with storage type storage_type and write it to a file.

Parameters:

• key (str) – A key identifying a storage blob
• file_obj (str|file-like) – File-like object or name of file to which retrieved data will be written. If this is a file-like object it must accept bytes (for example, a file opened in binary mode such as with open(filename, 'wb')).
• storage_type (str) – A type of data storage. Possible values: "data", "tmp", "result". Default: "data".

Raises:

descarteslabs.exceptions.NotFoundError – if no data exists for the given key and storage type

get_signed_url(key, storage_type='data')

Gets a temporary signed URL for downloading data stored at a key.

Parameters:

• key (str) – A key a storage blob
• storage_type (str) – A type of data storage. Possible values: "data", "tmp", "result". Default: "data".

Returns:

A temporary signed URL used to download the data

Return type:

str

get_upload_url(key, storage_type='data', **kwargs)

Gets a temporary signed URL for uploading data to a key.

Parameters:

• key (str) – A key a storage blob
• storage_type (str) – A type of data storage. Possible values: "data", "tmp", "result". Default: "data".

Returns:

A temporary signed URL used to upload data to

Return type:

str

iter_list(prefix=None, storage_type='data')

Yields keys stored under the given storage_type, optionally with a required prefix.

Parameters:

• prefix (str) – Only include keys with this prefix if given
• storage_type (str) – A type of data storage. Possible values: "data", "tmp", "result". Default: "data".

Returns:

An iterator over all matching keys

Return type:

generator(str)

list(prefix=None, storage_type='data')

Lists keys stored under the given storage_type, optionally with a required prefix. Includes up to 1000 results, use Storage.iter_list() to iterate over more results.

Parameters:

• prefix (str) – Only include keys with this prefix if given
• storage_type (str) – A type of data storage. Possible values: "data", "tmp", "result". Default: "data".

Returns:

A list of matching keys

Return type:

list(str)

set(key, value, storage_type='data')

Stores a value at location key with storage type storage_type.

Parameters:

• key (str) – A key identifying a storage blob
• value (str|file-like) – A string value or file-like object to be stored at location key
• storage_type (str) – A type of data storage. Possible values: "data", "tmp", "result". Default: "data".

set_file(key, file_obj, storage_type='data')

Stores data from a file or file-like object at location key with storage type storage_type.

Parameters:

• key (str) – A key identifying a storage blob
• value (str|file-like) – A file name or a file-like object with data to be stored at location key
• storage_type (str) – A type of data storage. Possible values: "data", "tmp", "result". Default: "data".

TIMEOUT = (9.5, 120)