Containers¶

Classes:

`CollectionMixin`()
`Dict`(dct, *kwargs)	`Dict[KeyType, ValueType]`: Proxy mapping, from keys of a specific type to values of a specific type.
`Ellipsis`()	Proxy Ellipsis object.
`List`(iterable)	`List[ValueType]`: Proxy sequence of any number of elements, all of the same type.
`Slice`([start, stop, step])	Proxy Slice object.
`Struct`(**kwargs)	`Struct[{field_name: type, ...}]`: Proxy container with named fields of specific types, meant as a helper base class.
`Tuple`(iterable)	`Tuple[item1_type, item2_type, ...]`: Proxy sequence of a fixed number of elements of specific types.
`KnownDict`(args, *kwargs)	`KnownDict[<{key: KnownType, ...}>, KeyType, ValueType]`: Proxy mapping from specific keys to specific value types, with default type for unknown values.

Functions:

`range`([start, stop, step])	Returns a `List` of `Int` containing a sequence of numbers starting from 0 by default, incremented by 1 (default), ending at a specified number.
`zip`(*sequences)	Returns a `List` of `Tuple`, where each tuple contains the i-th element from each of the arguments.

class CollectionMixin[source]¶

Methods:

`contains`(other)	Contains is equivalient to the Python `in` operator.
`filter`(func)	Filter elements from an iterable proxytype.
`length`()	Length is equivalent to the Python `len` operator.
`map`(func)	Map a function over an iterable proxytype.
`reduce`(func[, initial])	Reduce a collection of elements to a single element.
`sorted`([key, reverse])	Copy of this collection, sorted by a key function.

contains(other)[source]¶

Contains is equivalient to the Python in operator.

Parameters:	other (Proxytype) – A Proxytype or type that can be promoted to a Proxytype
Returns:	A Bool Proxytype
Return type:	Bool

Example

>>> from descarteslabs.workflows import List, Int
>>> my_list = List[Int]([1, 2, 3])
>>> my_list.contains(2).compute() 
True

filter(func)[source]¶

Filter elements from an iterable proxytype.

Parameters:	func (Python callable) – A function that takes a single argument and returns a `Bool` Proxytype.
Returns:	A Proxytype of the same type having only the elements where `func` returns `True`.
Return type:	Proxtype

Example

>>> from descarteslabs.workflows import ImageCollection
>>> col = ImageCollection.from_id("sentinel-2:L1C")
>>> filtered = col.filter(lambda img: img.properties["date"].year == 2018)

length()[source]¶

Length is equivalent to the Python len operator.

Returns:	An Int Proxytype
Return type:	Int

Example

>>> from descarteslabs.workflows import List, Int
>>> my_list = List[Int]([1, 2, 3])
>>> my_list.length().compute() 
3

map(func)[source]¶

Map a function over an iterable proxytype.

Parameters:	func (Python callable) – A function that takes a single argument and returns another proxytype.
Returns:	The return type is dependent on the `func` and the type of the element returned. For example, calling `map` over an `ImageCollection` with a function that returns the date of the `Image` will now be a `List[Datetime]`.
Return type:	Proxtype

Example

>>> from descarteslabs.workflows import ImageCollection
>>> col = ImageCollection.from_id("sentinel-2:L1C")
>>> dates = col.map(lambda img: img.properties["date"])
>>> type(dates).__name__
'List[Datetime]'

reduce(func, initial='__NO_INITIAL_REDUCE_VALUE__')[source]¶

Reduce a collection of elements to a single element.

NOTE: Optimized reducers such as sum have been implemented for some classes.

Parameters:	func (Python callable) – A function where the left argument is the accumulated value (the result of the previous call to `func`) and the right argument is the next value from the iterable. The function should return a single proxtype. initial (Proxytype, optional) – An optional proxtype to provide for the first iteration. If no value is provided, the first element will be used.
Returns:	The return value of `func` for the last iteration of the collection.
Return type:	Proxytype

Example

>>> from descarteslabs.workflows import List, Int
>>> def add(accumulated, current):
...     return accumulated + current
>>> my_list = List[Int]([1, 2])
>>> my_list.reduce(add, initial=Int(10)).compute()  
13

sorted(key=None, reverse=False)[source]¶

Copy of this collection, sorted by a key function.

Parameters:	key (Function, optional, default None) – Function which takes an element and returns a value to sort by. If not given, sorts by the elements themselves. reverse (Bool, default False) – Sorts in ascending order if False (default), descending if True.

Example

>>> from descarteslabs.workflows import List, Int
>>> my_list = List[Int]([1, 4, 2, 3])
>>> my_list.sorted().compute() 
[1, 2, 3, 4]

class Dict(*dct, **kwargs)[source]¶

Dict[KeyType, ValueType]: Proxy mapping, from keys of a specific type to values of a specific type.

Can be instantiated from a Python mapping and/or keyword arguments.

NOTE: Proxy types are not hashable and thus can not be used inside regular old Python dicts. In general, prefer the Dict.from_pairs constructor when dealing with proxy types directly.

Examples

>>> from descarteslabs.workflows import Dict, List, Str, Int, Float
>>> Dict[Str, Int](a=1, b=2) # dict of Str to Int
<descarteslabs.workflows.types.containers.dict_.Dict[Str, Int] object at 0x...>
>>> Dict[Str, Int]({'a': 1, 'b': 2}, b=100, c=3) # dict of Str to Int
<descarteslabs.workflows.types.containers.dict_.Dict[Str, Int] object at 0x...>
>>> Dict[Str, List[Float]](a=[1.1, 2.2], b=[3.3]) # dict of Str to List of Floats
<descarteslabs.workflows.types.containers.dict_.Dict[Str, List[Float]] object at 0x...>

>>> from descarteslabs.workflows import Dict, Str, Float
>>> my_dict = Dict[Str, Float]({"red": 100.5, "blue": 67.6})
>>> my_dict
<descarteslabs.workflows.types.containers.dict_.Dict[Str, Float] object at 0x...>
>>> my_dict.compute() 
{"red": 100.5, "blue": 67.6}
>>> my_dict.keys().compute() 
['red', 'blue']
>>> my_dict["red"].compute() 
100.5

Methods:

`compute`([format, destination, file, …])	Compute a proxy object and wait for its result.
`contains`(key)	Whether the dictionary contains the given key.
`from_pairs`(pairs)	Construct a Dict from a list of key-value pairs.
`get`(key, default)	Return the value for `key` if `key` is in the dictionary, else `default`.
`inspect`([format, file, cache, _ruster, …])	Quickly compute a proxy object using a low-latency, lower-reliability backend.
`items`()	List of tuples of key-value pairs in the dictionary.
`keys`()	List of all the dictionary keys.
`length`()	The number of items in the dictionary
`publish`(version[, title, description, …])	Publish a proxy object as a `Workflow` with the given version.
`values`()	List of all the dictionary values.

Attributes:

`key_type`	Type of the keys in the dictionary
`value_type`	Type of the values in the dictionary

compute(format='pyarrow', destination='download', file=None, timeout=None, block=True, progress_bar=None, cache=True, _ruster=None, _trace=False, client=None, num_retries=None, **arguments)¶

Compute a proxy object and wait for its result.

If the caller has too many outstanding compute jobs, this will raise a ResourceExhausted exception.

Parameters:	geoctx (`GeoContext`, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a `GeoContext`, but for operations that only involve non-geospatial types, this parameter is optional. format (Str or Dict, default "pyarrow") – The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes or None. destination (str or dict, default "download") – The destination for the result. See the destinations documentation for more information. file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them. timeout (Int, optional) – The number of seconds to wait for the result, if `block` is True. Raises `JobTimeoutError` if the timeout passes. block (Bool, default True) – If True (default), block until the job is completed, or `timeout` has passed. If False, immediately returns a `Job` (which has already had `execute` called). progress_bar (Bool, default None) – Whether to draw the progress bar. If `None` (default), will display a progress bar in Jupyter Notebooks, but not elsewhere. Ignored if `block==False`. client (`workflows.client.Client`, optional) – Allows you to use a specific client instance with non-default auth and parameters num_retries (Int, optional) – The number of retries to make in the event of a request failure. If you are making numerous long-running asynchronous requests, you can use this parameter as a way to indicate that you are comfortable waiting and retrying in response to RESOURCE EXHAUSTED errors. By default, most failures will trigger a small number of retries, but if you have reached your outstanding job limit, by default, the client will not retry. This parameter is unnecessary when making synchronous `compute` requests (ie. block=True, the default). See the compute section of the Workflows Guide for more information. *arguments (Any*) – Values for all parameters that `obj` depends on (or arguments that `obj` takes, if it’s a `Function`). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.
Returns:	result – When `format="pyarrow"` (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from `descarteslabs.workflows.result_types`. For other formats, returns raw bytes. Consider using `file` in that case to save the results to a file. If the destination doesn’t support retrieving results (like “email”), returns None.
Return type:	Python object, bytes, or None
Raises:	RetryError – Raised if there are too many failed retries. Inspect `RetryError.exceptions` to determine the ultimate cause of the error. If you reach your maximum number of outstanding compute jobs, there will be one or more `ResourceExhausted` exceptions.

contains(key)¶

Whether the dictionary contains the given key.

Parameters:	key – Key to look up. Must be the same type as `self.key_type`.
Returns:
Return type:	Bool

Example

>>> from descarteslabs.workflows import Dict, Str, Int
>>> my_dict = Dict[Str, Int]({"foo": 1, "bar": 2, "baz": 3})
>>> my_dict.contains("foo").compute() 
True
>>> my_dict.contains("hello").compute() 
False

classmethod from_pairs(pairs)[source]¶

Construct a Dict from a list of key-value pairs.

Parameters:	List[Tuple] –
Returns:
Return type:	Dict

Example

>>> from descarteslabs.workflows import Dict, Str, Int
>>> pairs = [("foo", 1), ("bar", 2), ("baz", 3)]
>>> my_dict = Dict[Str, Int].from_pairs(pairs)
>>> my_dict
<descarteslabs.workflows.types.containers.dict_.Dict[Str, Int] object at 0x...>
>>> my_dict.compute() 
{'foo': 1, 'bar': 2, 'baz': 3}

get(key, default)¶

Return the value for key if key is in the dictionary, else default.

Parameters:	key – Key to look up. Must be the same type as `key_type`. default – Value returned if `key` does not exist. Must be the same type as `value_type`.

Example

>>> from descarteslabs.workflows import Dict, Str, Int
>>> my_dict = Dict[Str, Int]({"foo": 1, "bar": 2, "baz": 3})
>>> my_dict.get("baz").compute() 
3

inspect(format='pyarrow', file=None, cache=True, _ruster=None, timeout=60, client=None, **arguments)¶

Quickly compute a proxy object using a low-latency, lower-reliability backend.

Inspect is meant for getting simple computations out of Workflows, primarily for interactive use. It’s quicker but less resilient, won’t be retried if it fails, and has no progress updates.

If you have a larger computation (longer than ~30sec), or you want to be sure the computation will succeed, use compute instead. compute creates a Job, which runs asynchronously, will be retried if it fails, and stores its results for later retrieval.

Parameters:	geoctx (`common.geo.geocontext.GeoContext`, `GeoContext`, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a `GeoContext`, but for operations that only involve non-geospatial types, this parameter is optional. format (str or dict, default "pyarrow") – The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes. file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them. cache (bool, default True) – Whether to use the cache for this job. timeout (int, optional, default 60) – The number of seconds to wait for the result. Raises `JobTimeoutError` if the timeout passes. client (`workflows.inspect.InspectClient`, optional) – Allows you to use a specific InspectClient instance with non-default auth and parameters *arguments (Any*) – Values for all parameters that `obj` depends on (or arguments that `obj` takes, if it’s a `Function`). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.
Returns:	result – When `format="pyarrow"` (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from `descarteslabs.workflows.result_types`. For other formats, returns raw bytes. Consider using `file` in that case to save the results to a file.
Return type:	Python object or bytes

items()¶

List of tuples of key-value pairs in the dictionary.

Returns:
Return type:	List[Tuple[KeyType, ValueType]]

Example

>>> from descarteslabs.workflows import Dict, Str, Int
>>> my_dict = Dict[Str, Int]({"foo": 1, "bar": 2, "baz": 3})
>>> my_dict.items().compute() 
[('foo', 1), ('bar', 2), ('baz', 3)]

keys()¶

List of all the dictionary keys.

Returns:
Return type:	List

Example

>>> from descarteslabs.workflows import Dict, Str, Int
>>> my_dict = Dict[Str, Int]({"foo": 1, "bar": 2, "baz": 3})
>>> my_dict.keys().compute() 
['foo', 'bar', 'baz']

length()¶

The number of items in the dictionary

Returns:
Return type:	Int

Example

>>> from descarteslabs.workflows import Dict, Str, Int
>>> my_dict = Dict[Str, Int]({"foo": 1, "bar": 2, "baz": 3})
>>> my_dict.length().compute() 
3

publish(version, title='', description='', labels=None, tags=None, docstring='', version_labels=None, viz_options=None, client=None)¶

Publish a proxy object as a Workflow with the given version.

If the proxy object depends on any parameters (obj.params is not empty), it’s first internally converted to a Function that takes those parameters (using Function.from_object).

Parameters:	id (Str) – ID for the new Workflow object. This should be of the form `email:workflow_name` and should be globally unique. If this ID is not of the proper format, you will not be able to save the Workflow. version (Str) – The version to be set, tied to the given `obj`. This should adhere to the semantic versioning schema. title (Str, default "") – User-friendly title for the `Workflow`. description (str, default "") – Long-form description of this `Workflow`. Markdown is supported. labels (Dict, optional) – Key-value pair labels to add to the `Workflow`. tags (list, optional) – A list of strings to add as tags to the `Workflow`. docstring (Str, default "") – The docstring for this version. version_labels (Dict, optional) – Key-value pair labels to add to the version. client (`workflows.client.Client`, optional) – Allows you to use a specific client instance with non-default auth and parameters
Returns:	workflow – The saved `Workflow` object. `workflow.id` contains the ID of the new Workflow.
Return type:	Workflow or Function

values()¶

List of all the dictionary values.

Returns:
Return type:	List

Example

>>> from descarteslabs.workflows import Dict, Str, Int
>>> my_dict = Dict[Str, Int]({"foo": 1, "bar": 2, "baz": 3})
>>> my_dict.values().compute() 
[1, 2, 3]

property key_type¶: Type of the keys in the dictionary

property value_type¶: Type of the values in the dictionary

class Ellipsis[source]¶

Proxy Ellipsis object. You shouldn’t use this directly; Python ellipsis can be used to achieve the same result.

Methods:

`compute`(args, *kwargs)	Compute a proxy object and wait for its result.
`inspect`([format, file, cache, _ruster, …])	Quickly compute a proxy object using a low-latency, lower-reliability backend.
`publish`(version[, title, description, …])	Publish a proxy object as a `Workflow` with the given version.

compute(*args, **kwargs)[source]¶

Compute a proxy object and wait for its result.

If the caller has too many outstanding compute jobs, this will raise a ResourceExhausted exception.

Parameters:	geoctx (`GeoContext`, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a `GeoContext`, but for operations that only involve non-geospatial types, this parameter is optional. format (Str or Dict, default "pyarrow") – The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes or None. destination (str or dict, default "download") – The destination for the result. See the destinations documentation for more information. file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them. timeout (Int, optional) – The number of seconds to wait for the result, if `block` is True. Raises `JobTimeoutError` if the timeout passes. block (Bool, default True) – If True (default), block until the job is completed, or `timeout` has passed. If False, immediately returns a `Job` (which has already had `execute` called). progress_bar (Bool, default None) – Whether to draw the progress bar. If `None` (default), will display a progress bar in Jupyter Notebooks, but not elsewhere. Ignored if `block==False`. client (`workflows.client.Client`, optional) – Allows you to use a specific client instance with non-default auth and parameters num_retries (Int, optional) – The number of retries to make in the event of a request failure. If you are making numerous long-running asynchronous requests, you can use this parameter as a way to indicate that you are comfortable waiting and retrying in response to RESOURCE EXHAUSTED errors. By default, most failures will trigger a small number of retries, but if you have reached your outstanding job limit, by default, the client will not retry. This parameter is unnecessary when making synchronous `compute` requests (ie. block=True, the default). See the compute section of the Workflows Guide for more information. *arguments (Any*) – Values for all parameters that `obj` depends on (or arguments that `obj` takes, if it’s a `Function`). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.
Returns:	result – When `format="pyarrow"` (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from `descarteslabs.workflows.result_types`. For other formats, returns raw bytes. Consider using `file` in that case to save the results to a file. If the destination doesn’t support retrieving results (like “email”), returns None.
Return type:	Python object, bytes, or None
Raises:	RetryError – Raised if there are too many failed retries. Inspect `RetryError.exceptions` to determine the ultimate cause of the error. If you reach your maximum number of outstanding compute jobs, there will be one or more `ResourceExhausted` exceptions.

inspect(format='pyarrow', file=None, cache=True, _ruster=None, timeout=60, client=None, **arguments)¶

Quickly compute a proxy object using a low-latency, lower-reliability backend.

Inspect is meant for getting simple computations out of Workflows, primarily for interactive use. It’s quicker but less resilient, won’t be retried if it fails, and has no progress updates.

If you have a larger computation (longer than ~30sec), or you want to be sure the computation will succeed, use compute instead. compute creates a Job, which runs asynchronously, will be retried if it fails, and stores its results for later retrieval.

Parameters:	geoctx (`common.geo.geocontext.GeoContext`, `GeoContext`, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a `GeoContext`, but for operations that only involve non-geospatial types, this parameter is optional. format (str or dict, default "pyarrow") – The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes. file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them. cache (bool, default True) – Whether to use the cache for this job. timeout (int, optional, default 60) – The number of seconds to wait for the result. Raises `JobTimeoutError` if the timeout passes. client (`workflows.inspect.InspectClient`, optional) – Allows you to use a specific InspectClient instance with non-default auth and parameters *arguments (Any*) – Values for all parameters that `obj` depends on (or arguments that `obj` takes, if it’s a `Function`). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.
Returns:	result – When `format="pyarrow"` (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from `descarteslabs.workflows.result_types`. For other formats, returns raw bytes. Consider using `file` in that case to save the results to a file.
Return type:	Python object or bytes

publish(version, title='', description='', labels=None, tags=None, docstring='', version_labels=None, viz_options=None, client=None)¶

Publish a proxy object as a Workflow with the given version.

If the proxy object depends on any parameters (obj.params is not empty), it’s first internally converted to a Function that takes those parameters (using Function.from_object).

Parameters:	id (Str) – ID for the new Workflow object. This should be of the form `email:workflow_name` and should be globally unique. If this ID is not of the proper format, you will not be able to save the Workflow. version (Str) – The version to be set, tied to the given `obj`. This should adhere to the semantic versioning schema. title (Str, default "") – User-friendly title for the `Workflow`. description (str, default "") – Long-form description of this `Workflow`. Markdown is supported. labels (Dict, optional) – Key-value pair labels to add to the `Workflow`. tags (list, optional) – A list of strings to add as tags to the `Workflow`. docstring (Str, default "") – The docstring for this version. version_labels (Dict, optional) – Key-value pair labels to add to the version. client (`workflows.client.Client`, optional) – Allows you to use a specific client instance with non-default auth and parameters
Returns:	workflow – The saved `Workflow` object. `workflow.id` contains the ID of the new Workflow.
Return type:	Workflow or Function

class List(iterable)[source]¶

List[ValueType]: Proxy sequence of any number of elements, all of the same type.

Can be instantiated from any Python iterable, or another List of the same type.

Examples

>>> from descarteslabs.workflows import List, Str, Int
>>> List[Str](["foo", "bar", "baz"]) # list of Strs
<descarteslabs.workflows.types.containers.list_.List[Str] object at 0x...>
>>> List[List[Int]]([[1, 2], [-1], [10, 11, 12]]) # list of lists of Ints
<descarteslabs.workflows.types.containers.list_.List[List[Int]] object at 0x...>

>>> from descarteslabs.workflows import List, Float
>>> my_list = List[Float]([1.1, 2.2, 3.3, 4.4])
>>> my_list
<descarteslabs.workflows.types.containers.list_.List[Float] object at 0x...>
>>> my_list.compute() 
[1.1, 2.2, 3.3, 4.4]
>>> my_list[2].compute() 
3.3
>>> (my_list * 2).compute() 
[1.1, 2.2, 3.3, 4.4, 1.1, 2.2, 3.3, 4.4]
>>> (my_list == my_list).compute() 
True

Methods:

`compute`([format, destination, file, …])	Compute a proxy object and wait for its result.
`contains`(other)	Contains is equivalient to the Python `in` operator.
`filter`(func)	Filter elements from an iterable proxytype.
`inspect`([format, file, cache, _ruster, …])	Quickly compute a proxy object using a low-latency, lower-reliability backend.
`length`()	Length is equivalent to the Python `len` operator.
`map`(func)	Map a function over an iterable proxytype.
`publish`(version[, title, description, …])	Publish a proxy object as a `Workflow` with the given version.
`reduce`(func[, initial])	Reduce a collection of elements to a single element.
`sorted`([key, reverse])	Copy of this collection, sorted by a key function.

compute(format='pyarrow', destination='download', file=None, timeout=None, block=True, progress_bar=None, cache=True, _ruster=None, _trace=False, client=None, num_retries=None, **arguments)¶

Compute a proxy object and wait for its result.

If the caller has too many outstanding compute jobs, this will raise a ResourceExhausted exception.

Parameters:	geoctx (`GeoContext`, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a `GeoContext`, but for operations that only involve non-geospatial types, this parameter is optional. format (Str or Dict, default "pyarrow") – The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes or None. destination (str or dict, default "download") – The destination for the result. See the destinations documentation for more information. file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them. timeout (Int, optional) – The number of seconds to wait for the result, if `block` is True. Raises `JobTimeoutError` if the timeout passes. block (Bool, default True) – If True (default), block until the job is completed, or `timeout` has passed. If False, immediately returns a `Job` (which has already had `execute` called). progress_bar (Bool, default None) – Whether to draw the progress bar. If `None` (default), will display a progress bar in Jupyter Notebooks, but not elsewhere. Ignored if `block==False`. client (`workflows.client.Client`, optional) – Allows you to use a specific client instance with non-default auth and parameters num_retries (Int, optional) – The number of retries to make in the event of a request failure. If you are making numerous long-running asynchronous requests, you can use this parameter as a way to indicate that you are comfortable waiting and retrying in response to RESOURCE EXHAUSTED errors. By default, most failures will trigger a small number of retries, but if you have reached your outstanding job limit, by default, the client will not retry. This parameter is unnecessary when making synchronous `compute` requests (ie. block=True, the default). See the compute section of the Workflows Guide for more information. *arguments (Any*) – Values for all parameters that `obj` depends on (or arguments that `obj` takes, if it’s a `Function`). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.
Returns:	result – When `format="pyarrow"` (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from `descarteslabs.workflows.result_types`. For other formats, returns raw bytes. Consider using `file` in that case to save the results to a file. If the destination doesn’t support retrieving results (like “email”), returns None.
Return type:	Python object, bytes, or None
Raises:	RetryError – Raised if there are too many failed retries. Inspect `RetryError.exceptions` to determine the ultimate cause of the error. If you reach your maximum number of outstanding compute jobs, there will be one or more `ResourceExhausted` exceptions.

contains(other)¶

Contains is equivalient to the Python in operator.

Parameters:	other (Proxytype) – A Proxytype or type that can be promoted to a Proxytype
Returns:	A Bool Proxytype
Return type:	Bool

Example

>>> from descarteslabs.workflows import List, Int
>>> my_list = List[Int]([1, 2, 3])
>>> my_list.contains(2).compute() 
True

filter(func)¶

Filter elements from an iterable proxytype.

Parameters:	func (Python callable) – A function that takes a single argument and returns a `Bool` Proxytype.
Returns:	A Proxytype of the same type having only the elements where `func` returns `True`.
Return type:	Proxtype

Example

>>> from descarteslabs.workflows import ImageCollection
>>> col = ImageCollection.from_id("sentinel-2:L1C")
>>> filtered = col.filter(lambda img: img.properties["date"].year == 2018)

inspect(format='pyarrow', file=None, cache=True, _ruster=None, timeout=60, client=None, **arguments)¶

Quickly compute a proxy object using a low-latency, lower-reliability backend.

Inspect is meant for getting simple computations out of Workflows, primarily for interactive use. It’s quicker but less resilient, won’t be retried if it fails, and has no progress updates.

If you have a larger computation (longer than ~30sec), or you want to be sure the computation will succeed, use compute instead. compute creates a Job, which runs asynchronously, will be retried if it fails, and stores its results for later retrieval.

Parameters:	geoctx (`common.geo.geocontext.GeoContext`, `GeoContext`, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a `GeoContext`, but for operations that only involve non-geospatial types, this parameter is optional. format (str or dict, default "pyarrow") – The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes. file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them. cache (bool, default True) – Whether to use the cache for this job. timeout (int, optional, default 60) – The number of seconds to wait for the result. Raises `JobTimeoutError` if the timeout passes. client (`workflows.inspect.InspectClient`, optional) – Allows you to use a specific InspectClient instance with non-default auth and parameters *arguments (Any*) – Values for all parameters that `obj` depends on (or arguments that `obj` takes, if it’s a `Function`). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.
Returns:	result – When `format="pyarrow"` (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from `descarteslabs.workflows.result_types`. For other formats, returns raw bytes. Consider using `file` in that case to save the results to a file.
Return type:	Python object or bytes

length()¶

Length is equivalent to the Python len operator.

Returns:	An Int Proxytype
Return type:	Int

Example

>>> from descarteslabs.workflows import List, Int
>>> my_list = List[Int]([1, 2, 3])
>>> my_list.length().compute() 
3

map(func)¶

Map a function over an iterable proxytype.

Parameters:	func (Python callable) – A function that takes a single argument and returns another proxytype.
Returns:	The return type is dependent on the `func` and the type of the element returned. For example, calling `map` over an `ImageCollection` with a function that returns the date of the `Image` will now be a `List[Datetime]`.
Return type:	Proxtype

Example

>>> from descarteslabs.workflows import ImageCollection
>>> col = ImageCollection.from_id("sentinel-2:L1C")
>>> dates = col.map(lambda img: img.properties["date"])
>>> type(dates).__name__
'List[Datetime]'

publish(version, title='', description='', labels=None, tags=None, docstring='', version_labels=None, viz_options=None, client=None)¶

Publish a proxy object as a Workflow with the given version.

If the proxy object depends on any parameters (obj.params is not empty), it’s first internally converted to a Function that takes those parameters (using Function.from_object).

Parameters:	id (Str) – ID for the new Workflow object. This should be of the form `email:workflow_name` and should be globally unique. If this ID is not of the proper format, you will not be able to save the Workflow. version (Str) – The version to be set, tied to the given `obj`. This should adhere to the semantic versioning schema. title (Str, default "") – User-friendly title for the `Workflow`. description (str, default "") – Long-form description of this `Workflow`. Markdown is supported. labels (Dict, optional) – Key-value pair labels to add to the `Workflow`. tags (list, optional) – A list of strings to add as tags to the `Workflow`. docstring (Str, default "") – The docstring for this version. version_labels (Dict, optional) – Key-value pair labels to add to the version. client (`workflows.client.Client`, optional) – Allows you to use a specific client instance with non-default auth and parameters
Returns:	workflow – The saved `Workflow` object. `workflow.id` contains the ID of the new Workflow.
Return type:	Workflow or Function

reduce(func, initial='__NO_INITIAL_REDUCE_VALUE__')¶

Reduce a collection of elements to a single element.

NOTE: Optimized reducers such as sum have been implemented for some classes.

Parameters:	func (Python callable) – A function where the left argument is the accumulated value (the result of the previous call to `func`) and the right argument is the next value from the iterable. The function should return a single proxtype. initial (Proxytype, optional) – An optional proxtype to provide for the first iteration. If no value is provided, the first element will be used.
Returns:	The return value of `func` for the last iteration of the collection.
Return type:	Proxytype

Example

>>> from descarteslabs.workflows import List, Int
>>> def add(accumulated, current):
...     return accumulated + current
>>> my_list = List[Int]([1, 2])
>>> my_list.reduce(add, initial=Int(10)).compute()  
13

sorted(key=None, reverse=False)¶

Copy of this collection, sorted by a key function.

Parameters:	key (Function, optional, default None) – Function which takes an element and returns a value to sort by. If not given, sorts by the elements themselves. reverse (Bool, default False) – Sorts in ascending order if False (default), descending if True.

Example

>>> from descarteslabs.workflows import List, Int
>>> my_list = List[Int]([1, 4, 2, 3])
>>> my_list.sorted().compute() 
[1, 2, 3, 4]

class Slice(start=None, stop=None, step=None)[source]¶

Proxy Slice object. Internal class used as a promotion target for Python slices. You shouldn’t use this directly; Python slices can be used to achieve the same result.

Examples

>>> from descarteslabs.workflows import List, Int
>>> from descarteslabs.workflows.types.containers import Slice
>>> my_slice = Slice(0, 4, 2)
>>> my_slice
<descarteslabs.workflows.types.containers.slice.Slice object at 0x...>
>>> my_slice.compute() 
slice(0, 4, 2)
>>> my_list = List[Int]([1, 2, 3, 4, 5])
>>> my_list[my_slice].compute() 
[1, 3]

Methods:

`compute`([format, destination, file, …])	Compute a proxy object and wait for its result.
`from_slice`(obj)	Construct a Workflows Slice from a Python slice object.
`inspect`([format, file, cache, _ruster, …])	Quickly compute a proxy object using a low-latency, lower-reliability backend.
`publish`(version[, title, description, …])	Publish a proxy object as a `Workflow` with the given version.

Attributes:

literal_value Python literal value this proxy object was constructed with, or None if not constructed from a literal value.

compute(format='pyarrow', destination='download', file=None, timeout=None, block=True, progress_bar=None, cache=True, _ruster=None, _trace=False, client=None, num_retries=None, **arguments)¶

Compute a proxy object and wait for its result.

If the caller has too many outstanding compute jobs, this will raise a ResourceExhausted exception.

Parameters:	geoctx (`GeoContext`, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a `GeoContext`, but for operations that only involve non-geospatial types, this parameter is optional. format (Str or Dict, default "pyarrow") – The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes or None. destination (str or dict, default "download") – The destination for the result. See the destinations documentation for more information. file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them. timeout (Int, optional) – The number of seconds to wait for the result, if `block` is True. Raises `JobTimeoutError` if the timeout passes. block (Bool, default True) – If True (default), block until the job is completed, or `timeout` has passed. If False, immediately returns a `Job` (which has already had `execute` called). progress_bar (Bool, default None) – Whether to draw the progress bar. If `None` (default), will display a progress bar in Jupyter Notebooks, but not elsewhere. Ignored if `block==False`. client (`workflows.client.Client`, optional) – Allows you to use a specific client instance with non-default auth and parameters num_retries (Int, optional) – The number of retries to make in the event of a request failure. If you are making numerous long-running asynchronous requests, you can use this parameter as a way to indicate that you are comfortable waiting and retrying in response to RESOURCE EXHAUSTED errors. By default, most failures will trigger a small number of retries, but if you have reached your outstanding job limit, by default, the client will not retry. This parameter is unnecessary when making synchronous `compute` requests (ie. block=True, the default). See the compute section of the Workflows Guide for more information. *arguments (Any*) – Values for all parameters that `obj` depends on (or arguments that `obj` takes, if it’s a `Function`). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.
Returns:	result – When `format="pyarrow"` (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from `descarteslabs.workflows.result_types`. For other formats, returns raw bytes. Consider using `file` in that case to save the results to a file. If the destination doesn’t support retrieving results (like “email”), returns None.
Return type:	Python object, bytes, or None
Raises:	RetryError – Raised if there are too many failed retries. Inspect `RetryError.exceptions` to determine the ultimate cause of the error. If you reach your maximum number of outstanding compute jobs, there will be one or more `ResourceExhausted` exceptions.

classmethod from_slice(obj)[source]¶

Construct a Workflows Slice from a Python slice object.

Example

>>> from descarteslabs.workflows.types.containers import Slice
>>> py_slice = slice(0, 4, 2)
>>> my_slice = Slice.from_slice(py_slice)
>>> my_slice.compute() 
slice(0, 4, 2)

inspect(format='pyarrow', file=None, cache=True, _ruster=None, timeout=60, client=None, **arguments)¶

Quickly compute a proxy object using a low-latency, lower-reliability backend.

Inspect is meant for getting simple computations out of Workflows, primarily for interactive use. It’s quicker but less resilient, won’t be retried if it fails, and has no progress updates.

If you have a larger computation (longer than ~30sec), or you want to be sure the computation will succeed, use compute instead. compute creates a Job, which runs asynchronously, will be retried if it fails, and stores its results for later retrieval.

Parameters:	geoctx (`common.geo.geocontext.GeoContext`, `GeoContext`, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a `GeoContext`, but for operations that only involve non-geospatial types, this parameter is optional. format (str or dict, default "pyarrow") – The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes. file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them. cache (bool, default True) – Whether to use the cache for this job. timeout (int, optional, default 60) – The number of seconds to wait for the result. Raises `JobTimeoutError` if the timeout passes. client (`workflows.inspect.InspectClient`, optional) – Allows you to use a specific InspectClient instance with non-default auth and parameters *arguments (Any*) – Values for all parameters that `obj` depends on (or arguments that `obj` takes, if it’s a `Function`). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.
Returns:	result – When `format="pyarrow"` (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from `descarteslabs.workflows.result_types`. For other formats, returns raw bytes. Consider using `file` in that case to save the results to a file.
Return type:	Python object or bytes

publish(version, title='', description='', labels=None, tags=None, docstring='', version_labels=None, viz_options=None, client=None)¶

Publish a proxy object as a Workflow with the given version.

If the proxy object depends on any parameters (obj.params is not empty), it’s first internally converted to a Function that takes those parameters (using Function.from_object).

Parameters:	id (Str) – ID for the new Workflow object. This should be of the form `email:workflow_name` and should be globally unique. If this ID is not of the proper format, you will not be able to save the Workflow. version (Str) – The version to be set, tied to the given `obj`. This should adhere to the semantic versioning schema. title (Str, default "") – User-friendly title for the `Workflow`. description (str, default "") – Long-form description of this `Workflow`. Markdown is supported. labels (Dict, optional) – Key-value pair labels to add to the `Workflow`. tags (list, optional) – A list of strings to add as tags to the `Workflow`. docstring (Str, default "") – The docstring for this version. version_labels (Dict, optional) – Key-value pair labels to add to the version. client (`workflows.client.Client`, optional) – Allows you to use a specific client instance with non-default auth and parameters
Returns:	workflow – The saved `Workflow` object. `workflow.id` contains the ID of the new Workflow.
Return type:	Workflow or Function

property literal_value¶: Python literal value this proxy object was constructed with, or None if not constructed from a literal value.

class Struct(**kwargs)[source]¶

Struct[{field_name: type, ...}]: Proxy container with named fields of specific types, meant as a helper base class.

Can be instantiated from kwargs.

In general, Struct is used as an internal base class to help create proxy objects, and is not meant to be worked with directly.

Notes

Adding certain fields to subclasses of concrete Structs changes their behavior:

_doc (dict of str): Docstrings for struct fields. The keys must match keys in the typespec (though not all fields must be documented). Resulting docstrings are formatted like "{}: {}".format(return_type_name, given_docstring). If no docstring is given, the field getter will just be documented with its return type.
_constructor (str): Function name to use in graft for instantiation
_optional (set of str, or None): Optional field names for constructor
_read_only: (set of str, or None): Field names that can’t be given to constructor

Examples

>>> from descarteslabs.workflows import Struct, Str, Int
>>> MyStructType = Struct[{'x': Str, 'y': Int}] # struct where type of 'x' is Str and type of 'y' is Int
>>> instance = MyStructType(x="foo", y=1)
>>> instance
<descarteslabs.workflows.types.containers.struct.Struct[{'x': Str, 'y': Int}] object at 0x...>

>>> from descarteslabs.workflows import Struct, Str, Int
>>> class Foo(Struct[{'x': Int, 'y': Str}]): 
...     _doc = {
...         'x': "the x field",
...         'y': "the y field, derived from x",
...     }
...     _constructor = "make_foo"
...     _optional = {'x'}
...     _read_only = {'y'}
>>> my_struct = Foo(10) 
>>> my_struct.x.compute() 
10

Methods:

`compute`([format, destination, file, …])	Compute a proxy object and wait for its result.
`inspect`([format, file, cache, _ruster, …])	Quickly compute a proxy object using a low-latency, lower-reliability backend.
`publish`(version[, title, description, …])	Publish a proxy object as a `Workflow` with the given version.

compute(format='pyarrow', destination='download', file=None, timeout=None, block=True, progress_bar=None, cache=True, _ruster=None, _trace=False, client=None, num_retries=None, **arguments)¶

Compute a proxy object and wait for its result.

If the caller has too many outstanding compute jobs, this will raise a ResourceExhausted exception.

Parameters:	geoctx (`GeoContext`, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a `GeoContext`, but for operations that only involve non-geospatial types, this parameter is optional. format (Str or Dict, default "pyarrow") – The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes or None. destination (str or dict, default "download") – The destination for the result. See the destinations documentation for more information. file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them. timeout (Int, optional) – The number of seconds to wait for the result, if `block` is True. Raises `JobTimeoutError` if the timeout passes. block (Bool, default True) – If True (default), block until the job is completed, or `timeout` has passed. If False, immediately returns a `Job` (which has already had `execute` called). progress_bar (Bool, default None) – Whether to draw the progress bar. If `None` (default), will display a progress bar in Jupyter Notebooks, but not elsewhere. Ignored if `block==False`. client (`workflows.client.Client`, optional) – Allows you to use a specific client instance with non-default auth and parameters num_retries (Int, optional) – The number of retries to make in the event of a request failure. If you are making numerous long-running asynchronous requests, you can use this parameter as a way to indicate that you are comfortable waiting and retrying in response to RESOURCE EXHAUSTED errors. By default, most failures will trigger a small number of retries, but if you have reached your outstanding job limit, by default, the client will not retry. This parameter is unnecessary when making synchronous `compute` requests (ie. block=True, the default). See the compute section of the Workflows Guide for more information. *arguments (Any*) – Values for all parameters that `obj` depends on (or arguments that `obj` takes, if it’s a `Function`). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.
Returns:	result – When `format="pyarrow"` (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from `descarteslabs.workflows.result_types`. For other formats, returns raw bytes. Consider using `file` in that case to save the results to a file. If the destination doesn’t support retrieving results (like “email”), returns None.
Return type:	Python object, bytes, or None
Raises:	RetryError – Raised if there are too many failed retries. Inspect `RetryError.exceptions` to determine the ultimate cause of the error. If you reach your maximum number of outstanding compute jobs, there will be one or more `ResourceExhausted` exceptions.

inspect(format='pyarrow', file=None, cache=True, _ruster=None, timeout=60, client=None, **arguments)¶

Quickly compute a proxy object using a low-latency, lower-reliability backend.

Inspect is meant for getting simple computations out of Workflows, primarily for interactive use. It’s quicker but less resilient, won’t be retried if it fails, and has no progress updates.

If you have a larger computation (longer than ~30sec), or you want to be sure the computation will succeed, use compute instead. compute creates a Job, which runs asynchronously, will be retried if it fails, and stores its results for later retrieval.

Parameters:	geoctx (`common.geo.geocontext.GeoContext`, `GeoContext`, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a `GeoContext`, but for operations that only involve non-geospatial types, this parameter is optional. format (str or dict, default "pyarrow") – The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes. file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them. cache (bool, default True) – Whether to use the cache for this job. timeout (int, optional, default 60) – The number of seconds to wait for the result. Raises `JobTimeoutError` if the timeout passes. client (`workflows.inspect.InspectClient`, optional) – Allows you to use a specific InspectClient instance with non-default auth and parameters *arguments (Any*) – Values for all parameters that `obj` depends on (or arguments that `obj` takes, if it’s a `Function`). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.
Returns:	result – When `format="pyarrow"` (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from `descarteslabs.workflows.result_types`. For other formats, returns raw bytes. Consider using `file` in that case to save the results to a file.
Return type:	Python object or bytes

publish(version, title='', description='', labels=None, tags=None, docstring='', version_labels=None, viz_options=None, client=None)¶

Publish a proxy object as a Workflow with the given version.

If the proxy object depends on any parameters (obj.params is not empty), it’s first internally converted to a Function that takes those parameters (using Function.from_object).

Parameters:	id (Str) – ID for the new Workflow object. This should be of the form `email:workflow_name` and should be globally unique. If this ID is not of the proper format, you will not be able to save the Workflow. version (Str) – The version to be set, tied to the given `obj`. This should adhere to the semantic versioning schema. title (Str, default "") – User-friendly title for the `Workflow`. description (str, default "") – Long-form description of this `Workflow`. Markdown is supported. labels (Dict, optional) – Key-value pair labels to add to the `Workflow`. tags (list, optional) – A list of strings to add as tags to the `Workflow`. docstring (Str, default "") – The docstring for this version. version_labels (Dict, optional) – Key-value pair labels to add to the version. client (`workflows.client.Client`, optional) – Allows you to use a specific client instance with non-default auth and parameters
Returns:	workflow – The saved `Workflow` object. `workflow.id` contains the ID of the new Workflow.
Return type:	Workflow or Function

class Tuple(iterable)[source]¶

Tuple[item1_type, item2_type, ...]: Proxy sequence of a fixed number of elements of specific types. A type must be specified for each item in the tuple.

Can be instantiated from any Python iterable.

When indexing with a modified proxy integer, the return type will be Tuple[Any] because the type of the value at the index is unknown (the index is unknown), but we know the length is 1. When slicing a Tuple with modified proxy integers, an Any will be returned because the types of the values at the indices is unknown, and the length of the resulting Tuple is unknown.

Note: A modified proxy integer is a proxy integer that has been changed through other operations (wf.Int(1) + 1 is a proxy Int that has been modified with an addition, wf.Int(2) has not been modified)

Examples

>>> from descarteslabs.workflows import Tuple, Int, Float, Str, Bool
>>> Tuple[Int, Float]([1, 2.2]) # 2-tuple of Int and Float
<descarteslabs.workflows.types.containers.tuple_.Tuple[Int, Float] object at 0x...>
>>> Tuple[Int]([1]) # 1-tuple of Int (NOT a variable length tuple of Ints)
<descarteslabs.workflows.types.containers.tuple_.Tuple[Int] object at 0x...>
>>> Tuple[Float, Float, Float]([1.1, 2.2, 3.3]) # 3-tuple of Floats
<descarteslabs.workflows.types.containers.tuple_.Tuple[Float, Float, Float] object at 0x...>
>>> Tuple[Str, Tuple[Int, Bool]](["foo", (1, True)]) # 2-tuple of Str and 2-tuple of Int and Bool
<descarteslabs.workflows.types.containers.tuple_.Tuple[Str, Tuple[Int, Bool]] object at 0x...>

>>> from descarteslabs.workflows import Tuple, Str
>>> my_tuple = Tuple[Str, Str](["hello", "world"])
>>> my_tuple
<descarteslabs.workflows.types.containers.tuple_.Tuple[Str, Str] object at 0x...>
>>> my_tuple.compute() 
('hello', 'world')
>>> my_tuple[0].compute() 
'hello'

>>> from descarteslabs.workflows import Tuple, Int, Float, Str
>>> tuple_a = Tuple[Int, Float]([1, 2.2])
>>> tuple_b = Tuple[Str, Float](["foo", 3.3])
>>> tuple_a + tuple_b
<descarteslabs.workflows.types.containers.tuple_.Tuple[Int, Float, Str, Float] object at 0x...>
>>> (tuple_a + ("x", False)).compute() 
(1, 2.2, "x", False)

Methods:

`compute`([format, destination, file, …])	Compute a proxy object and wait for its result.
`inspect`([format, file, cache, _ruster, …])	Quickly compute a proxy object using a low-latency, lower-reliability backend.
`length`()	Length is equivalent to the Python `len` operator.
`publish`(version[, title, description, …])	Publish a proxy object as a `Workflow` with the given version.

compute(format='pyarrow', destination='download', file=None, timeout=None, block=True, progress_bar=None, cache=True, _ruster=None, _trace=False, client=None, num_retries=None, **arguments)¶

Compute a proxy object and wait for its result.

If the caller has too many outstanding compute jobs, this will raise a ResourceExhausted exception.

Parameters:	geoctx (`GeoContext`, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a `GeoContext`, but for operations that only involve non-geospatial types, this parameter is optional. format (Str or Dict, default "pyarrow") – The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes or None. destination (str or dict, default "download") – The destination for the result. See the destinations documentation for more information. file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them. timeout (Int, optional) – The number of seconds to wait for the result, if `block` is True. Raises `JobTimeoutError` if the timeout passes. block (Bool, default True) – If True (default), block until the job is completed, or `timeout` has passed. If False, immediately returns a `Job` (which has already had `execute` called). progress_bar (Bool, default None) – Whether to draw the progress bar. If `None` (default), will display a progress bar in Jupyter Notebooks, but not elsewhere. Ignored if `block==False`. client (`workflows.client.Client`, optional) – Allows you to use a specific client instance with non-default auth and parameters num_retries (Int, optional) – The number of retries to make in the event of a request failure. If you are making numerous long-running asynchronous requests, you can use this parameter as a way to indicate that you are comfortable waiting and retrying in response to RESOURCE EXHAUSTED errors. By default, most failures will trigger a small number of retries, but if you have reached your outstanding job limit, by default, the client will not retry. This parameter is unnecessary when making synchronous `compute` requests (ie. block=True, the default). See the compute section of the Workflows Guide for more information. *arguments (Any*) – Values for all parameters that `obj` depends on (or arguments that `obj` takes, if it’s a `Function`). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.
Returns:	result – When `format="pyarrow"` (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from `descarteslabs.workflows.result_types`. For other formats, returns raw bytes. Consider using `file` in that case to save the results to a file. If the destination doesn’t support retrieving results (like “email”), returns None.
Return type:	Python object, bytes, or None
Raises:	RetryError – Raised if there are too many failed retries. Inspect `RetryError.exceptions` to determine the ultimate cause of the error. If you reach your maximum number of outstanding compute jobs, there will be one or more `ResourceExhausted` exceptions.

inspect(format='pyarrow', file=None, cache=True, _ruster=None, timeout=60, client=None, **arguments)¶

Quickly compute a proxy object using a low-latency, lower-reliability backend.

Inspect is meant for getting simple computations out of Workflows, primarily for interactive use. It’s quicker but less resilient, won’t be retried if it fails, and has no progress updates.

If you have a larger computation (longer than ~30sec), or you want to be sure the computation will succeed, use compute instead. compute creates a Job, which runs asynchronously, will be retried if it fails, and stores its results for later retrieval.

Parameters:	geoctx (`common.geo.geocontext.GeoContext`, `GeoContext`, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a `GeoContext`, but for operations that only involve non-geospatial types, this parameter is optional. format (str or dict, default "pyarrow") – The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes. file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them. cache (bool, default True) – Whether to use the cache for this job. timeout (int, optional, default 60) – The number of seconds to wait for the result. Raises `JobTimeoutError` if the timeout passes. client (`workflows.inspect.InspectClient`, optional) – Allows you to use a specific InspectClient instance with non-default auth and parameters *arguments (Any*) – Values for all parameters that `obj` depends on (or arguments that `obj` takes, if it’s a `Function`). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.
Returns:	result – When `format="pyarrow"` (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from `descarteslabs.workflows.result_types`. For other formats, returns raw bytes. Consider using `file` in that case to save the results to a file.
Return type:	Python object or bytes

length()[source]¶

Length is equivalent to the Python len operator.

Returns:	An Int Proxytype
Return type:	Int

Example

>>> from descarteslabs.workflows import Tuple, Str
>>> my_tuple = Tuple[Str, Str, Str](("foo", "bar", "baz"))
>>> my_tuple.length().compute() 
3

publish(version, title='', description='', labels=None, tags=None, docstring='', version_labels=None, viz_options=None, client=None)¶

Publish a proxy object as a Workflow with the given version.

If the proxy object depends on any parameters (obj.params is not empty), it’s first internally converted to a Function that takes those parameters (using Function.from_object).

Parameters:	id (Str) – ID for the new Workflow object. This should be of the form `email:workflow_name` and should be globally unique. If this ID is not of the proper format, you will not be able to save the Workflow. version (Str) – The version to be set, tied to the given `obj`. This should adhere to the semantic versioning schema. title (Str, default "") – User-friendly title for the `Workflow`. description (str, default "") – Long-form description of this `Workflow`. Markdown is supported. labels (Dict, optional) – Key-value pair labels to add to the `Workflow`. tags (list, optional) – A list of strings to add as tags to the `Workflow`. docstring (Str, default "") – The docstring for this version. version_labels (Dict, optional) – Key-value pair labels to add to the version. client (`workflows.client.Client`, optional) – Allows you to use a specific client instance with non-default auth and parameters
Returns:	workflow – The saved `Workflow` object. `workflow.id` contains the ID of the new Workflow.
Return type:	Workflow or Function

class KnownDict(*args, **kwargs)[source]¶

KnownDict[<{key: KnownType, ...}>, KeyType, ValueType]: Proxy mapping from specific keys to specific value types, with default type for unknown values.

Cannot be instantiated directly; meant to be used as an element type in a container, or ._cast to.

Examples

>>> from descarteslabs.workflows import Float, Bool, Str, Int, Any
>>> from descarteslabs.workflows.types.containers import KnownDict, Tuple
>>> kd_type = KnownDict[Str, Int] # same as Dict[Str, Int]: no known keys given
>>> kd_type = KnownDict[Str, Tuple[Int, Float]] # known dict of Str to 2-tuple of Int and Float
>>> kd_type = KnownDict[{'x': Float, 'y': Bool}, Str, Int] # known dict where 'x' is Float, 'y' is Bool
>>> # all other keys are Str, and all other values are Int
>>> kd = Any({'x': 1, 'y': 2.2}).cast(kd_type)
>>> kd['x']  # Float
<descarteslabs.workflows.types.primitives.number.Float object at 0x...>
>>> kd['foo']  # Int
<descarteslabs.workflows.types.primitives.number.Int object at 0x...>

Methods:

`compute`([format, destination, file, …])	Compute a proxy object and wait for its result.
`contains`(key)	Whether the dictionary contains the given key.
`get`(key, default)	Return the value for `key` if `key` is in the dictionary, else `default`.
`inspect`([format, file, cache, _ruster, …])	Quickly compute a proxy object using a low-latency, lower-reliability backend.
`items`()	List of tuples of key-value pairs in the dictionary.
`keys`()	List of all the dictionary keys.
`length`()	The number of items in the dictionary
`publish`(version[, title, description, …])	Publish a proxy object as a `Workflow` with the given version.
`values`()	List of all the dictionary values.

Attributes:

`key_type`	Type of the keys in the dictionary
`value_type`	Type of the values in the dictionary

compute(format='pyarrow', destination='download', file=None, timeout=None, block=True, progress_bar=None, cache=True, _ruster=None, _trace=False, client=None, num_retries=None, **arguments)¶

Compute a proxy object and wait for its result.

If the caller has too many outstanding compute jobs, this will raise a ResourceExhausted exception.

Parameters:	geoctx (`GeoContext`, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a `GeoContext`, but for operations that only involve non-geospatial types, this parameter is optional. format (Str or Dict, default "pyarrow") – The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes or None. destination (str or dict, default "download") – The destination for the result. See the destinations documentation for more information. file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them. timeout (Int, optional) – The number of seconds to wait for the result, if `block` is True. Raises `JobTimeoutError` if the timeout passes. block (Bool, default True) – If True (default), block until the job is completed, or `timeout` has passed. If False, immediately returns a `Job` (which has already had `execute` called). progress_bar (Bool, default None) – Whether to draw the progress bar. If `None` (default), will display a progress bar in Jupyter Notebooks, but not elsewhere. Ignored if `block==False`. client (`workflows.client.Client`, optional) – Allows you to use a specific client instance with non-default auth and parameters num_retries (Int, optional) – The number of retries to make in the event of a request failure. If you are making numerous long-running asynchronous requests, you can use this parameter as a way to indicate that you are comfortable waiting and retrying in response to RESOURCE EXHAUSTED errors. By default, most failures will trigger a small number of retries, but if you have reached your outstanding job limit, by default, the client will not retry. This parameter is unnecessary when making synchronous `compute` requests (ie. block=True, the default). See the compute section of the Workflows Guide for more information. *arguments (Any*) – Values for all parameters that `obj` depends on (or arguments that `obj` takes, if it’s a `Function`). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.
Returns:	result – When `format="pyarrow"` (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from `descarteslabs.workflows.result_types`. For other formats, returns raw bytes. Consider using `file` in that case to save the results to a file. If the destination doesn’t support retrieving results (like “email”), returns None.
Return type:	Python object, bytes, or None
Raises:	RetryError – Raised if there are too many failed retries. Inspect `RetryError.exceptions` to determine the ultimate cause of the error. If you reach your maximum number of outstanding compute jobs, there will be one or more `ResourceExhausted` exceptions.

contains(key)¶

Whether the dictionary contains the given key.

Parameters:	key – Key to look up. Must be the same type as `self.key_type`.
Returns:
Return type:	Bool

Example

>>> from descarteslabs.workflows import Dict, Str, Int
>>> my_dict = Dict[Str, Int]({"foo": 1, "bar": 2, "baz": 3})
>>> my_dict.contains("foo").compute() 
True
>>> my_dict.contains("hello").compute() 
False

get(key, default)[source]¶

Return the value for key if key is in the dictionary, else default.

Parameters:	key – Key to look up. Must be the same type as `key_type`. default – Value returned if `key` does not exist. Must be the same type as `value_type`.

Example

>>> from descarteslabs.workflows import Dict, Str, Int
>>> my_dict = Dict[Str, Int]({"foo": 1, "bar": 2, "baz": 3})
>>> my_dict.get("baz").compute() 
3

inspect(format='pyarrow', file=None, cache=True, _ruster=None, timeout=60, client=None, **arguments)¶

Quickly compute a proxy object using a low-latency, lower-reliability backend.

Inspect is meant for getting simple computations out of Workflows, primarily for interactive use. It’s quicker but less resilient, won’t be retried if it fails, and has no progress updates.

If you have a larger computation (longer than ~30sec), or you want to be sure the computation will succeed, use compute instead. compute creates a Job, which runs asynchronously, will be retried if it fails, and stores its results for later retrieval.

Parameters:	geoctx (`common.geo.geocontext.GeoContext`, `GeoContext`, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a `GeoContext`, but for operations that only involve non-geospatial types, this parameter is optional. format (str or dict, default "pyarrow") – The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes. file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them. cache (bool, default True) – Whether to use the cache for this job. timeout (int, optional, default 60) – The number of seconds to wait for the result. Raises `JobTimeoutError` if the timeout passes. client (`workflows.inspect.InspectClient`, optional) – Allows you to use a specific InspectClient instance with non-default auth and parameters *arguments (Any*) – Values for all parameters that `obj` depends on (or arguments that `obj` takes, if it’s a `Function`). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.
Returns:	result – When `format="pyarrow"` (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from `descarteslabs.workflows.result_types`. For other formats, returns raw bytes. Consider using `file` in that case to save the results to a file.
Return type:	Python object or bytes

items()¶

List of tuples of key-value pairs in the dictionary.

Returns:
Return type:	List[Tuple[KeyType, ValueType]]

Example

>>> from descarteslabs.workflows import Dict, Str, Int
>>> my_dict = Dict[Str, Int]({"foo": 1, "bar": 2, "baz": 3})
>>> my_dict.items().compute() 
[('foo', 1), ('bar', 2), ('baz', 3)]

keys()¶

List of all the dictionary keys.

Returns:
Return type:	List

Example

>>> from descarteslabs.workflows import Dict, Str, Int
>>> my_dict = Dict[Str, Int]({"foo": 1, "bar": 2, "baz": 3})
>>> my_dict.keys().compute() 
['foo', 'bar', 'baz']

length()¶

The number of items in the dictionary

Returns:
Return type:	Int

Example

>>> from descarteslabs.workflows import Dict, Str, Int
>>> my_dict = Dict[Str, Int]({"foo": 1, "bar": 2, "baz": 3})
>>> my_dict.length().compute() 
3

publish(version, title='', description='', labels=None, tags=None, docstring='', version_labels=None, viz_options=None, client=None)¶

Publish a proxy object as a Workflow with the given version.

If the proxy object depends on any parameters (obj.params is not empty), it’s first internally converted to a Function that takes those parameters (using Function.from_object).

Parameters:	id (Str) – ID for the new Workflow object. This should be of the form `email:workflow_name` and should be globally unique. If this ID is not of the proper format, you will not be able to save the Workflow. version (Str) – The version to be set, tied to the given `obj`. This should adhere to the semantic versioning schema. title (Str, default "") – User-friendly title for the `Workflow`. description (str, default "") – Long-form description of this `Workflow`. Markdown is supported. labels (Dict, optional) – Key-value pair labels to add to the `Workflow`. tags (list, optional) – A list of strings to add as tags to the `Workflow`. docstring (Str, default "") – The docstring for this version. version_labels (Dict, optional) – Key-value pair labels to add to the version. client (`workflows.client.Client`, optional) – Allows you to use a specific client instance with non-default auth and parameters
Returns:	workflow – The saved `Workflow` object. `workflow.id` contains the ID of the new Workflow.
Return type:	Workflow or Function

values()¶

List of all the dictionary values.

Returns:
Return type:	List

Example

>>> from descarteslabs.workflows import Dict, Str, Int
>>> my_dict = Dict[Str, Int]({"foo": 1, "bar": 2, "baz": 3})
>>> my_dict.values().compute() 
[1, 2, 3]

property key_type¶: Type of the keys in the dictionary

property value_type¶: Type of the values in the dictionary

range(start=None, stop=None, step=1)[source]¶

Returns a List of Int containing a sequence of numbers starting from 0 by default, incremented by 1 (default), ending at a specified number.

Example

>>> import descarteslabs.workflows as wf
>>> my_range = wf.range(start=2, stop=10, step=2)
>>> my_range
<descarteslabs.workflows.types.containers.list_.List[Int] object at 0x...>
>>> my_range.compute() 
[2, 4, 6, 8]

zip(*sequences)[source]¶

Returns a List of Tuple, where each tuple contains the i-th element from each of the arguments. All arguments must be Proxytype List, ImageCollection (zips along axis=”images”), Image (zips along axis=”bands”), FeatureCollection, GeometryCollection, or Str.

The returned List is truncated in length to the length of the shortest argument sequence.

Example

>>> import descarteslabs.workflows as wf
>>> imagecollection = wf.ImageCollection.from_id("sentinel-2:L1C")
>>> int_list = wf.List[wf.Int]([1, 2, 3, 4])
>>> str_list = wf.List[wf.Str](["foo", "bar", "baz"])
>>> zipped = wf.zip(imagecollection, int_list, str_list)
>>> zipped
<descarteslabs.workflows.types.containers.list_.List[Tuple[Image, Int, Str]] object at 0x...>
>>> wf.zip(int_list, str_list, wf.Str("abcdefg")).compute() 
[(1, 'foo', 'a'), (2, 'bar', 'b'), (3, 'baz', 'c')]