Execution

Exceptions:

JobComputeError(job) Generic error raised when a job computation fails.
JobTimeoutError Raised when a computation took longer to complete than a specified timeout.

Classes:

VizOption(id, bands[, checkerboard, …]) Construct a VizOption instance.
UserEmail(email) The given string is interpreted as a user email
Organization(org) The given string is interpreted as an organization
AllAuthenticatedUsers()
Workflow(id[, title, description, labels, …]) A collection of versions of a proxy object, with associated metadata.
VersionedGraft(version, proxy_object[, …]) A specific version of a Workflow.
Job(obj[, geoctx, format, destination, …]) A Job represents the computation of a proxy object within a GeoContext, with values (arguments) set for any parameters it depends on.
XYZ(proxy_object[, name, description, …]) Stores proxy objects to be rendered by an XYZ tile server.
XYZLogListener(xyz_id[, client]) Calls callback functions in a background thread when XYZ log records occur.

Functions:

inspect(obj[, geoctx, format, file, cache, …]) Quickly compute a proxy object using a low-latency, lower-reliability backend.
wmts_url([tile_matrix_sets, dimensions, client]) Get the WMTS endpoint which gives access to all workflows accessible to this user.
compute(obj[, geoctx, format, destination, …]) Compute a proxy object and wait for its result.
publish(id, version[, obj, title, …]) Publish a proxy object as a Workflow with the given version.
use(workflow_id, version[, client]) Use like import: load the proxy object of a published Workflow version.
class VizOption(id: str, bands: Union[str, Sequence[str]], checkerboard: bool = False, colormap: Optional[str] = None, reduction: str = 'mosaic', scales: Optional[Sequence[Sequence[float]]] = None, description: str = '')[source]

Construct a VizOption instance.

Parameters:
  • id (str) – Name of the VizOption instance.
  • bands (str or list) – List of band names to render. Can be a list of band names, or a string of space separated band names.
  • checkerboard (bool, default False) – True if missing pixels should hold checkerboard pattern. Defaults to False.
  • colormap (str, optional) – Name of colormap to use if single band. Defaults to None.
  • reduction (str, default "mosaic".) – Name of reduction operation. Defaults to mosaic.
  • scales (list, optional) – List of list of floats representing scales for rendering bands, must have same length as bands, or None for automatic scaling. Defaults to None.
  • description (str, optional) – Description of the visualization option.

Example

>>> import descarteslabs.workflows as wf
>>> v = wf.VizOption('RGB', bands='red green blue', reduction='median', description='Median RGB composite') 
>>> v 
VizOption(
... id='RGB',
... bands=['red', 'green', 'blue'],
... checkerboard=False,
... colormap=None,
... reduction='median',
... scales=None,
... description='Median RGB composite'
)

Attributes:

bands Get the bands to visualize.
checkerboard Get the checkerboard setting for visualization.
colormap Get the colormap for visualization.
description Get the description of the visualization option.
id Get the id of the visualization parameters.
reduction Get the reduction operation for visualization.
scales Get the scales for visualization.
property bands

Get the bands to visualize.

property checkerboard

Get the checkerboard setting for visualization.

property colormap

Get the colormap for visualization.

property description

Get the description of the visualization option.

property id

Get the id of the visualization parameters.

property reduction

Get the reduction operation for visualization.

property scales

Get the scales for visualization.

class UserEmail(email)[source]

The given string is interpreted as a user email

Parameters:email (str) – The user email to use.
class Organization(org)[source]

The given string is interpreted as an organization

Parameters:org (str) – The name of the organization to use. You can only share with your own organization (visit https://iam.descarteslabs.com to see your organization’s name). An organization name is always all lowercase and only contains ascii characters, the - character and/or the _ character.
class Workflow(id, title='', description='', labels=None, tags=None, client=None)[source]

A collection of versions of a proxy object, with associated metadata.

Examples

>>> from descarteslabs.workflows import Int, Workflow
>>> workflow = Workflow(
...     id="bob@gmail.com:cool_addition_model",
...     title="Bob's Super Cool Addition Model",
...     description="The result of 1 plus 1",
...     labels={"project": "arithmetic"},
...     tags=["test", "cool", "deep learning", "AI", "digital twin"])  
>>> num = Int(1) + 1  
>>> workflow.set_version(num, version="0.0.1")  
>>> workflow  
Workflow: "Bob's Super Cool Addition Model"
    - id: bob@gmail.com:cool_addition_model
    - labels: {'project': 'arithmetic'}
    - tags: ['test', 'cool', 'deep learning', 'AI', 'digital twin']
    - versions: '0.0.1'
    The result of 1 plus 1
>>> workflow.get_version("0.0.1").object  
<descarteslabs.workflows.types.primitives.number.Int object at 0x...>
>>> workflow.get_version("0.0.1").object.inspect()  
2

>>> workflow.set_version(num + 2, version="0.0.2", docstring="1 + 2")  
>>> workflow.description = "The result of 1 + 2"  
>>> workflow.save()  
>>> workflow.get_version("0.0.2").object.inspect()  
3
>>> workflow  
Workflow: "Bob's Super Cool Addition Model"
    - id: bob@gmail.com:cool_addition_model
    - labels: {'project': 'arithmetic'}
    - tags: ['test', 'cool', 'deep learning', 'AI', 'digital twin']
    - versions: '0.0.1', '0.0.2'
    The result of 1 plus 2

Construct a Workflow object referencing a new or existing Workflow.

Parameters:
  • id (str) – ID for the new Workflow. This should be of the form "email:workflow_name" and should be globally unique. If this ID is not of the proper format, you will not be able to save the Workflow. Cannot be changed once set.
  • title (str, default "") – User-friendly title for the Workflow.
  • description (str, default "") – Long-form description of this Workflow. Markdown is supported.
  • labels (dict, optional) – Key-value pair labels to add to the Workflow.
  • tags (list, optional) – A list of strings of tags to add to the Workflow.
  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters.
Returns:
Return type:

Workflow

Example

>>> from descarteslabs.workflows import Workflow, Int
>>> workflow = Workflow(
...     id="bob@gmail.com:cool_addition_model",
...     title="Bob's Super Cool Addition Model",
...     description="The result of 1 plus 1",
...     labels={"project": "arithmetic"},
...     tags=["test", "cool", "deep learning", "AI", "digital twin"]) 
>>> workflow 
Workflow: "Bob's Super Cool Addition Model"
    - id: bob@gmail.com:cool_addition_model
    - labels: {'project': 'arithmetic'}
    - tags: ['test', 'cool', 'deep learning', 'AI', 'digital twin']
    - versions: '0.0.1', '0.0.2'
    The result of 1 plus 1

Methods:

add_public_reader([client]) Share the workflow with all users of the Descartes Labs Platform.
add_reader(target[, client]) Share the workflow with another user or organization.
delete([client]) Delete this Workflow and every VersionedGraft it contains.
delete_id(id[, client]) Delete the Workflow that has the provided ID, and every VersionedGraft it contains.
duplicate(new_id[, include_versions, client]) Duplicate this Workflow object with a new ID.
get(id[, client]) Get an existing Workflow by ID.
get_version(version) Get the VersionedGraft object for an existing version.
has_public_reader([client]) Check if workflow is shared with all users.
list_readers([client]) Iterator over users with whom the workflow has been shared.
remove_public_reader([client]) Revoke sharing of the workflow with all users.
remove_reader(target[, client]) Revoke sharing of the workflow with another user or organization.
save([client]) Save the current state of this Workflow and its VersionedGrafts.
search([email, name, tags, client]) Iterator over Workflows you have access to.
set_version(version[, obj, docstring, …]) Register a version to an object.
update_version(version[, docstring, labels]) Update the docstring and/or labels for a Workflow version.
wmts_url([tile_matrix_sets, dimensions]) Get the WMTS endpoint which gives access to this workflow.

Attributes:

created_timestamp The UTC date this Workflow was created, or None if it hasn’t been saved yet.
description A long-form description of this Workflow.
id The globally unique identifier for the Workflow, of the format "user@domain.com:workflow-name"
labels The labels associated with this Workflow, expressed as key-value pairs.
name The name of this Workflow (everything following the first colon in id).
tags The tags associated with this Workflow, expressed as a list of stirngs.
title The user-friendly title of this Workflow.
updated_timestamp The UTC date this Workflow was most recently modified, or None if it hasn’t been saved yet.
version_names The names of all the versions of this Workflow, in the order they were created.
versions Every VersionedGraft in this Workflow, in the order they were created.
add_public_reader(client=None)[source]

Share the workflow with all users of the Descartes Labs Platform. Additional permissions are required to share a workflow publicly. Please contact Descartes Labs if you have a need to do so.

The workflow must have previously been saved to the backend; to do so, call save.

Parameters:client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters.

Example

>>> from descarteslabs.workflows import Workflow
>>> workflow = Workflow.get("bob@gmail.com:cool_model") 
>>> workflow.add_public_reader()
add_reader(target, client=None)[source]

Share the workflow with another user or organization.

The workflow must have previously been saved to the backend; to do so, call save.

Parameters:

Example

>>> from descarteslabs.workflows import Workflow, UserEmail, Organization
>>> workflow = Workflow.get("bob@gmail.com:cool_model") 
>>> workflow.add_reader(UserEmail("betty@gmail.com")) 
>>> workflow.add_reader(Organization("myorg"))
delete(client=None)[source]

Delete this Workflow and every VersionedGraft it contains.

After deletion, all fields on self are reset.

Warning: this cannot be undone!

Parameters:client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters.

Example

>>> import descarteslabs.workflows as wf
>>> workflow = wf.Workflow.get("bob@gmail.com:cool_thing_wf") 
>>> workflow.delete()
static delete_id(id, client=None)[source]

Delete the Workflow that has the provided ID, and every VersionedGraft it contains.

Warning: this cannot be undone!

Parameters:
  • id (str) – The ID of the Workflow that we wish to delete.
  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters.

Example

>>> import descarteslabs.workflows as wf
>>> workflow = wf.Workflow.get("bob@gmail.com:cool_thing_wf") 
>>> wf.Workflow.delete_id(workflow.id)
duplicate(new_id, include_versions=True, client=None)[source]

Duplicate this Workflow object with a new ID.

This does not save the new object to the backend; to do so, call save.

Parameters:
  • new_id (str) – ID for the copied Workflow
  • include_versions (bool, optional, default True) – Whether to also copy over all the versions currently set on this Workflow
Returns:

new_workflow – Copy of self with a new ID
Return type:

Workflow

Example

>>> from descarteslabs.workflows import Workflow
>>> workflow = Workflow.get("bob@gmail.com:cool_model") 
>>> new_workflow = workflow.duplicate("bob@gmail.com:even_cooler_model") 
>>> new_workflow.id 
"bob@gmail.com:even_cooler_model"
>>> new_workflow.save() 
>>> # ^ save the duplicated workflow to the backend
classmethod get(id, client=None)[source]

Get an existing Workflow by ID.

Parameters:
  • id (string) – The ID of the Workflow (like name@example.com:workflow_id)
  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters.
Returns:
Return type:

Workflow

Example

>>> from descarteslabs.workflows import Workflow
>>> workflow = Workflow.get('bob@gmail.com:cool_addition_model') 
>>> workflow 
Workflow: "Bob's Super Cool Addition Model"
    - id: bob@gmail.com:cool_addition_model
    - labels: {'project': 'arithmetic'}
    - tags: ['test', 'cool', 'deep learning', 'AI', 'digital twin']
    - versions: '0.0.1', '0.0.2'
    The result of 1 plus 1
get_version(version)[source]

Get the VersionedGraft object for an existing version.

Raises a KeyError if this Workflow object doesn’t contain that version. Note that that doesn’t necessarily mean the version doesn’t exist—if you constructed this Workflow object directly and haven’t called save yet, the object won’t contain existing versions that are defined on the backend, but not in this session.

However, if you constructed this Workflow from Workflow.get, then an error here means the version actually doesn’t exist (unless it was just added, after the get call).

Note

You can also use indexing syntax as shorthand for get_version: workflow["1.0.0"] is equivalent to workflow.get_version("1.0.0").

Parameters:version (str) – A version of this Workflow.
Returns:version – The version requested, represented as a VersionedGraft.
Return type:VersionedGraft
Raises:KeyError – If this object doesn’t contain the given version.

Example

>>> import descarteslabs.workflows as wf
>>> workflow = wf.Workflow.get("bob@gmail.com:cool_model") 
>>> workflow.get_version('0.0.1') 
VersionedGraft: 0.0.1
    - type: Int
    - channel: master
    - labels: {}
    - viz_options: []
has_public_reader(client=None)[source]

Check if workflow is shared with all users.

The workflow must have previously been saved to the backend; to do so, call save.

Parameters:client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters.

Example

>>> from descarteslabs.workflows import Workflow
>>> workflow = Workflow.get("bob@gmail.com:cool_model") 
>>> workflow.has_public_reader()
list_readers(client=None)[source]

Iterator over users with whom the workflow has been shared.

The workflow must have previously been saved to the backend; to do so, call save.

Parameters:client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters.
Yields:user (UserEmail or Organization) – UserEmail or Organization with whom the workflow has been shared for reading.

Example

>>> from descarteslabs.workflows import Workflow
>>> workflow = Workflow.get("bob@gmail.com:cool_model") 
>>> for user in workflow.list_readers(): 
...     print(user)
remove_public_reader(client=None)[source]

Revoke sharing of the workflow with all users.

The workflow must have previously been saved to the backend; to do so, call save.

Parameters:client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters.

Example

>>> from descarteslabs.workflows import Workflow
>>> workflow = Workflow.get("bob@gmail.com:cool_model") 
>>> workflow.remove_public_reader()
remove_reader(target, client=None)[source]

Revoke sharing of the workflow with another user or organization.

The workflow must have previously been saved to the backend; to do so, call save.

Parameters:
  • user (str) – The UserEmail or Organization for whom premission to access the workflow is to be revoked.
  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters.

Example

>>> from descarteslabs.workflows import Workflow, UserEmail, Organization
>>> workflow = Workflow.get("bob@gmail.com:cool_model") 
>>> workflow.remove_reader(UserEmail("betty@gmail.com")) 
>>> workflow.remove_reader(Organization("myorg"))
save(client=None)[source]

Save the current state of this Workflow and its VersionedGrafts.

If the Workflow doesn’t exist yet, it will be created. You can save up to 1000 workflows; saving more will cause a ResourceExhausted exception.

If it does exist, it’ll attempt to update the VersionedGrafts with the ones set on this object. Any new versions will be added. Since VersionedGrafts are immutable, if a version already exists on the backend that is also set on this object, save will raise an error if the grafts of those version are different, otherwise updating that version will be a no-op if the grafts are the same.

If the Workflow does exist, all Workflow-level metadata (title, description, labels, etc) will be overwritten with the values set on this object.

Parameters:client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters.
Returns:self – Same Workflow object, after its state has been updated to match the backend’s state
Return type:Workflow
Raises:RetryError – Within RetryError.exceptions there will be ResourceExhausted exceptions. You reached your maximum number (1000) of saved workflows

Example

>>> from descarteslabs.workflows import Workflow
>>> workflow = Workflow.get("bob@gmail.com:cool_model") 
>>> workflow.update_version("0.0.1", docstring="Runs Bob's cool model") 
>>> # ^ update version 0.0.1's docstring
>>> workflow.save() 
>>> # ^ save the updated version to the backend
classmethod search(email='', name='', tags=None, client=None)[source]

Iterator over Workflows you have access to.

All search options are logically ANDed together.

Parameters:
  • email (str, optional) – Restrict to Workflows belonging to this email address (exact match).
  • name (str, optional) – Restrict to Workflow names that start with this string (prefix search). (The name follows the colon in a Workflow ID: email@example.com:this_is_the_name.)
  • tags (List[str], optional) – Restrict to Workflows that have _any_ of these tags.
  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters.
Yields:

workflow (Workflow) – Workflow matching the search criteria

Example

>>> import descarteslabs.workflows as wf
>>> for workflow in wf.Workflow.search():  
...     print(workflow.id)  
you@example.com:my_workflow
someone_else@example.com:my_workflow
someone_else@example.com:my_other_workflow
set_version(version, obj=None, docstring='', labels=None, viz_options=None)[source]

Register a version to an object. Can also be used as a decorator.

Note: this doesn’t save the new version to the backend. Call save afterwords to do so.

If the object depends on any parameters (obj.params is not empty), it’s first internally converted to a Function that takes those parameters (using Function.from_object).

Parameters:
  • version (str) – The version to be set, tied to the given obj. This should adhere to the semantic versioning schema (https://semver.org).
  • obj (Proxytype, optional) –

    The object to store as this version. If it depends on parameters, obj is first converted to a Function that takes those parameters.

    If not provided, it’s assumed that set_version is being used as a decorator on a function.

  • docstring (str, default "") – The docstring for this version. If not provided and set_version isn’t being used as a decorator, then the version’s docstring will be set to the empty string. If not provided and set_version is being used as a decorator, then the version’s docstring will be set to the docstring of the decorated Python function.
  • labels (dict, optional) – Key-value pair labels to add to the version.
  • viz_options (list, default None) – List of VizOption visualization parameter sets.
Returns:

version – The version set. If used as a decorator, returns the Function instead.
Return type:

VersionedGraft or Function

Example

>>> from descarteslabs.workflows import Int, Workflow
>>> workflow = Workflow(id="bob@gmail.com:cool_model") 
>>> num = Int(1) + 1 
>>> workflow.set_version('0.0.1', num)  
>>> workflow.get_version('0.0.1') 
VersionedGraft: 0.0.1
    - type: Int
    - channel: master
    - labels: {}
    - viz_options: []
update_version(version, docstring=None, labels=None)[source]

Update the docstring and/or labels for a Workflow version.

Note: this doesn’t save the updated version to the backend. Call save afterwords to do so.

Parameters:
  • version (str) – The version to be updated.
  • docstring (str, optional) – The new docstring for this version. One of docstring and labels must be provided.
  • labels (dict, optional) – Key-value pair labels to add to the version. These labels will overwrite the existing labels for this version. One of docstring and labels must be provided.
Returns:

version – The updated version.
Return type:

VersionedGraft

Example

>>> import descarteslabs.workflows as wf
>>> workflow = wf.Workflow.get("bob@gmail.com:cool_model") 
>>> workflow.get_version("0.0.1")  
VersionedGraft: 0.0.1
    - type: Int
    - channel: master
    - labels: {'source_product': 'sentinel2:L1C'}
    - viz_options: []
>>> workflow.update_version('0.0.1', labels={'source_product': 'usda:cdl:v1'}) 
VersionedGraft: 0.0.1
    - type: Int
    - channel: master
    - labels: {'source_product': 'usda:cdl:v1'}
    - viz_options: []
>>> workflow.save() 
>>> # ^ save the updated version to the backend
wmts_url(tile_matrix_sets=None, dimensions=None) → str[source]

Get the WMTS endpoint which gives access to this workflow.

Parameters:
  • tile_matrix_sets (str or list, optional) – Desired tile matrix sets. Defaults to EPSG:3857.
  • dimensions (bool, optional) – If True, then provide dimensions definitions to WMTS. If False, then generate a layer for each possible dimensions attribute combination. If not specified, the WMTS service itself will determine how to handle dimensions (currently it does not provide dimensions definitions).
Returns:

wmts_url – The URL for the WMTS service endpoint corresponding to this workflow.
Return type:

str

Example

>>> from descarteslabs.workflows import Workflow
>>> workflow = Workflow.get("bob@gmail.com:cool_model") 
>>> workflow.wmts_url() 
'https://workflows.prod.descarteslabs.com/master/wmts/workflow/...'
property created_timestamp

The UTC date this Workflow was created, or None if it hasn’t been saved yet. Cannot be modified.

Type:datetime.datetime or None
property description

A long-form description of this Workflow. Markdown is supported.

Type:str
property id

The globally unique identifier for the Workflow, of the format "user@domain.com:workflow-name"

Type:str
property labels

The labels associated with this Workflow, expressed as key-value pairs.

Type:dict
property name

The name of this Workflow (everything following the first colon in id).

Type:str
property tags

The tags associated with this Workflow, expressed as a list of stirngs.

Type:list
property title

The user-friendly title of this Workflow.

Type:str
property updated_timestamp

The UTC date this Workflow was most recently modified, or None if it hasn’t been saved yet. Updated automatically.

Type:datetime.datetime or None
property version_names

The names of all the versions of this Workflow, in the order they were created.

Type:tuple
property versions

Every VersionedGraft in this Workflow, in the order they were created.

Type:tuple
class VersionedGraft(version: str, proxy_object: descarteslabs.workflows.types.core.core.Proxytype, docstring: str = '', labels: Optional[Dict[str, str]] = None, viz_options: Optional[Sequence[descarteslabs.workflows.models.visualization.VizOption]] = None, client: Optional[descarteslabs.workflows.client.client.Client] = None)[source]

A specific version of a Workflow.

Except in advanced cases, you shouldn’t need to interact with this object much—you’ll primarily use the Workflow object and wf.use.

Construct a VersionedGraft object from a proxy object.

You shouldn’t construct a VersionedGraft directly; use Workflow.set_version or wf.publish instead.

If the proxy object depends on any parameters (proxy_object.params is not empty), it’s first internally converted to a Function that takes those parameters (using Function.from_object).

Parameters:
  • version (str) – Version of the graft. This should adhere to the semantic versioning schema (https://semver.org).
  • proxy_object (Proxytype) – The proxy object source of the graft. If it depends on parameters, proxy_obj is first converted to a Function that takes those parameters.
  • docstring (str, default "") – Docstring for the VersionedGraft.
  • labels (dict, optional) – Key-value pair labels to add to the VersionedGraft.
  • viz_options (list, default None) – List of VizOption visualization parameter sets.
  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters
Returns:
Return type:

VersionedGraft

Attributes:

channel The channel the object is compatible with.
client_version The client version the object was created with, and will run under.
docstring The docstring for this VersionedGraft.
labels The labels attached to this VersionedGraft.
object The stored proxy object.
params Parameter objects corresponding to the arguments to object, if it’s a Function.
type The type of the proxy object.
version The version of this VersionedGraft.
viz_options VizOption parameter sets for this VersionedGraft.

Methods:

get(workflow_id, version[, client]) Get a specific VersionedGraft of a Workflow.
url([colormap, bands, scales, reduction, …]) URL template for displaying this VersionedGraft on a web map, like https://workflows.descarteslabs.com/v0-5/workflow/test@example.com:foo/1.0.1/{z}/{x}/{y}.png
wmts_url([tile_matrix_sets, dimensions]) Get the WMTS endpoint which gives access to this versioned graft.
classmethod get(workflow_id: str, version: str, client: Optional[descarteslabs.workflows.client.client.Client] = None)[source]

Get a specific VersionedGraft of a Workflow.

Parameters:
  • workflow_id (str) – The ID of the Workflow.
  • version (str) – The version of the Workflow that you wish to fetch.
  • client (workflows.client.Client, optional, default None) – Allows you to use a specific client instance with non-default auth and parameters.
Returns:
Return type:

VersionedGraft
url(colormap=None, bands=None, scales=None, reduction=None, checkerboard=None, **arguments)[source]

URL template for displaying this VersionedGraft on a web map, like https://workflows.descarteslabs.com/v0-5/workflow/test@example.com:foo/1.0.1/{z}/{x}/{y}.png

The returned URL uses the XYZ/OpenStreetMap tiling scheme.

object must be an Image, an ImageCollection, or a Function that returns an Image or ImageCollection for the URL to work.

Parameters:
  • colormap (str, optional, default None) – Name of the colormap to use. If set, the displayed Image or ImageCollection must have 1 band.
  • bands (list of str, optional, default None) – The band names to select from the imagery. If None (default), the imagery should already have 1-3 bands selected.
  • scales (list of lists, optional, default None) –

    The scaling to apply to each band in the Image. If displaying an ImageCollection, it is reduced into an Image before scaling.

    If the Image or ImageCollection contains 3 bands, scales must be a list like [(0, 1), (0, 1), (-1, 1)].

    If the Image or ImageCollection contains 1 band, scales must be a list like [(0, 1)], or just (0, 1) for convenience

    If None, each 256x256 tile will be scaled independently.

  • reduction (str, optional, default None) – One of “mosaic”, “min”, “max”, “mean”, “median”, “sum”, “std”, or “count”. If displaying an ImageCollection, this method is used to reduce it into an Image. The reduction is performed before applying a colormap or scaling. If displaying an Image, reduction is ignored.
  • checkerboard (bool, default None) – Whether to display a checkerboarded background for missing or masked data.
  • **arguments (Any) – Values for all the arguments that object takes, if it’s a Function. Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.
Returns:

url – Tile URL containing {z}, {x}, and {y} as Python format string parameters, and query arguments URL-quoted.
Return type:

str
Raises:
  • ValueError – If Workflow containing this VersionedGraft object has not been saved yet.
  • TypeError – If the scales are of invalid type. If the arguments names or types don’t match the arguments that the object takes. (Those required arguments are equivalent to params.)

Example

>>> import descarteslabs.workflows as wf
>>> img = wf.Image.from_id("sentinel-2:L1C:2019-05-04_13SDV_99_S2B_v1")
>>> red = img.pick_bands("red")
>>> viz = red ** wf.parameter("exponent", wf.Float)
>>> flow = viz.publish("you@example.com:s2_red", "1.0.0") 
>>> vg = flow["1.0.0"] 
>>> vg.url(colormap="magma", scales=[(0.2, 0.8)], exponent=2.5) 
'https://workflows.descarteslabs.com/v0-0/workflow/you@example.com:s2_red/1.0.0/
{z}/{x}/{y}.png?colormap=magma&scales=%5B%5B0.2%2C+0.8%5D%5D&exponent=2.5'

>>> import descarteslabs.workflows as wf
>>> col = wf.ImageCollection.from_id("landsat:LC08:01:RT:TOAR",
...        start_datetime="2017-01-01",
...        end_datetime="2017-12-31")
>>> red = col.pick_bands("red")
>>> viz = red ** wf.parameter("exponent", wf.Float)
>>> flow = viz.publish("you@example.com:l8_red", "1.0.0") 
>>> vg = flow["1.0.0"] 
>>> vg.url(reduction="min", exponent=2.5) 
'https://workflows.descarteslabs.com/v0-0/workflow/you@example.com:l8_red/1.0.0/
{z}/{x}/{y}.png?colormap=magma&scales=%5B%5B0.2%2C+0.8%5D%5D&reduction=min&exponent=2.5'
wmts_url(tile_matrix_sets=None, dimensions=None) → str[source]

Get the WMTS endpoint which gives access to this versioned graft.

Parameters:
  • tile_matrix_sets (str or list, optional) – Desired tile matrix sets. Defaults to EPSG:3857.
  • dimensions (bool, optional) – If True, then provide dimensions definitions to WMTS. If False, then generate a layer for each possible dimensions attribute combination. If not specified, the WMTS service itself will determine how to handle dimensions (currently it does not provide dimensions definitions).
Returns:

wmts_url – The URL for the WMTS service endpoint corresponding to this versioned graft.
Return type:

str
property channel

The channel the object is compatible with.

Type:str
property client_version

The client version the object was created with, and will run under.

Type:str
property docstring

The docstring for this VersionedGraft.

Type:str
property labels

The labels attached to this VersionedGraft.

Type:dict
property object

The stored proxy object.

Note that if this was originally constructed from a Workflows object that depended on parameters, then this object property won’t hold the same object, but rather a Function which takes those parameters and returns that object.

Raises ValueError if not compatible with the current channel.

Type:Proxytype
property params

Parameter objects corresponding to the arguments to object, if it’s a Function.

These represent any parameters (including widget objects from wf.widgets) that object depended on before it became a Function.

Type:tuple
property type

The type of the proxy object.

Raises ValueError if the object is not compatible with the current client version.

Type:type
property version

The version of this VersionedGraft.

Type:str
property viz_options

VizOption parameter sets for this VersionedGraft.

Type:list
class Job(obj, geoctx=None, format='pyarrow', destination='download', cache=True, _ruster=None, _trace=False, client=None, num_retries=None, **arguments)[source]

A Job represents the computation of a proxy object within a GeoContext, with values (arguments) set for any parameters it depends on.

If the proxy object depends on any parameters (obj.params is not empty), it’s first internally converted to a Function that takes those parameters (using Function.from_object).

Example

>>> import descarteslabs.workflows as wf
>>> num = wf.Int(1) + wf.parameter("x", wf.Int)
>>> job = num.compute(block=False, x=1)  
>>> job 
<descarteslabs.workflows.models.job.Job object at 0x...>
>>> job.id 
'3754676080bbb2b857fbc04a3e48f6312732e1bc42e0bd7b'
>>> job.result() 
2
>>> same_job = wf.Job.get('3754676080bbb2b857fbc04a3e48f6312732e1bc42e0bd7b') 
>>> same_job.stage 
'STAGE_DONE'
>>> same_job.result() 
2
>>> same_job.arguments() 
{'x': 1}

Creates a new Job to compute the provided proxy object with the given arguments.

Parameters:
  • obj (Proxytype) – Proxy object to compute, or list/tuple of proxy objects. If it depends on parameters, obj is first converted to a Function that takes those parameters.
  • geoctx (GeoContext, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a GeoContext, but for operations that only involve non-geospatial types, this parameter is optional.
  • format (str or dict, default "pyarrow") – The serialization format for the result.
  • destination (str or dict, default "download") – The destination for the result.
  • cache (bool, default True) – Whether to use the cache for this job.
  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters
  • num_retries (Int, optional) – The number of retries to make in the event of a request failure. If you are making numerous long-running asynchronous requests, you can use this parameter as a way to indicate that you are comfortable waiting and retrying in response to RESOURCE EXHAUSTED errors. By default, most failures will trigger a small number of retries, but if you have reached your outstanding job limit, by default, the client will not retry. See the compute section of the Workflows Guide for more information.
  • **arguments (Any) – Values for all parameters that obj depends on (or arguments that obj takes, if it’s a Function). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.

Example

>>> from descarteslabs.workflows import Job, Int, parameter
>>> my_int = Int(1) + parameter("other_int", Int)
>>> job = Job(my_int, other_int=10) 
>>> job.stage 
QUEUED

Attributes:

arguments The arguments of the Job, as a dict of names to Python primitives or Workflows objects.
cache_enabled Whether caching is enabled for this job.
cancelled Whether the job has been cancelled.
channel The channel name where this Job will execute.
created_datetime The time the Job was created.
destination The destination for the Job results, as a dictionary.
done Whether the Job has completed or not.
error The error of the Job, or None if it finished successfully.
expires_datetime The UTC date this Job will be expired.
format The serialization format of the Job, as a dictionary.
geoctx The Workflows GeoContext the Job was run within, or None
id The globally unique identifier for the Job, or None if it hasn’t been executed yet.
object The proxy object this Job computes.
result_type Name of the type of object that will be used to hold the result
runtime The total time it took the Job to run.
stage The current stage of the Job (queued, preparing, running, saving, succeeded, failed).
type The type of the proxy object.
updated_datetime The time of the most recent Job update.
url The download URL for this Job’s results.
version The descarteslabs client version that constructed this Job.

Methods:

cancel() Cancel a running job.
get(id[, client]) Get a currently-running Job by its ID.
refresh() Refresh the attributes and state of the job.
resubmit() Resubmit this job, returning a new Job object.
result([timeout, progress_bar]) Get the result of the job.
result_to_file(file[, timeout, progress_bar]) Save the result of the job to a file.
wait([timeout, progress_bar, …]) Block until the Job is complete, optionally displaying a progress bar.
watch([timeout]) Generator that yields self each time an update to the Job occurs.
cancel()[source]

Cancel a running job.

Example

>>> from descarteslabs.workflows import Job, Int
>>> my_int = Int(1)
>>> job = Job(my_int) 
>>> job.cancel()
classmethod get(id, client=None)[source]

Get a currently-running Job by its ID.

Parameters:
  • id (string) – The ID of a running job.
  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters

Example

>>> from descarteslabs.workflows import Job
>>> job = Job.get('3754676080bbb2b857fbc04a3e48f6312732e1bc42e0bd7b')
refresh()[source]

Refresh the attributes and state of the job.

Example

>>> from descarteslabs.workflows import Job, Int
>>> job = Job(Int(1)) 
>>> job.stage 
QUEUED
>>> job.refresh() 
>>> job.stage 
SUCCEEDED
resubmit()[source]

Resubmit this job, returning a new Job object.

Example

>>> from descarteslabs.workflows import Job, Int
>>> job = Job(Int(1)) 
>>> job.id 
abc123
>>> job.result() 
1
>>> new_job = job.resubmit() 
>>> new_job.id 
xyz456
>>> new_job.result() 
1
result(timeout=None, progress_bar=None)[source]

Get the result of the job. This blocks until the job is complete.

Only the “download” destination can be retrieved. Raises NotImplementedError for other destinations.

Parameters:
  • timeout (int, optional) – The number of seconds to wait for the result.
  • progress_bar (bool, optional) – Flag to draw the progress bar. Default is to True if in Jupyter Notebook.
Returns:

result – When the Job’s format is “pyarrow”, returns a Python object representing the result, either as a plain Python type, or object from descarteslabs.workflows.result_types. For other formats, returns raw bytes. Consider using result_to_file in that case to save the results to a file.
Return type:

Python object or bytes

Example

>>> from descarteslabs.workflows import Job, Int
>>> job = Job(Int(1)) 
>>> job.result() 
1
result_to_file(file, timeout=None, progress_bar=None)[source]

Save the result of the job to a file. This blocks until the job is complete.

Only the “download” destination can be written to a file. For destinations like “catalog”, where the data is handed off to another service, you’ll need to use that service to retrieve it. (In the “catalog” case, that’s Raster and Metadata.)

Parameters:
  • file (path or file-like object) – Path or file where results will be written
  • timeout (int, optional) – The number of seconds to wait for the result.
  • progress_bar (bool, optional) – Flag to draw the progress bar. Default is to True if in Jupyter Notebook.

Example

>>> from descarteslabs.workflows import Job, Int
>>> job = Job(Int(1), format="json") 
>>> job.result_to_file("one.json") 

>>> import io
>>> from descarteslabs.workflows import Job, Int
>>> job = Job(Int(2), format="json") 
>>> bytestream = io.BytesIO() 
>>> job.result_to_file(bytestream) 
>>> print(bytestream.read()) 
b'2'
wait(timeout=None, progress_bar=False, cancel_on_timeout=True, cancel_on_interrupt=True)[source]

Block until the Job is complete, optionally displaying a progress bar.

Raises any error that occurs with the Job, or JobTimeoutError if the timeout passes before the Job is complete.

Parameters:
  • timeout (int, optional) – The number of seconds to wait for the result.
  • progress_bar (bool, optional) – Flag to draw the progress bar. Default is to True if in Jupyter Notebook.
  • cancel_on_timeout (bool, optional) – Whether to cancel the job on client timeout. Default is True.
  • cancel_on_interrupt (bool, optional) – Whether to cancel the job on interrupt (e.g. ctrl + c). Default is True.

Example

>>> import descarteslabs.workflows as wf
>>> job = wf.Int(1).compute(block=False) 
>>> job.wait() 
>>> # ^ blocks until `job` is done
>>> job.result() 
1
watch(timeout=None)[source]

Generator that yields self each time an update to the Job occurs.

Parameters:timeout (int, optional) – The number of seconds to watch for Job updates. Defaults to self._client.STREAM_TIMEOUT, which is also the maximum allowed.

Example

>>> from descarteslabs.workflows import Job, Int
>>> job = Job(Int(1)) 
>>> for job in job.watch(): 
...     print(job.stage)
QUEUED
PREPARING
RUNNING
RUNNING
SAVING
SUCCEEDED
property arguments

The arguments of the Job, as a dict of names to Python primitives or Workflows objects.

property cache_enabled

Whether caching is enabled for this job.

property cancelled

Whether the job has been cancelled.

property channel

The channel name where this Job will execute.

Type:str
property created_datetime

The time the Job was created.

Type:datetime
property destination

The destination for the Job results, as a dictionary.

property done

Whether the Job has completed or not.

Type:bool
property error

The error of the Job, or None if it finished successfully.

property expires_datetime

The UTC date this Job will be expired.

Type:datetime.datetime
property format

The serialization format of the Job, as a dictionary.

property geoctx

The Workflows GeoContext the Job was run within, or None

property id

The globally unique identifier for the Job, or None if it hasn’t been executed yet.

Type:str or None
property object

The proxy object this Job computes.

Type:Proxytype
property result_type

Name of the type of object that will be used to hold the result

Type:str
property runtime

The total time it took the Job to run.

Type:datetime
property stage

The current stage of the Job (queued, preparing, running, saving, succeeded, failed).

property type

The type of the proxy object.

Type:type
property updated_datetime

The time of the most recent Job update.

Type:datetime
property url

The download URL for this Job’s results.

If format is not “download” or “email”, url will be None.

property version

The descarteslabs client version that constructed this Job.

Type:str
class XYZ(proxy_object: descarteslabs.workflows.types.core.core.Proxytype, name: str = '', description: str = '', public: bool = True, viz_options: Optional[Sequence[descarteslabs.workflows.models.visualization.VizOption]] = None, days_to_expiration: int = None, client: Optional[descarteslabs.workflows.client.client.Client] = None)[source]

Stores proxy objects to be rendered by an XYZ tile server.

Similar to a Workflow, but meant for storing proxy objects so the XYZ tile service can display them, rather than for persisting and sharing workflows between users.

Use url to generate an XYZ URL template, and iter_tile_logs or log_listener to retrieve log messages that happen while computing them.

Examples

>>> from descarteslabs.workflows import Image, XYZ
>>> img = Image.from_id("sentinel-2:L1C:2019-05-04_13SDV_99_S2B_v1")
>>> rgb = img.pick_bands("red green blue")
>>> xyz = XYZ(rgb, name="My RGB") 
>>> xyz 
<descarteslabs.workflows.models.xyz.XYZ object at 0x...>
>>> xyz.id 
'24d0e79c5c1e1f10a0b1177ef3974d7edefd5988291cf2c6'
>>> same_xyz = XYZ.get('24d0e79c5c1e1f10a0b1177ef3974d7edefd5988291cf2c6') 
>>> same_xyz.url() 
'https://workflows.descarteslabs.com/master/xyz/24d0e79c5c1e1f10a0b1177ef3974d7edefd5988291cf2c6/{z}/{x}/{y}.png'
>>> same_xyz.object 
<descarteslabs.workflows.types.geospatial.image.Image object at 0x...>

>>> from descarteslabs.workflows import ImageCollection, XYZ
>>> col = ImageCollection.from_id("landsat:LC08:01:RT:TOAR",
...        start_datetime="2017-01-01",
...        end_datetime="2017-12-31")
>>> rgb = col.pick_bands("red green blue")
>>> xyz = XYZ(rgb, name="RGB") 
>>> xyz.url() 
'https://workflows.descarteslabs.com/tiles-ic/xyz/bdbeb4706f3025f4ee4eeff4dec46f6f2554c583d830e5e9/{z}/{x}/{y}.png'

Construct a new XYZ from a proxy object.

If the proxy object depends on any parameters (proxy_object.params is not empty), it’s first internally converted to a Function that takes those parameters (using Function.from_object).

Parameters:
  • proxy_object (Proxytype) – The proxy object to store in this XYZ. If it depends on parameters, proxy_object is first converted to a Function that takes those parameters.
  • name (str, default "") – Name for the new XYZ
  • description (str, default "") – Long-form description of this XYZ. Markdown is supported.
  • public (bool, default True) – If True then this object is shared and accessible to all, otherwise it is private.
  • viz_options (list, default None) – List of VizOption visualization parameter sets.
  • days_to_expiration (int, default None) – Days until this XYZ object will expire. If None defaults to 10 days.
  • client (workflows.client.Client, optional, default None) – Allows you to use a specific client instance with non-default auth and parameters

Example

>>> import descarteslabs.workflows as wf
>>> img = wf.Image.from_id("sentinel-2:L1C:2019-05-04_13SDV_99_S2B_v1")
>>> rgb = img.pick_bands("red green blue")
>>> xyz = wf.XYZ(rgb, name="My RGB") 
>>> xyz.id 
'24d0e79c5c1e1f10a0b1177ef3974d7edefd5988291cf2c6'

Attributes:

channel The channel the object is compatible with.
client_version The client version the object was created with, and will run under.
created_timestamp The UTC date this XYZ was created, or None if it hasn’t been saved yet.
description A long-form description of this XYZ.
expires_timestamp The UTC date this XYZ will be expired.
id The globally unique identifier for the XYZ
name The name of this XYZ.
object The stored proxy object.
org The org of the user which created this XYZ.
params Parameter objects corresponding to the arguments to object, if it’s a Function.
public True if this XYZ is shared.
type The type of the proxy object.
updated_timestamp The UTC date this XYZ was most recently modified, or None if it hasn’t been saved yet.
user The user ID which created this XYZ.
viz_options VizOption parameter sets for this VersionedGraft.

Methods:

delete([client]) Delete this XYZ object.
delete_id(id[, client]) Delete the XYZ that has the provided ID.
get(xyz_id[, client]) Get an existing XYZ by id.
iter_tile_logs(session_id[, start_datetime, …]) Iterator over log messages generated while computing tiles
list([public, client]) Get all XYZ objects created by this user.
log_listener() An XYZLogListener to trigger callbacks when logs occur computing tiles.
url([session_id, colormap, bands, scales, …]) URL template for displaying this XYZ object on a web map, like https://workflows.descarteslabs.com/v0-5/xyz/1234567/{z}/{x}/{y}.png
wmts_url([tile_matrix_sets, dimensions]) Get the WMTS endpoint which gives access to this XYZ object.
delete(client: descarteslabs.workflows.client.client.Client = None) → None[source]

Delete this XYZ object. Only the user that created the XYZ can delete it.

Parameters:client (Client, default None) – Allows you to use a specific client instance with non-default auth and parameters

Example

Returns:
Return type:None

Example

>>> from descarteslabs.workflows import XYZ
>>> xyz = XYZ.get('24d0e79c5c1e1f10a0b1177ef3974d7edefd5988291cf2c6') 
>>> xyz.delete()
static delete_id(id, client=None)[source]

Delete the XYZ that has the provided ID. Only the user that created the XYZ can delete it.

Warning: this cannot be undone!

Parameters:
  • id (str) – The ID of the XYZ that we wish to delete.
  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters.

Example

>>> from descarteslabs.workflows import XYZ
>>> xyz_id = '24d0e79c5c1e1f10a0b1177ef3974d7edefd5988291cf2c6'
>>> xyz = XYZ.get(xyz_id) 
>>> xyz 
<descarteslabs.workflows.models.xyz.XYZ object at 0x...>
>>> XYZ.delete_id(xyz_id) 
>>> XYZ.get(xyz_id) 
...
NotFound: 404 XYZ '24d0e79c5c1e1f10a0b1177ef3974d7edefd5988291cf2c6' does not exist.
classmethod get(xyz_id: str, client: Optional[descarteslabs.workflows.client.client.Client] = None) → descarteslabs.workflows.models.xyz.XYZ[source]

Get an existing XYZ by id.

Parameters:
  • id (str) – The unique id of a XZY object
  • client (Optional[Client], default None) – Allows you to use a specific client instance with non-default auth and parameters
Returns:
Return type:

XYZ

Example

>>> from descarteslabs.workflows import XYZ
>>> xyz = XYZ.get('24d0e79c5c1e1f10a0b1177ef3974d7edefd5988291cf2c6') 
>>> xyz 
<descarteslabs.workflows.models.xyz.XYZ object at 0x...>
iter_tile_logs(session_id: str, start_datetime: datetime.datetime = None, level: int = None) → Iterator[descarteslabs.common.proto.xyz.xyz_pb2.XYZLogRecord][source]

Iterator over log messages generated while computing tiles

Parameters:
  • session_id (str) – Unique, client-generated that logs are stored under.
  • start_datetime (datetime.datetime) – Only return log records occuring after this datetime
  • level (int, default logging.DEBUG) – Only return log records at or above this log level. See https://docs.python.org/3/library/logging.html#logging-levels for valid log levels.
Yields:

log_record (descarteslabs.common.proto.xyz_pb2.XYZLogRecord) – Logs in protobuf message objects, with fields session_id and record, with the record field containing level, message, and timestamp.

Example

>>> from descarteslabs.workflows import Image, XYZ
>>> img = Image.from_id("sentinel-2:L1C:2019-05-04_13SDV_99_S2B_v1")
>>> rgb = img.pick_bands("red green blue")
>>> xyz = XYZ(rgb, name="My RGB") 
>>> url = xyz.url(session_id="some_session") 
>>> for record in xyz.iter_tile_logs("some_session", start_datetime=datetime.datetime.now()): 
...     print(record.level, record.message)
>>>     # any logs that occur loading tiles from the generated URL will be printed here
classmethod list(public: bool = False, client: Optional[descarteslabs.workflows.client.client.Client] = None) → Iterator[descarteslabs.workflows.models.xyz.XYZ][source]

Get all XYZ objects created by this user.

Parameters:
  • public (bool, default False) – If True then return public (shared) XYZ objects created by this user. If False then return non-public XYZ objects created through the use of workflow map widgets.
  • client (Optional[Client], default None) – Allows you to use a specific client instance with non-default auth and parameters
Returns:
Return type:

Iterator[XYZ]

Example

>>> from descarteslabs.workflows import XYZ
>>> for xyz in XYZ.list(): 
...     print(xyz.id) 
24d0e79c5c1e1f10a0b1177ef3974d7edefd5988291cf2c6
log_listener() → descarteslabs.workflows.models.xyz.XYZLogListener[source]

An XYZLogListener to trigger callbacks when logs occur computing tiles.

Example

>>> from descarteslabs.workflows import Image, XYZ
>>> img = Image.from_id("sentinel-2:L1C:2019-05-04_13SDV_99_S2B_v1")
>>> rgb = img.pick_bands("red green blue")
>>> xyz = XYZ(rgb, name="My RGB") 
>>> url = xyz.url(session_id="some_session") 
>>> listener = xyz.log_listener() 
>>> log = []
>>> listener.add_callback(lambda record: log.append(record.message)) 
>>> listener.listen("some_session") 
>>> # any logs that occur loading tiles from the generated URL will be appended
>>> # to `log` in the background
url(session_id=None, colormap=None, bands=None, scales=None, reduction=None, checkerboard=None, **arguments)[source]

URL template for displaying this XYZ object on a web map, like https://workflows.descarteslabs.com/v0-5/xyz/1234567/{z}/{x}/{y}.png

The returned URL uses the XYZ/OpenStreetMap tiling scheme.

object must be an Image, an ImageCollection, or a Function that returns an Image or ImageCollection for the URL to work.

Parameters:
  • session_id (str, optional, default None) – Unique, client-generated ID that logs will be stored under. Since multiple users may access tiles from the same XYZ object, each user should set their own session_id to get individual logs.
  • colormap (str, optional, default None) – Name of the colormap to use. If set, the displayed Image or ImageCollection must have 1 band.
  • bands (list of str, optional, default None) – The band names to select from the imagery. If None (default), the imagery should already have 1-3 bands selected.
  • scales (list of lists, optional, default None) –

    The scaling to apply to each band in the Image. If displaying an ImageCollection, it is reduced into an Image before scaling.

    If the Image or ImageCollection contains 3 bands, scales must be a list like [(0, 1), (0, 1), (-1, 1)].

    If the Image or ImageCollection contains 1 band, scales must be a list like [(0, 1)], or just (0, 1) for convenience

    If None, each 256x256 tile will be scaled independently.

  • reduction (str, optional, default None) – One of “mosaic”, “min”, “max”, “mean”, “median”, “sum”, “std”, or “count”. If displaying an ImageCollection, this method is used to reduce it into an Image. The reduction is performed before applying a colormap or scaling. If displaying an Image, reduction is ignored.
  • checkerboard (bool, optional, default None) – Whether to display a checkerboarded background for missing or masked data.
  • **arguments (Any) – Values for all the arguments that object takes, if it’s a Function. Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.
Returns:

url – Tile URL containing {z}, {x}, and {y} as Python format string parameters, and query arguments URL-quoted.
Return type:

str
Raises:

TypeError – If the scales are of invalid type. If the arguments names or types don’t match the arguments that the object takes. (Those required arguments are equivalent to params.)

Example

>>> import descarteslabs.workflows as wf
>>> img = wf.Image.from_id("sentinel-2:L1C:2019-05-04_13SDV_99_S2B_v1")
>>> red = img.pick_bands("red")
>>> viz = red ** wf.parameter("exponent", wf.Float)
>>> xyz = wf.XYZ(viz, name="Red band raised to exponent") 
>>> xyz.url("some_session", colormap="magma", scales=[(0.2, 0.8)], exponent=2.5) 
'https://workflows.descarteslabs.com/v0-0/xyz/0d21037edb4bdd16b735f24bb3bff6d4202a71c20404b101/
 {z}/{x}/{y}.png?session_id=some_session&colormap=magma&scales=[[0.2, 0.8]]&exponent=2.5'

>>> import descarteslabs.workflows as wf
>>> col = wf.ImageCollection.from_id("landsat:LC08:01:RT:TOAR",
...        start_datetime="2017-01-01",
...        end_datetime="2017-12-31")
>>> red = col.pick_bands("red")
>>> viz = red ** wf.parameter("exponent", wf.Float)
>>> xyz = wf.XYZ(viz, name="Red band ImageCollection raised to exponent") 
>>> xyz.url("some_session", reduction="min", exponent=2.5) 
'https://workflows.descarteslabs.com/v0-0/xyz/bdbeb4706f3025f4ee4eeff4dec46f6f2554c583d830e5e9/
 {z}/{x}/{y}.png?session_id=some_session&reduction=min&exponent=2.5'
wmts_url(tile_matrix_sets=None, dimensions=None) → str[source]

Get the WMTS endpoint which gives access to this XYZ object.

Parameters:
  • tile_matrix_sets (str or list, optional) – Desired tile matrix sets. Defaults to EPSG:3857.
  • dimensions (bool, optional) – If True, then provide dimensions definitions to WMTS. If False, then generate a layer for each possible dimensions attribute combination. If not specified, the WMTS service itself will determine how to handle dimensions (currently it does not provide dimensions definitions).
Returns:

wmts_url – The URL for the WMTS service endpoint corresponding to this XYZ object.
Return type:

str

Example

>>> from descarteslabs.workflows import Image, XYZ
>>> img = Image.from_id("sentinel-2:L1C:2019-05-04_13SDV_99_S2B_v1")
>>> rgb = img.pick_bands("red green blue")
>>> xyz = XYZ(rgb, name="My RGB") 
>>> xyz.wmts_url() 
'https://workflows.prod.descarteslabs.com/master/wmts/xyz/...'
property channel

The channel the object is compatible with.

Type:str
property client_version

The client version the object was created with, and will run under.

Type:str
property created_timestamp

The UTC date this XYZ was created, or None if it hasn’t been saved yet. Cannot be modified.

Type:datetime.datetime or None
property description

A long-form description of this XYZ. Markdown is supported.

Type:str
property expires_timestamp

The UTC date this XYZ will be expired.

Type:datetime.datetime
property id

The globally unique identifier for the XYZ

Type:str
property name

The name of this XYZ.

Type:str
property object

The stored proxy object.

Note that if this was originally constructed from a Workflows object that depended on parameters, then this object property won’t hold the same object, but rather a Function which takes those parameters and returns that object.

Raises ValueError if not compatible with the current channel.

Type:Proxytype
property org

The org of the user which created this XYZ.

Type:str
property params

Parameter objects corresponding to the arguments to object, if it’s a Function.

These represent any parameters (including widget objects from wf.widgets) that object depended on before it became a Function.

Type:tuple
property public

True if this XYZ is shared.

Type:bool
property type

The type of the proxy object.

Raises ValueError if the object is not compatible with the current client version.

Type:type
property updated_timestamp

The UTC date this XYZ was most recently modified, or None if it hasn’t been saved yet. Updated automatically.

Type:datetime.datetime or None
property user

The user ID which created this XYZ.

Type:str
property viz_options

VizOption parameter sets for this VersionedGraft.

Type:list
class XYZLogListener(xyz_id: str, client: Optional[descarteslabs.workflows.client.client.Client] = None)[source]

Calls callback functions in a background thread when XYZ log records occur.

Note: the thread is automatically cleaned up on garbage collection.

Example

>>> from descarteslabs.workflows import Image, XYZ
>>> img = Image.from_id("landsat:LC08:PRE:TOAR:meta_LC80270312016188_v1")
>>> xyz = XYZ(img) 
>>> listener = xyz.log_listener() 
>>> def callback(msg):
...     print(msg.level, msg.message)
>>> listener.add_callback(callback) 
>>> listener.listen("session_id", start_datetime=datetime.datetime.now())  
>>> # later
>>> listener.stop()

Methods:

add_callback(callback) Function will be called with descarteslabs.common.proto.xyz_pb2.XYZLogRecord on each log record.
listen(session_id[, start_datetime, level]) Start listening for logs.
running() bool: whether this is an active listener
stop([timeout]) Cancel and clean up the listener.
add_callback(callback: Callable[[descarteslabs.common.proto.xyz.xyz_pb2.XYZLogRecord], Any])[source]

Function will be called with descarteslabs.common.proto.xyz_pb2.XYZLogRecord on each log record.

Parameters:callback (callable) –

Function that takes one argument, a descarteslabs.common.proto.xyz_pb2.XYZLogRecord protobuf message object. This message contains the fields code, message, timestamp, session_id. This message contains the fields session_id and record, with the record field containing level, message, and timestamp.

The function will be called within a separate thread, therefore it must behave thread-safely. Any errors raised by the function will terminate the listener.

Example

>>> from descarteslabs.workflows import XYZLogListener
>>> listener = XYZLogListener("xyz_id") 
>>> def callback(msg):
...     print(msg.level, msg.message)
>>> listener.add_callback(callback)
listen(session_id: str, start_datetime: datetime.datetime = None, level: int = None)[source]

Start listening for logs.

Parameters:
  • session_id (str) – Unique, client-generated ID that logs are stored under. See XYZ.url for more information.
  • start_datetime (datetime.datetime) – Only listen for log records occuring after this datetime. Must be tz-aware.
  • level (int, default logging.DEBUG) – Only listen for log records at or above this log level. See https://docs.python.org/3/library/logging.html#logging-levels for valid log levels.

Example

>>> from descarteslabs.workflows import XYZLogListener
>>> listener = XYZLogListener("xyz_id") 
>>> listener.listen(
...     "session-id",
...     start_datetime=datetime.datetime.now(datetime.timezone.utc),
...     level=logging.WARNING,
... )
running() → bool[source]

bool: whether this is an active listener

Example

>>> from descarteslabs.workflows import XYZLogListener
>>> listener = XYZLogListener("xyz_id") 
>>> listener.listen("session-id", start_datetime=datetime.datetime.now()) 
>>> listener.running() 
True
stop(timeout: Optional[int] = None) → bool[source]

Cancel and clean up the listener. Blocks up to timeout seconds, or forever if None.

Returns True if the background thread stopped successfully.

Example

>>> from descarteslabs.workflows import XYZLogListener
>>> listener = XYZLogListener("xyz_id") 
>>> listener.listen("session-id", start_datetime=datetime.datetime.now()) 
>>> listener.stop() 
>>> listener.running() 
False
inspect(obj, geoctx=None, format='pyarrow', file=None, cache=True, _ruster=None, timeout=60, client=None, **arguments)[source]

Quickly compute a proxy object using a low-latency, lower-reliability backend.

Inspect is meant for getting simple computations out of Workflows, primarily for interactive use. It’s quicker but less resilient, won’t be retried if it fails, and has no progress updates.

If you have a larger computation (longer than ~30sec), or you want to be sure the computation will succeed, use compute instead. compute creates a Job, which runs asynchronously, will be retried if it fails, and stores its results for later retrieval.

Parameters:
  • obj (Proxytype) – Proxy object to compute, or list/tuple of proxy objects. If it depends on parameters, obj is first converted to a Function that takes those parameters.
  • geoctx (common.geo.geocontext.GeoContext, GeoContext, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a GeoContext, but for operations that only involve non-geospatial types, this parameter is optional.
  • format (str or dict, default "pyarrow") – The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes.
  • file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them.
  • cache (bool, default True) – Whether to use the cache for this job.
  • timeout (int, optional, default 60) – The number of seconds to wait for the result. Raises JobTimeoutError if the timeout passes.
  • client (workflows.inspect.InspectClient, optional) – Allows you to use a specific InspectClient instance with non-default auth and parameters
  • **arguments (Any) – Values for all parameters that obj depends on (or arguments that obj takes, if it’s a Function). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.
Returns:

result – When format="pyarrow" (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from descarteslabs.workflows.result_types. For other formats, returns raw bytes. Consider using file in that case to save the results to a file.
Return type:

Python object or bytes
wmts_url(tile_matrix_sets=None, dimensions=None, client=None) → str[source]

Get the WMTS endpoint which gives access to all workflows accessible to this user.

Parameters:
  • tile_matrix_sets (str or list, optional) – Desired tile matrix sets. Defaults to EPSG:3857.
  • dimensions (bool, optional) – If True, then provide dimensions definitions to WMTS. If False, then generate a layer for each possible dimensions attribute combination. If not specified, the WMTS service itself will determine how to handle dimensions (currently it does not provide dimensions definitions).
  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters.
Returns:

wmts_url – The URL for the WMTS service endpoint corresponding to this channel.
Return type:

str

Example

>>> import descarteslabs.workflows as wf
>>> wf.wmts_url() 
'https://workflows.prod.descarteslabs.com/master/wmts/workflow/1.0.0/WMTSCapabilities.xml'
compute(obj, geoctx=None, format='pyarrow', destination='download', file=None, timeout=None, block=True, progress_bar=None, cache=True, _ruster=None, _trace=False, client=None, num_retries=None, **arguments)[source]

Compute a proxy object and wait for its result.

If the caller has too many outstanding compute jobs, this will raise a ResourceExhausted exception.

Parameters:
  • obj (Proxytype) – Proxy object to compute, or list/tuple of proxy objects. If it depends on parameters, obj is first converted to a Function that takes those parameters.
  • geoctx (GeoContext, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a GeoContext, but for operations that only involve non-geospatial types, this parameter is optional.
  • format (Str or Dict, default "pyarrow") –

    The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes or None.

  • destination (str or dict, default "download") – The destination for the result. See the destinations documentation for more information.
  • file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them.
  • timeout (Int, optional) – The number of seconds to wait for the result, if block is True. Raises JobTimeoutError if the timeout passes.
  • block (Bool, default True) – If True (default), block until the job is completed, or timeout has passed. If False, immediately returns a Job (which has already had execute called).
  • progress_bar (Bool, default None) – Whether to draw the progress bar. If None (default), will display a progress bar in Jupyter Notebooks, but not elsewhere. Ignored if block==False.
  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters
  • num_retries (Int, optional) – The number of retries to make in the event of a request failure. If you are making numerous long-running asynchronous requests, you can use this parameter as a way to indicate that you are comfortable waiting and retrying in response to RESOURCE EXHAUSTED errors. By default, most failures will trigger a small number of retries, but if you have reached your outstanding job limit, by default, the client will not retry. This parameter is unnecessary when making synchronous compute requests (ie. block=True, the default). See the compute section of the Workflows Guide for more information.
  • **arguments (Any) – Values for all parameters that obj depends on (or arguments that obj takes, if it’s a Function). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.
Returns:

result – When format="pyarrow" (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from descarteslabs.workflows.result_types. For other formats, returns raw bytes. Consider using file in that case to save the results to a file. If the destination doesn’t support retrieving results (like “email”), returns None.
Return type:

Python object, bytes, or None
Raises:

RetryError – Raised if there are too many failed retries. Inspect RetryError.exceptions to determine the ultimate cause of the error. If you reach your maximum number of outstanding compute jobs, there will be one or more ResourceExhausted exceptions.

Examples

>>> import descarteslabs.workflows as wf
>>> num = wf.Int(1) + 1
>>> wf.compute(num) 
2
>>> # same computation but do not block
>>> job = wf.compute(num, block=False) 
>>> job 
<descarteslabs.workflows.models.job.Job object at 0x...>
>>> job.result() 
2
>>> # pass multiple proxy objects to `wf.compute` to compute all at once
>>> wf.compute((num, num, num)) 
(2, 2, 2)

>>> # specifying a format
>>> img = wf.Image.from_id("sentinel-2:L1C:2019-05-04_13SDV_99_S2B_v1").pick_bands("red")
>>> wf.compute(img, geoctx=ctx, format="pyarrow") # default 
ImageResult:
...
>>> # same computation but with json format
>>> wf.compute(img, geoctx=ctx, format="json") 
b'{"ndarray":[[[0.39380000000000004,0.3982,0.3864,...
>>> # same computation but with geotiff format (and some format options)
>>> bytes_ = wf.compute(img, geoctx=ctx, format={"type": "geotiff", "tiled": False}) 
>>> # you probably want to save the geotiff to a file:
>>> wf.compute(img, geoctx=ctx, file="my_geotiff.tif", format={"type": "geotiff", "tiled": False}) 

>>> # specifying a destination
>>> num = wf.Int(1) + 1
>>> wf.compute(num, destination="download") # default 
2
>>> # same computation but with email destination
>>> wf.compute(num, destination="email") 
>>> # now with some destination options
>>> wf.compute(
...     num,
...     destination={
...         "type": "email",
...         "subject": "My Computation is Done"
...     },
...     format="json",
... )
publish(id, version, obj=None, title='', description='', labels=None, tags=None, docstring='', version_labels=None, viz_options=None, client=None)[source]

Publish a proxy object as a Workflow with the given version. Can also be used as a decorator.

If the proxy object depends on any parameters (obj.params is not empty), it’s first internally converted to a Function that takes those parameters (using Function.from_object).

Parameters:
  • id (Str) – ID for the new Workflow object. This should be of the form email:workflow_name and should be globally unique. If this ID is not of the proper format, you will not be able to save the Workflow.
  • version (Str) – The version to be set, tied to the given obj. This should adhere to the semantic versioning schema.
  • obj (Proxytype, optional) –

    The object to store as this version. If it depends on parameters, obj is first converted to a Function that takes those parameters.

    If not provided, it’s assumed that set_version is being used as a decorator on a function.

  • title (Str, default "") – User-friendly title for the Workflow.
  • description (str, default "") – Long-form description of this Workflow. Markdown is supported.
  • labels (Dict, optional) – Key-value pair labels to add to the Workflow.
  • tags (list, optional) – A list of strings to add as tags to the Workflow.
  • docstring (Str, default "") – The docstring for this version.
  • version_labels (Dict, optional) – Key-value pair labels to add to the version.
  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters
Returns:

workflow – The saved Workflow object. workflow.id contains the ID of the new Workflow. If used as a decorator, returns the Function instead.
Return type:

Workflow or Function

Examples

>>> import descarteslabs.workflows as wf
>>> @wf.publish("bob@gmail.com:ndvi", "0.0.1") 
... def ndvi(img: wf.Image) -> wf.Image:
...     "Compute the NDVI of an Image"
...     nir, red = img.unpack_bands("nir red")
...     return (nir - red) / (nir + red)
>>> # `ndvi` becomes a Function proxy object
>>> ndvi  
<descarteslabs.workflows.types.Function[Image, {}, Image] object at 0x...>

>>> two = wf.Int(1) + 1
>>> workflow = wf.publish("bob@gmail.com:two", "1.0.0", two) 
>>> workflow 
<descarteslabs.workflows.models.workflow.Workflow object at 0x...>
>>> workflow.version_names 
["1.0.0"]
>>> workflow["1.0.0"].object 
<descarteslabs.workflows.types.Int object at 0x...>

If you publish an object that depends on parameters, it gets turned into a Function that takes those parameters:

>>> something_plus_one = wf.parameter("x", wf.Int) + 1
>>> workflow = wf.publish("bob@gmail.com:plus_one", "1.0.0", something_plus_one) 
>>> # `something_plus_one` depended on an Int parameter,
>>> # so the stored object turned into a Function that takes an Int
>>> add_one_func = workflow["1.0.0"].object 
>>> add_one_func 
<descarteslabs.workflows.types.Function[Int, {}, Int] object at 0x...>
>>> add_one_func(2).inspect() 
3
use(workflow_id, version, client=None)[source]

Use like import: load the proxy object of a published Workflow version.

Parameters:
  • workflow_id (Str) – ID of the Workflow to retrieve
  • version (Str) – Version of the workflow to retrive
  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters
Returns:

obj – Proxy object of the Workflow version.
Return type:

Proxytype

Examples

>>> import descarteslabs.workflows as wf
>>> @wf.publish("bob@gmail.com:ndvi", "0.0.1") 
... def ndvi(img: wf.Image) -> wf.Image:
...     nir, red = img.unpack_bands("nir red")
...     return (nir - red) / (nir + red)

>>> same_function = wf.use("bob@gmail.com:ndvi", "0.0.1") 
>>> same_function 
<descarteslabs.workflows.types.function.function.Function[Image, {}, Image] object at 0x...>
>>> img = wf.Image.from_id("sentinel-2:L1C:2019-05-04_13SDV_99_S2B_v1")
>>> same_function(img).compute(geoctx) # geoctx is an arbitrary geocontext for 'img' 
ImageResult:
...