Execution

Exceptions

JobComputeError(job)

TimeoutError

Classes

Workflow(proxy_object, proto_message[, client])

A proxy object, and metadata about it.

Job(message[, client])

A Job represents the computation of a proxy object’s graft within a specific environment of parameters.

XYZ(proxy_object, proto_message[, client])

Stores proxy objects to be rendered by an XYZ tile server.

XYZErrorListener(xyz_id[, client])

Calls callback functions in a background thread when XYZ errors occur.

Functions

compute(obj[, timeout, block, progress_bar, …])

Compute a proxy object and wait for its result.

publish(obj[, name, description, client])

Publish a proxy object as a Workflow.

retrieve(workflow_id[, client])

Load a published Workflow object.

use(workflow_id[, client])

Use like import: load the proxy object of a published Workflow.

exception JobComputeError(job)[source]
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

args
property code
property id
property message
exception TimeoutError[source]
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

args
class Workflow(proxy_object, proto_message, client=None)[source]

A proxy object, and metadata about it.

To construct, use:

Methods

build(proxy_object[, name, description, client])

Construct a new Workflow from a proxy object.

get(workflow_id[, client])

Get an existing workflow by id.

save()

Persist this Workflow.

update([proxy_object, name, description])

Update the proxy object and/or metadata of this Workflow.

Attributes

channel

The channel name this Workflow is compatible with.

created_timestamp

The UTC date this Workflow was created, or None if it hasn’t been saved yet.

description

A long-form description of this workflow.

id

The globally unique identifier for the Workflow, or None if it hasn’t been saved yet.

name

The name of this Workflow.

object

The proxy object of this Workflow.

owners

readers

type

The type of the proxy object.

updated_timestamp

The UTC date this Workflow was most recently modified, or None if it hasn’t been saved yet.

writers

Construct a Workflow object from a proxy object and Protobuf message.

Do not use this method directly; use the Workflow.build and Workflow.get classmethods instead.

Parameters
  • proxy_object (Proxytype) – The proxy object to store in this Workflow

  • proto_message (workflow_pb2.Workflow message) – Protobuf message for the Workflow

  • client (Compute, optional) – Allows you to use a specific client instance with non-default auth and parameters

classmethod build(proxy_object, name='', description='', client=None)[source]

Construct a new Workflow from a proxy object.

Note that this does not persist the Workflow, call save() on the returned Workflow to do that.

Parameters
  • proxy_object (Proxytype) – The proxy object to store in this Workflow

  • name (str, default "") – Name for the new Workflow

  • description (str, default "") – Long-form description of this Workflow. Markdown is supported.

  • client (Compute, optional) – Allows you to use a specific client instance with non-default auth and parameters

Returns

Return type

Workflow

classmethod get(workflow_id, client=None)[source]

Get an existing workflow by id.

Parameters
  • id (string) – The unique id of a Workflow

  • client (Compute, optional) – Allows you to use a specific client instance with non-default auth and parameters

Returns

Return type

Workflow

save()[source]

Persist this Workflow.

After saving, self.id will contain the new ID of the Workflow.

update(proxy_object=None, name=None, description=None)[source]

Update the proxy object and/or metadata of this Workflow.

Values given as None are left unchanged.

Parameters
  • proxy_object (Proxytype or None, default None) – New proxy object for this workflow

  • name (str or None, default None) – New name for the Workflow

  • description (str or None, default None) – New long-form description of this Workflow. Markdown is supported.

property channel

The channel name this Workflow is compatible with.

Type

str

property created_timestamp

The UTC date this Workflow was created, or None if it hasn’t been saved yet. Cannot be modified.

Type

datetime.datetime or None

property description

A long-form description of this workflow. Markdown is supported.

Type

str

property id

The globally unique identifier for the Workflow, or None if it hasn’t been saved yet.

Type

str or None

property name

The name of this Workflow.

Type

str

property object

The proxy object of this Workflow.

Raises ValueError if the Workflow is not compatible with the current channel.

Type

Proxytype

property owners
property readers
property type

The type of the proxy object.

Type

type

property updated_timestamp

The UTC date this Workflow was most recently modified, or None if it hasn’t been saved yet. Updated automatically.

Type

datetime.datetime or None

property writers
class Job(message, client=None)[source]

A Job represents the computation of a proxy object’s graft within a specific environment of parameters.

Example

Attributes

BUCKET_PREFIX

str(object=’‘) -> str

WAIT_INTERVAL

Convert a string or number to a floating point number, if possible.

channel

The channel name where this Job will execute.

created_datetime

done

error

id

The globally unique identifier for the Job, or None if it hasn’t been executed yet.

object

The proxy object this Job computes.

parameters

result_type

Name of the type of object that will be used to hold the result

runtime

stage

status

type

The type of the proxy object.

updated_datetime

Methods

build(proxy_object, parameters[, channel, …])

Build a new Job for computing a proxy object under certain parameters.

cancel()

Cancel a running job.

execute()

Asynchronously submit a job for execution.

get(id[, client])

Get a currently-running Job by its ID.

refresh()

Refresh the attributes and status of the job.

result([timeout, progress_bar])

Get the result of the job.

watch()

>>> import descarteslabs.workflows as wf
>>> img = wf.Image.from_id("landsat:LC08:PRE:TOAR:meta_LC80270312016188_v1")
>>> job = img.compute(geoctx, block=False)  
>>> result = job.result()  
classmethod build(proxy_object, parameters, channel=None, client=None)[source]

Build a new Job for computing a proxy object under certain parameters. Does not actually trigger computation; call Job.execute on the result to do so.

Parameters
  • proxy_object (Proxytype) – Proxy object to compute

  • parameters (dict[str, Proxytype]) – Python dictionary of parameter names and values

  • channel (str or None, optional) –

    Channel name to submit the Job to. If None, uses the default channel for this client (descarteslabs.workflows.__channel__).

    Channels are different versions of the backend, to allow for feature changes without breaking existing code. Not all clients are compatible with all channels. This client is only guaranteed to work with its default channel, whose name can be found under descarteslabs.workflows.__channel__.

  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters

Returns

The job waiting to be executed.

Return type

Job

cancel()[source]

Cancel a running job.

execute()[source]

Asynchronously submit a job for execution.

After submission, self.id will be the ID of the running job.

This method is idempotent: calling it multiple times on the same Job object will only trigger execution once.

classmethod get(id, client=None)[source]

Get a currently-running Job by its ID.

refresh()[source]

Refresh the attributes and status of the job.

result(timeout=None, progress_bar=None)[source]

Get the result of the job. This blocks until the job is complete.

Parameters
  • timeout (int, optional) – The number of seconds to wait for the result.

  • progress_bar (bool, optional) – Flag to draw the progress bar. Default is to True if in Jupyter Notebook.

Returns

Appropriate Python object representing the result, either as a plain Python type, or object from descarteslabs.common.workflows.results.

Return type

result

Example

>>> import descarteslabs.workflows as wf
>>> job = wf.Int(1).compute()  
>>> job.result(timeout=10)  
1
watch()[source]
BUCKET_PREFIX = 'https://storage.googleapis.com/dl-compute-dev-results/{}'
WAIT_INTERVAL = 0.1
property channel

The channel name where this Job will execute.

Type

str

property created_datetime
property done
property error
property id

The globally unique identifier for the Job, or None if it hasn’t been executed yet.

Type

str or None

property object

The proxy object this Job computes.

Type

Proxytype

property parameters
property result_type

Name of the type of object that will be used to hold the result

Type

str

property runtime
property stage
property status
property type

The type of the proxy object.

Type

type

property updated_datetime
class XYZ(proxy_object, proto_message, client=None)[source]

Stores proxy objects to be rendered by an XYZ tile server.

Similar to a Workflow, but meant for storing proxy objects so the XYZ tile service can display them, rather than for persisting and sharing workflows between users.

Use url to generate an XYZ URL template, and iter_tile_errors or error_listener to retrieve error messages that happen while computing them.

Construct a XYZ object from a proxy object and Protobuf message.

Do not use this method directly; use the XYZ.build and XYZ.get classmethods instead.

Parameters
  • proxy_object (Proxytype) – The proxy object to store in this XYZ

  • proto_message (xyz_pb2.XYZ message) – Protobuf message for the XYZ

  • client (Compute, optional) – Allows you to use a specific client instance with non-default auth and parameters

Attributes

BASE_URL

str(object=’‘) -> str

channel

The channel name this XYZ is compatible with.

created_timestamp

The UTC date this XYZ was created, or None if it hasn’t been saved yet.

description

A long-form description of this xyz.

id

The globally unique identifier for the XYZ, or None if it hasn’t been saved yet.

name

The name of this XYZ.

object

The proxy object of this XYZ.

owners

readers

type

The type of the proxy object.

updated_timestamp

The UTC date this XYZ was most recently modified, or None if it hasn’t been saved yet.

writers

Methods

build(proxy_object[, name, description, client])

Construct a new XYZ from a proxy object.

error_listener()

An XYZErrorListener to trigger callbacks when errors occur computing tiles

get(xyz_id[, client])

Get an existing XYZ by id.

iter_tile_errors(session_id[, start_datetime])

Iterator over errors generated while computing tiles

save()

Persist this XYZ layer.

update(*args, **kwargs)

url([session_id, colormap, scales, checkerboard])

XYZ tile URL format-string, like https://workflows.descarteslabs.com/v0-5/xyz/1234567/{z}/{x}/{y}.png

classmethod build(proxy_object, name='', description='', client=None)[source]

Construct a new XYZ from a proxy object.

Note that this does not persist the XYZ, call save() on the returned XYZ to do that.

Parameters
  • proxy_object (Proxytype) – The proxy object to store in this XYZ

  • name (str, default "") – Name for the new XYZ

  • description (str, default "") – Long-form description of this XYZ. Markdown is supported.

  • client (Compute, optional) – Allows you to use a specific client instance with non-default auth and parameters

Returns

Return type

XYZ

error_listener()[source]

An XYZErrorListener to trigger callbacks when errors occur computing tiles

classmethod get(xyz_id, client=None)[source]

Get an existing XYZ by id.

Parameters
  • id (string) – The unique id of a XZY object

  • client (Compute, optional) – Allows you to use a specific client instance with non-default auth and parameters

Returns

Return type

XYZ

iter_tile_errors(session_id, start_datetime=None)[source]

Iterator over errors generated while computing tiles

Parameters
  • session_id (str) – Unique, client-generated that error logs are stored under.

  • start_datetime (datetime.datetime) – Only return errors occuring after this datetime

Yields

error (descarteslabs.common.proto.xyz_pb2.XYZError) – Errors in protobuf message objects, with fields code, message, timestamp, session_id.

save()[source]

Persist this XYZ layer.

After saving, self.id will contain the new ID of the XYZ layer.

update(*args, **kwargs)[source]
url(session_id=None, colormap=None, scales=None, checkerboard=False, **parameters)[source]

XYZ tile URL format-string, like https://workflows.descarteslabs.com/v0-5/xyz/1234567/{z}/{x}/{y}.png

Parameters
  • session_id (str, optional, default None) – Unique, client-generated ID that error logs will be stored under. Since multiple users may access tiles from the same XYZ object, each user should set their own session_id to get individual error logs.

  • colormap (str, optional, default None) – Name of the colormap to use. If set, the displayed Image must have 1 band.

  • scales (list of lists, optional, default None) –

    The scaling to apply to each band in the Image this XYZ object will display.

    If the Image contains 3 bands, scales must be a list like [(0, 1), (0, 1), (-1, 1)].

    If the Image contains 1 band, scales must be a list like [(0, 1)], or just (0, 1) for convenience

    If None, each 256x256 tile will be scaled independently.

  • checkerboard (bool, default False) – Whether to display a checkerboarded background for missing or masked data.

  • parameters (dict[str, Union[Proxytype, json_serializable_value]]) –

    Parameters to use while computing.

    Each argument must be the name of a parameter created with parameter. Each value must be a JSON-serializable type (bool, int, float, str, list, dict, etc.), a Proxytype (like Image or Timedelta), or a value that proxify can handle (like a datetime.datetime).

Returns

url

Return type

str

Raises
  • ValueError – If the XYZ object has no id and save has not been called yet.

  • TypeError – If the scales or parameters are of invalid type.

BASE_URL = 'https://workflows.descarteslabs.com'
property channel

The channel name this XYZ is compatible with.

Type

str

property created_timestamp

The UTC date this XYZ was created, or None if it hasn’t been saved yet. Cannot be modified.

Type

datetime.datetime or None

property description

A long-form description of this xyz. Markdown is supported.

Type

str

property id

The globally unique identifier for the XYZ, or None if it hasn’t been saved yet.

Type

str or None

property name

The name of this XYZ.

Type

str

property object

The proxy object of this XYZ.

Raises ValueError if the XYZ is not compatible with the current channel.

Type

Proxytype

property owners
property readers
property type

The type of the proxy object.

Type

type

property updated_timestamp

The UTC date this XYZ was most recently modified, or None if it hasn’t been saved yet. Updated automatically.

Type

datetime.datetime or None

property writers
class XYZErrorListener(xyz_id, client=None)[source]

Calls callback functions in a background thread when XYZ errors occur.

Note: the thread is automatically cleaned up on garbage collection.

Example

Methods

add_callback(callback)

Function will be called with descarteslabs.common.proto.xyz_pb2.XYZError on each error.

listen(session_id[, start_datetime])

Start listening for errors.

running()

bool: whether this is an active listener

stop([timeout])

Cancel and clean up the listener.

>>> import descarteslabs.workflows as wf
>>> xyz = wf.XYZ.build(wf.Image.from_id("landsat:LC08:PRE:TOAR:meta_LC80270312016188_v1"))
>>> xyz.save()  
>>> listener = xyz.error_listener()
>>> def callback(msg):
...     print(msg.code, msg.message)
>>> listener.add_callback(callback)
>>> listener.listen("my_session_id", start_datetime=datetime.datetime.now())  
>>> # later
>>> listener.stop()  
add_callback(callback)[source]

Function will be called with descarteslabs.common.proto.xyz_pb2.XYZError on each error.

Parameters

callback (callable) –

Function that takes one argument, a descarteslabs.common.proto.xyz_pb2.XYZError protobuf message object. This message contains the fields code, message, timestamp, session_id.

The function will be called within a separate thread, therefore it must behave thread-safely. Any errors raised by the function will terminate the listener.

listen(session_id, start_datetime=None)[source]

Start listening for errors.

Parameters
  • session_id (str) – Unique, client-generated ID that error logs are stored under. See XYZ.url for more information.

  • start_datetime (datetime.datetime) – Only listen for errors occuring after this datetime.

running()[source]

bool: whether this is an active listener

stop(timeout=None)[source]

Cancel and clean up the listener. Blocks up to timeout seconds, or forever if None.

Returns True if the background thread stopped successfully.

compute(obj, timeout=None, block=True, progress_bar=None, channel=None, client=None, **params)[source]

Compute a proxy object and wait for its result.

Parameters
  • obj (Proxytype) – A proxy object to compute

  • timeout (int, optional) – The number of seconds to wait for the result, if block is True. Raises TimeoutError if the timeout passes.

  • block (bool, default True) – If True (default), block until the job is completed, or timeout has passed. If False, immediately returns a Job (which has already had execute called).

  • progress_bar (bool, default None) – Whether to draw the progress bar. If None (default), will display a progress bar in Jupyter Notebooks, but not elsewhere. Ignored if block==False.

  • channel (str or None, optional) –

    Channel name to submit the Job to. If None, uses the default channel for this client (descarteslabs.workflows.__channel__).

    Channels are different versions of the backend, to allow for feature changes without breaking existing code. Not all clients are compatible with all channels. This client is only guaranteed to work with its default channel, whose name can be found under descarteslabs.workflows.__channel__.

  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters

  • **params (Proxytype) – Parameters under which to run the computation, such as geoctx.

Returns

Appropriate Python object representing the result, either as a plain Python type, or object from descarteslabs.workflows.results.

Return type

result

publish(obj, name='', description='', client=None)[source]

Publish a proxy object as a Workflow.

Parameters
  • obj (Proxytype) – A proxy object to compute

  • name (str, default "") – Name for the new Workflow

  • description (str, default "") – Long-form description of this Workflow. Markdown is supported.

  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters

Returns

workflow – The saved Workflow object. workflow.id contains the ID of the new Workflow.

Return type

Workflow

retrieve(workflow_id, client=None)[source]

Load a published Workflow object.

Parameters
  • workflow_id (str) – ID of the Workflow to retrieve

  • client (Compute, optional) – Allows you to use a specific client instance with non-default auth and parameters

Returns

workflow – Object representing the workflow, including both its metadata (like workflow.name, workflow.description) and proxy object (workflow.object).

Return type

Workflow

use(workflow_id, client=None)[source]

Use like import: load the proxy object of a published Workflow.

Shorthand for retrieve(workflow_id).object.

Parameters
  • workflow_id (str) – ID of the Workflow to retrieve

  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters

Returns

obj – Proxy object of the Workflow.

Return type

Proxytype