Execution

Exceptions

JobComputeError(job)

Generic error raised when a job computation fails.

JobTimeoutError

Raised when a computation took longer to complete than a specified timeout.

Classes

Workflow(proxy_object, proto_message[, client])

A proxy object, and metadata about it.

Job(proxy_object, parameters[, format, …])

A Job represents the computation of a proxy object’s graft within a specific environment of parameters.

XYZ(proxy_object, proto_message[, client])

Stores proxy objects to be rendered by an XYZ tile server.

XYZErrorListener(xyz_id[, client])

Calls callback functions in a background thread when XYZ errors occur.

Functions

compute(obj[, geoctx, format, destination, …])

Compute a proxy object and wait for its result.

publish(obj[, name, description, client])

Publish a proxy object as a Workflow.

retrieve(workflow_id[, client])

Load a published Workflow object.

use(workflow_id[, client])

Use like import: load the proxy object of a published Workflow.

exception JobComputeError(job)[source]

Generic error raised when a job computation fails.

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

args
property code
property id
property message
exception JobTimeoutError[source]

Raised when a computation took longer to complete than a specified timeout.

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

args
class Workflow(proxy_object, proto_message, client=None)[source]

A proxy object, and metadata about it.

To construct, use:

Methods

build(proxy_object[, name, description, client])

Construct a new Workflow from a proxy object.

get(workflow_id[, client])

Get an existing workflow by id.

save()

Persist this Workflow.

Attributes

channel

The channel name this Workflow is compatible with.

created_timestamp

The UTC date this Workflow was created, or None if it hasn’t been saved yet.

description

A long-form description of this workflow.

id

The globally unique identifier for the Workflow, or None if it hasn’t been saved yet.

name

The name of this Workflow.

object

The proxy object of this Workflow.

type

The type of the proxy object.

updated_timestamp

The UTC date this Workflow was most recently modified, or None if it hasn’t been saved yet.

Examples

>>> from descarteslabs.workflows import Int, Workflow, retrieve
>>> num = Int(1) + 1
>>> workflow = Workflow.build(num, name="one-plus-one", description="The result of 1 plus 1")
>>> workflow
<descarteslabs.workflows.models.workflow.Workflow object at 0x...>
>>> workflow.save() 
>>> workflow.id 
'0eb6676dbe2de3ceb1990d9669f4ceb35c9309795d842c86'
>>> same_workflow = retrieve('0eb6676dbe2de3ceb1990d9669f4ceb35c9309795d842c86') 
>>> same_workflow.object 
<descarteslabs.workflows.types.primitives.number.Int object at 0x...>
>>> same_workflow.object.compute() 
2

Construct a Workflow object from a proxy object and Protobuf message.

Do not use this method directly; use the Workflow.build and Workflow.get classmethods instead.

Parameters
  • proxy_object (Proxytype) – The proxy object to store in this Workflow

  • proto_message (workflow_pb2.Workflow message) – Protobuf message for the Workflow

  • client (Compute, optional) – Allows you to use a specific client instance with non-default auth and parameters

classmethod build(proxy_object, name='', description='', client=None)[source]

Construct a new Workflow from a proxy object.

Note that this does not persist the Workflow, call save() on the returned Workflow to do that.

Parameters
  • proxy_object (Proxytype) – The proxy object to store in this Workflow

  • name (str, default "") – Name for the new Workflow

  • description (str, default "") – Long-form description of this Workflow. Markdown is supported.

  • client (Compute, optional) – Allows you to use a specific client instance with non-default auth and parameters

Returns

Return type

Workflow

Example

>>> from descarteslabs.workflows import Workflow, Int
>>> my_int = Int(1) + 1
>>> workflow = Workflow.build(my_int, name="one-plus-one", description="The result of 1 plus 1")
>>> workflow
<descarteslabs.workflows.models.workflow.Workflow object at 0x...>
classmethod get(workflow_id, client=None)[source]

Get an existing workflow by id.

Parameters
  • id (string) – The unique id of a Workflow

  • client (Compute, optional) – Allows you to use a specific client instance with non-default auth and parameters

Returns

Return type

Workflow

Example

>>> from descarteslabs.workflows import Workflow
>>> workflow = Workflow.get('0eb6676dbe2de3ceb1990d9669f4ceb35c9309795d842c86') 
>>> workflow 
<descarteslabs.workflows.models.workflow.Workflow object at 0x...>
save()[source]

Persist this Workflow.

After saving, self.id will contain the new ID of the Workflow.

Example

>>> from descarteslabs.workflows import Workflow, Int
>>> my_int = Int(1) + 1
>>> workflow = Workflow.build(my_int, name="one-plus-one", description="The result of 1 plus 1")
>>> workflow.save() 
>>> workflow.id 
'0eb6676dbe2de3ceb1990d9669f4ceb35c9309795d842c86'
property channel

The channel name this Workflow is compatible with.

Type

str

property created_timestamp

The UTC date this Workflow was created, or None if it hasn’t been saved yet. Cannot be modified.

Type

datetime.datetime or None

property description

A long-form description of this workflow. Markdown is supported.

Type

str

property id

The globally unique identifier for the Workflow, or None if it hasn’t been saved yet.

Type

str or None

property name

The name of this Workflow.

Type

str

property object

The proxy object of this Workflow.

Raises ValueError if the Workflow is not compatible with the current channel.

Type

Proxytype

property type

The type of the proxy object.

Type

type

property updated_timestamp

The UTC date this Workflow was most recently modified, or None if it hasn’t been saved yet. Updated automatically.

Type

datetime.datetime or None

class Job(proxy_object, parameters, format='pyarrow', destination='download', client=None, cache=True)[source]

A Job represents the computation of a proxy object’s graft within a specific environment of parameters.

Example

Attributes

BUCKET_PREFIX

str(object=’‘) -> str

cache_enabled

Whether caching is enabled for this job.

channel

The channel name where this Job will execute.

created_datetime

The time the Job was created.

destination

The destination for the Job results, as a dictionary.

done

Whether the Job has completed or not.

error

The error of the Job, or None if it finished successfully.

format

The serialization format of the Job, as a dictionary.

id

The globally unique identifier for the Job, or None if it hasn’t been executed yet.

object

The proxy object this Job computes.

parameters

The parameters of the Job, as a graft.

result_type

Name of the type of object that will be used to hold the result

runtime

The total time it took the Job to run.

stage

The current stage of the Job (queued, preparing, running, saving, succeeded, failed).

type

The type of the proxy object.

updated_datetime

The time of the most recent Job update.

url

The download URL for this Job’s results.

Methods

get(id[, client])

Get a currently-running Job by its ID.

refresh()

Refresh the attributes and state of the job.

result([timeout, progress_bar])

Get the result of the job.

result_to_file(file[, timeout, progress_bar])

Save the result of the job to a file.

wait([timeout, progress_bar])

Block until the Job is complete, optionally displaying a progress bar.

watch()

Generator that yields self each time an update to the Job occurs.

>>> from descarteslabs.workflows import Int, Job
>>> num = Int(1) + 1
>>> job = num.compute(block=False)  
>>> job 
<descarteslabs.workflows.models.job.Job object at 0x...>
>>> job.id 
'3754676080bbb2b857fbc04a3e48f6312732e1bc42e0bd7b'
>>> job.result() 
2
>>> same_job = Job.get('3754676080bbb2b857fbc04a3e48f6312732e1bc42e0bd7b') 
>>> same_job.stage 
'STAGE_DONE'
>>> same_job.result 
2

Creates a new Job to compute the provided proxy object with the given parameters.

Parameters
  • proxy_object (Proxytype) – Proxy object to compute

  • parameters (dict[str, Proxytype]) – Python dictionary of parameter names and values

  • format (str or dict, default "pyarrow") – The serialization format for the result.

  • destination (str or dict, default "download") – The destination for the result.

  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters

  • cache (bool, default True) – Whether to use the cache for this job.

Returns

The job that’s executing.

Return type

Job

Example

>>> from descarteslabs.workflows import Job, Int, parameter
>>> my_int = Int(1) + parameter("other_int", Int)
>>> job = Job(my_int, {"other_int": 10}) 
>>> job.stage 
QUEUED
classmethod get(id, client=None)[source]

Get a currently-running Job by its ID.

Parameters
  • id (string) – The ID of a running job.

  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters

Example

>>> from descarteslabs.workflows import Job
>>> job = Job.get('3754676080bbb2b857fbc04a3e48f6312732e1bc42e0bd7b') 
refresh()[source]

Refresh the attributes and state of the job.

Example

>>> from descarteslabs.workflows import Job, Int
>>> job = Job(Int(1), {}) 
>>> job.stage 
QUEUED
>>> job.refresh() 
>>> job.stage 
SUCCEEDED
result(timeout=None, progress_bar=None)[source]

Get the result of the job. This blocks until the job is complete.

Only the “download” destination can be retrieved. Raises NotImplementedError for other destinations.

Parameters
  • timeout (int, optional) – The number of seconds to wait for the result.

  • progress_bar (bool, optional) – Flag to draw the progress bar. Default is to True if in Jupyter Notebook.

Returns

result – When the Job’s format is “pyarrow”, returns a Python object representing the result, either as a plain Python type, or object from descarteslabs.workflows.result_types. For other formats, returns raw bytes. Consider using result_to_file in that case to save the results to a file.

Return type

Python object or bytes

Example

>>> from descarteslabs.workflows import Job, Int
>>> job = Job(Int(1), {}) 
>>> job.result() 
1
result_to_file(file, timeout=None, progress_bar=None)[source]

Save the result of the job to a file. This blocks until the job is complete.

Only the “download” destination can be written to a file. For destinations like “catalog”, where the data is handed off to another service, you’ll need to use that service to retrieve it. (In the “catalog” case, that’s Raster and Metadata.)

Parameters
  • file (path or file-like object) – Path or file where results will be written

  • timeout (int, optional) – The number of seconds to wait for the result.

  • progress_bar (bool, optional) – Flag to draw the progress bar. Default is to True if in Jupyter Notebook.

Example

>>> from descarteslabs.workflows import Job, Int
>>> job = Job(Int(1), {}, format="json") 
>>> job.result_to_file("one.json") 
>>> import io
>>> from descarteslabs.workflows import Job, Int
>>> job = Job(Int(2), {}, format="json") 
>>> bytestream = io.BytesIO() 
>>> job.result_to_file(bytestream) 
>>> print(bytestream.read()) 
b'2'
wait(timeout=None, progress_bar=False)[source]

Block until the Job is complete, optionally displaying a progress bar.

Raises any error that occurs with the Job, or JobTimeoutError if the timeout passes before the Job is complete.

Parameters
  • timeout (int, optional) – The number of seconds to wait for the result.

  • progress_bar (bool, optional) – Flag to draw the progress bar. Default is to True if in Jupyter Notebook.

watch()[source]

Generator that yields self each time an update to the Job occurs.

Example

>>> from descarteslabs.workflows import Job, Int
>>> job = Job(Int(1), {}) 
>>> for job in job.watch(): 
...     print(job.stage)
QUEUED
PREPARING
RUNNING
RUNNING
SAVING
SUCCEEDED
BUCKET_PREFIX = 'https://storage.googleapis.com/dl-compute-dev-results/{}'
property cache_enabled

Whether caching is enabled for this job.

property channel

The channel name where this Job will execute.

Type

str

property created_datetime

The time the Job was created.

Type

datetime

property destination

The destination for the Job results, as a dictionary.

property done

Whether the Job has completed or not.

Type

bool

property error

The error of the Job, or None if it finished successfully.

property format

The serialization format of the Job, as a dictionary.

property id

The globally unique identifier for the Job, or None if it hasn’t been executed yet.

Type

str or None

property object

The proxy object this Job computes.

Type

Proxytype

property parameters

The parameters of the Job, as a graft.

property result_type

Name of the type of object that will be used to hold the result

Type

str

property runtime

The total time it took the Job to run.

Type

datetime

property stage

The current stage of the Job (queued, preparing, running, saving, succeeded, failed).

property type

The type of the proxy object.

Type

type

property updated_datetime

The time of the most recent Job update.

Type

datetime

property url

The download URL for this Job’s results.

If format is not “download” or “email”, url will be None.

class XYZ(proxy_object, proto_message, client=None)[source]

Stores proxy objects to be rendered by an XYZ tile server.

Similar to a Workflow, but meant for storing proxy objects so the XYZ tile service can display them, rather than for persisting and sharing workflows between users.

Use url to generate an XYZ URL template, and iter_tile_errors or error_listener to retrieve error messages that happen while computing them.

Examples

Attributes

BASE_URL

str(object=’‘) -> str

channel

The channel name this XYZ is compatible with.

created_timestamp

The UTC date this XYZ was created, or None if it hasn’t been saved yet.

description

A long-form description of this xyz.

id

The globally unique identifier for the XYZ, or None if it hasn’t been saved yet.

name

The name of this XYZ.

object

The proxy object of this XYZ.

type

The type of the proxy object.

updated_timestamp

The UTC date this XYZ was most recently modified, or None if it hasn’t been saved yet.

Methods

build(proxy_object[, name, description, client])

Construct a new XYZ from a proxy object.

error_listener()

An XYZErrorListener to trigger callbacks when errors occur computing tiles.

get(xyz_id[, client])

Get an existing XYZ by id.

iter_tile_errors(session_id[, start_datetime])

Iterator over errors generated while computing tiles

save()

Persist this XYZ layer.

url([session_id, colormap, scales, checkerboard])

XYZ tile URL format-string, like https://workflows.descarteslabs.com/v0-5/xyz/1234567/{z}/{x}/{y}.png

>>> from descarteslabs.workflows import Image, XYZ
>>> img = Image.from_id("sentinel-2:L1C:2019-05-04_13SDV_99_S2B_v1")
>>> rgb = img.pick_bands("red green blue")
>>> xyz = XYZ.build(rgb, name="My RGB") 
>>> xyz.save() 
>>> xyz 
<descarteslabs.workflows.models.xyz.XYZ object at 0x...>
>>> xyz.id 
'24d0e79c5c1e1f10a0b1177ef3974d7edefd5988291cf2c6'
>>> same_xyz = XYZ.get('24d0e79c5c1e1f10a0b1177ef3974d7edefd5988291cf2c6') 
>>> same_xyz.url() 
'https://workflows.descarteslabs.com/master/xyz/24d0e79c5c1e1f10a0b1177ef3974d7edefd5988291cf2c6/{z}/{x}/{y}.png'
>>> same_xyz.object 
<descarteslabs.workflows.types.geospatial.image.Image object at 0x...>

Construct a XYZ object from a proxy object and Protobuf message.

Do not use this method directly; use the XYZ.build and XYZ.get classmethods instead.

Parameters
  • proxy_object (Proxytype) – The proxy object to store in this XYZ

  • proto_message (xyz_pb2.XYZ message) – Protobuf message for the XYZ

  • client (Compute, optional) – Allows you to use a specific client instance with non-default auth and parameters

classmethod build(proxy_object, name='', description='', client=None)[source]

Construct a new XYZ from a proxy object.

Note that this does not persist the XYZ, call save() on the returned XYZ to do that.

Parameters
  • proxy_object (Proxytype) – The proxy object to store in this XYZ

  • name (str, default "") – Name for the new XYZ

  • description (str, default "") – Long-form description of this XYZ. Markdown is supported.

  • client (Compute, optional) – Allows you to use a specific client instance with non-default auth and parameters

Returns

Return type

XYZ

Example

>>> from descarteslabs.workflows import Image, XYZ
>>> img = Image.from_id("sentinel-2:L1C:2019-05-04_13SDV_99_S2B_v1")
>>> rgb = img.pick_bands("red green blue")
>>> xyz = XYZ.build(rgb, name="My RGB") 
>>> xyz 
<descarteslabs.workflows.models.xyz.XYZ object at 0x...>
error_listener()[source]

An XYZErrorListener to trigger callbacks when errors occur computing tiles.

Example

>>> from descarteslabs.workflows import Image, XYZ
>>> img = Image.from_id("sentinel-2:L1C:2019-05-04_13SDV_99_S2B_v1")
>>> rgb = img.pick_bands("red green blue")
>>> xyz = XYZ.build(rgb, name="My RGB") 
>>> url = xyz.url(session_id="some_session") 
>>> listener = xyz.error_listener() 
>>> errors_log = []
>>> listener.add_callback(lambda error: errors_log.append(error.message)) 
>>> listener.listen("some_session") 
>>> # any errors that occur loading tiles from the generated URL will be appended
>>> # to `errors_log` in the background
classmethod get(xyz_id, client=None)[source]

Get an existing XYZ by id.

Parameters
  • id (string) – The unique id of a XZY object

  • client (Compute, optional) – Allows you to use a specific client instance with non-default auth and parameters

Returns

Return type

XYZ

Example

>>> from descarteslabs.workflows import XYZ
>>> xyz = XYZ.get('24d0e79c5c1e1f10a0b1177ef3974d7edefd5988291cf2c6') 
>>> xyz 
<descarteslabs.workflows.models.xyz.XYZ object at 0x...>
iter_tile_errors(session_id, start_datetime=None)[source]

Iterator over errors generated while computing tiles

Parameters
  • session_id (str) – Unique, client-generated that error logs are stored under.

  • start_datetime (datetime.datetime) – Only return errors occuring after this datetime

Yields

error (descarteslabs.common.proto.xyz_pb2.XYZError) – Errors in protobuf message objects, with fields code, message, timestamp, session_id.

Example

>>> from descarteslabs.workflows import Image, XYZ
>>> img = Image.from_id("sentinel-2:L1C:2019-05-04_13SDV_99_S2B_v1")
>>> rgb = img.pick_bands("red green blue")
>>> xyz = XYZ.build(rgb, name="My RGB") 
>>> url = xyz.url(session_id="some_session") 
>>> for error in xyz.iter_tile_errors("some_session", start_datetime=datetime.datetime.now()): 
...     print(error.code, error.message)
>>>     # any errors that occur loading tiles from the generated URL will be printed here
save()[source]

Persist this XYZ layer.

After saving, self.id will contain the new ID of the XYZ layer.

Example

>>> from descarteslabs.workflows import Image, XYZ
>>> img = Image.from_id("sentinel-2:L1C:2019-05-04_13SDV_99_S2B_v1")
>>> rgb = img.pick_bands("red green blue")
>>> xyz = XYZ.build(rgb, name="My RGB") 
>>> xyz.save() 
>>> xyz.id 
'24d0e79c5c1e1f10a0b1177ef3974d7edefd5988291cf2c6'
url(session_id=None, colormap=None, scales=None, checkerboard=False, **parameters)[source]

XYZ tile URL format-string, like https://workflows.descarteslabs.com/v0-5/xyz/1234567/{z}/{x}/{y}.png

Parameters
  • session_id (str, optional, default None) – Unique, client-generated ID that error logs will be stored under. Since multiple users may access tiles from the same XYZ object, each user should set their own session_id to get individual error logs.

  • colormap (str, optional, default None) – Name of the colormap to use. If set, the displayed Image must have 1 band.

  • scales (list of lists, optional, default None) –

    The scaling to apply to each band in the Image this XYZ object will display.

    If the Image contains 3 bands, scales must be a list like [(0, 1), (0, 1), (-1, 1)].

    If the Image contains 1 band, scales must be a list like [(0, 1)], or just (0, 1) for convenience

    If None, each 256x256 tile will be scaled independently.

  • checkerboard (bool, default False) – Whether to display a checkerboarded background for missing or masked data.

  • parameters (dict[str, Union[Proxytype, json_serializable_value]]) –

    Parameters to use while computing.

    Each argument must be the name of a parameter created with parameter. Each value must be a JSON-serializable type (bool, int, float, str, list, dict, etc.), a Proxytype (like Image or Timedelta), or a value that proxify can handle (like a datetime.datetime).

Returns

url – Tile URL containing {z}, {x}, and {y} as Python format string parameters, and query arguments URL-quoted.

Return type

str

Raises
  • ValueError – If the XYZ object has no id and save has not been called yet.

  • TypeError – If the scales or parameters are of invalid type.

Example

>>> import descarteslabs.workflows as wf
>>> img = wf.Image.from_id("sentinel-2:L1C:2019-05-04_13SDV_99_S2B_v1")
>>> red = img.pick_bands("red")
>>> viz = red ** wf.parameter("exponent", wf.Float)
>>> xyz = wf.XYZ.build(viz, name="Red band raised to exponent")
>>> xyz.save() 
>>> xyz.url("some_session", colormap="magma", scales=[(0.2, 0.8)], exponent=2.5) 
'https://workflows.descarteslabs.com/master/xyz/0d21037edb4bdd16b735f24bb3bff6d4202a71c20404b101/
 {z}/{x}/{y}.png?session_id=some_session&colormap=magma&scales=[[0.2, 0.8]]&exponent=2.5'
BASE_URL = 'https://workflows.descarteslabs.com'
property channel

The channel name this XYZ is compatible with.

Type

str

property created_timestamp

The UTC date this XYZ was created, or None if it hasn’t been saved yet. Cannot be modified.

Type

datetime.datetime or None

property description

A long-form description of this xyz. Markdown is supported.

Type

str

property id

The globally unique identifier for the XYZ, or None if it hasn’t been saved yet.

Type

str or None

property name

The name of this XYZ.

Type

str

property object

The proxy object of this XYZ.

Raises ValueError if the XYZ is not compatible with the current channel.

Type

Proxytype

property type

The type of the proxy object.

Type

type

property updated_timestamp

The UTC date this XYZ was most recently modified, or None if it hasn’t been saved yet. Updated automatically.

Type

datetime.datetime or None

class XYZErrorListener(xyz_id, client=None)[source]

Calls callback functions in a background thread when XYZ errors occur.

Note: the thread is automatically cleaned up on garbage collection.

Example

Methods

add_callback(callback)

Function will be called with descarteslabs.common.proto.xyz_pb2.XYZError on each error.

listen(session_id[, start_datetime])

Start listening for errors.

running()

bool: whether this is an active listener

stop([timeout])

Cancel and clean up the listener.

>>> from descarteslabs.workflows import Image, XYZ
>>> img = Image.from_id("landsat:LC08:PRE:TOAR:meta_LC80270312016188_v1")
>>> xyz = XYZ.build(img)
>>> xyz.save()  
>>> listener = xyz.error_listener()
>>> def callback(msg):
...     print(msg.code, msg.message)
>>> listener.add_callback(callback)
>>> listener.listen("session_id", start_datetime=datetime.datetime.now())  
>>> # later
>>> listener.stop()  
add_callback(callback)[source]

Function will be called with descarteslabs.common.proto.xyz_pb2.XYZError on each error.

Parameters

callback (callable) –

Function that takes one argument, a descarteslabs.common.proto.xyz_pb2.XYZError protobuf message object. This message contains the fields code, message, timestamp, session_id.

The function will be called within a separate thread, therefore it must behave thread-safely. Any errors raised by the function will terminate the listener.

Example

>>> from descarteslabs.workflows import XYZErrorListener
>>> listener = XYZErrorListener("xyz_id") 
>>> def callback(msg):
...     print(msg.code, msg.message)
>>> listener.add_callback(callback) 
listen(session_id, start_datetime=None)[source]

Start listening for errors.

Parameters
  • session_id (str) – Unique, client-generated ID that error logs are stored under. See XYZ.url for more information.

  • start_datetime (datetime.datetime) – Only listen for errors occuring after this datetime. Must be tz-aware.

Example

>>> from descarteslabs.workflows import XYZErrorListener
>>> listener = XYZErrorListener("xyz_id") 
>>> listener.listen("session-id", start_datetime=datetime.datetime.now(datetime.timezone.utc)) 
running()[source]

bool: whether this is an active listener

Example

>>> from descarteslabs.workflows import XYZErrorListener
>>> listener = XYZErrorListener("xyz_id") 
>>> listener.listen("session-id", start_datetime=datetime.datetime.now()) 
>>> listener.running() 
True
stop(timeout=None)[source]

Cancel and clean up the listener. Blocks up to timeout seconds, or forever if None.

Returns True if the background thread stopped successfully.

Example

>>> from descarteslabs.workflows import XYZErrorListener
>>> listener = XYZErrorListener("xyz_id") 
>>> listener.listen("session-id", start_datetime=datetime.datetime.now()) 
>>> listener.stop() 
>>> listener.running() 
False
compute(obj, geoctx=None, format='pyarrow', destination='download', file=None, timeout=None, block=True, progress_bar=None, client=None, cache=True, **params)[source]

Compute a proxy object and wait for its result.

Parameters
  • obj (Proxytype) – A proxy object to compute.

  • geoctx (GeoContext, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a GeoContext, but for operations that only involve non-geospatial types, this parameter is optional.

  • format (str or dict, default "pyarrow") – The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes.

  • destination (str or dict, default "download") – The destination for the result. See the destinations documentation for more information.

  • file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them.

  • timeout (int, optional) – The number of seconds to wait for the result, if block is True. Raises JobTimeoutError if the timeout passes.

  • block (bool, default True) – If True (default), block until the job is completed, or timeout has passed. If False, immediately returns a Job (which has already had execute called).

  • progress_bar (bool, default None) – Whether to draw the progress bar. If None (default), will display a progress bar in Jupyter Notebooks, but not elsewhere. Ignored if block==False.

  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters

  • cache (bool, default True) – Whether to use the cache for this job.

  • **params (Proxytype) – Parameters under which to run the computation, such as geoctx.

Returns

result – When format="pyarrow" (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from descarteslabs.workflows.result_types. For other formats, returns raw bytes. Consider using file in that case to save the results to a file. If the destination doesn’t support retrieving results (like “email”), returns None

Return type

Python object, bytes, or None

Example

>>> import descarteslabs.workflows as wf
>>> num = wf.Int(1) + 1
>>> wf.compute(num) 
2
>>> # same computation but do not block
>>> job = wf.compute(num, block=False) 
>>> job 
<descarteslabs.workflows.models.job.Job object at 0x...>
>>> job.result() 
2
>>> # pass multiple proxy objects to `wf.compute` to compute all at once
>>> wf.compute((num, num, num)) 
(2, 2, 2)
>>> # specifying a format
>>> img = wf.Image.from_id("sentinel-2:L1C:2019-05-04_13SDV_99_S2B_v1").pick_bands("red")
>>> wf.compute(img, geoctx=ctx, format="pyarrow") # default 
ImageResult:
...
>>> # same computation but with json format
>>> wf.compute(img, geoctx=ctx, format="json") 
b'{"ndarray":[[[0.39380000000000004,0.3982,0.3864,...
>>> # same computation but with geotiff format (and some format options)
>>> bytes_ = wf.compute(img, geoctx=ctx, format={"type": "geotiff", "tiled": False}) 
>>> # you probably want to save the geotiff to a file:
>>> wf.compute(img, geoctx=ctx, file="my_geotiff.tif", format={"type": "geotiff", "tiled": False}) 
>>> # specifying a destination
>>> num = wf.Int(1) + 1
>>> wf.compute(num, destination="download") # default 
2
>>> # same computation but with email destination
>>> wf.compute(num, destination="email") 
>>> # now with some destination options
>>> wf.compute(
...     num,
...     destination={
...         "type": "email",
...         "subject": "My Computation is Done"
...     },
...     format="json",
... ) 
publish(obj, name='', description='', client=None)[source]

Publish a proxy object as a Workflow.

Parameters
  • obj (Proxytype) – A proxy object to compute

  • name (str, default "") – Name for the new Workflow

  • description (str, default "") – Long-form description of this Workflow. Markdown is supported.

  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters

Returns

workflow – The saved Workflow object. workflow.id contains the ID of the new Workflow.

Return type

Workflow

Example

>>> from descarteslabs.workflows import Image, Function
>>> def ndvi(img):
...     nir, red = img.unpack_bands("nir red")
...     return (nir - red) / (nir + red)
>>> func = Function.from_callable(ndvi, Image)
>>> workflow = func.publish("NDVI") 
>>> workflow 
<descarteslabs.workflows.models.workflow.Workflow object at 0x...>
retrieve(workflow_id, client=None)[source]

Load a published Workflow object.

Parameters
  • workflow_id (str) – ID of the Workflow to retrieve

  • client (Compute, optional) – Allows you to use a specific client instance with non-default auth and parameters

Returns

workflow – Object representing the workflow, including both its metadata (like workflow.name, workflow.description) and proxy object (workflow.object).

Return type

Workflow

Example

>>> from descarteslabs.workflows import Image, Function, retrieve
>>> def ndvi(img):
...     nir, red = img.unpack_bands("nir red")
...     return (nir - red) / (nir + red)
>>> func = Function.from_callable(ndvi, Image) # create a function that can be called on an Image
>>> workflow = func.publish("NDVI") 
>>> workflow.id 
'42cea96a864811f00f0bcdb8177ba80d6dc9c7492e13e794'
>>> same_workflow = retrieve('42cea96a864811f00f0bcdb8177ba80d6dc9c7492e13e794') 
>>> same_workflow 
<descarteslabs.workflows.models.workflow.Workflow object at 0x...>
>>> img = Image.from_id("sentinel-2:L1C:2019-05-04_13SDV_99_S2B_v1")
>>> same_workflow.object(img).compute(geoctx) # geoctx is an arbitrary geocontext for 'img' 
>>> # notice the bandname is comprised of the operations called to create it
ImageResult:
  * ndarray: MaskedArray<shape=(1, 512, 512), dtype=float64>
  * properties: 'absolute_orbit', 'acquired', 'archived', 'area', ...
  * bandinfo: 'nir_sub_red_div_nir_add_red'
  * geocontext: 'geometry', 'key', 'resolution', 'tilesize', ...
use(workflow_id, client=None)[source]

Use like import: load the proxy object of a published Workflow.

Shorthand for retrieve(workflow_id).object.

Parameters
  • workflow_id (str) – ID of the Workflow to retrieve

  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters

Returns

obj – Proxy object of the Workflow.

Return type

Proxytype

Example

>>> from descarteslabs.workflows import Image, Function, use
>>> def ndvi(img):
...     nir, red = img.unpack_bands("nir red")
...     return (nir - red) / (nir + red)
>>> func = Function.from_callable(ndvi, Image) # create a function that can be called on an Image
>>> workflow = func.publish("NDVI") 
>>> workflow.id 
'42cea96a864811f00f0bcdb8177ba80d6dc9c7492e13e794'
>>> same_function = use('42cea96a864811f00f0bcdb8177ba80d6dc9c7492e13e794') 
>>> same_function 
<descarteslabs.workflows.types.function.function.Function[Image, {}, Image] object at 0x...>
>>> img = Image.from_id("sentinel-2:L1C:2019-05-04_13SDV_99_S2B_v1")
>>> same_function(img).compute(geoctx) # geoctx is an arbitrary geocontext for 'img' 
ImageResult:
...