Function

Classes:

Function(function)

Function[arg_type, ..., {kwarg: type, ...}, return_type]: Proxy function.

class Function(function)[source]

Function[arg_type, ..., {kwarg: type, ...}, return_type]: Proxy function.

You can create a Function from any Python function, usually using Function.from_callable (or proxify). You can also turn a Workflows object that depends on parameters into a Function using Function.from_object.

Functions have positional-only arguments, named arguments, and a return value of specific types. All the arguments are required. Like Python functions, the named arguments can be given positionally or by name. For example, if func is a Function[{'x': Int, 'y': Str}, Int], func(1, "hello"), func(x=1, y="hello"), and func(y="hello", x=1) are all equivalent.

If you’re creating a Function yourself, you should always use named arguments—positional-only arguments are primarily for internal use. Just use Function.from_callable or Function.from_object and it will take care of everything for you.

isinstance and issubclass have special behavior for Functions, since unlike Python, Functions are strongly typed (you know what type of arguments they take, and what type of value they return, without having to run them). In general, if x and y are both Functions, isinstance(x, type(y)) means that you can safely use x wherever you could use y—the types they accept and return are compatible. Formally, Function is contravariant in its argument types and covariant in its return type. That means that a Function[Number, ...] is considered a subtype of Function[Int, ...], because any function that can handle a Number can also handle an Int. Whereas Function[... Int] (Function that returns an Int) is a subtype of Function[..., Number], since Int is a subtype of Number.

Examples

>>> import descarteslabs.workflows as wf
>>> @wf.Function.from_callable
... def pow(base: wf.Int, exp: wf.Float) -> wf.Float:
...     return base ** exp
>>> pow
<descarteslabs.workflows.types.function.function.Function[{'base': Int, 'exp': Float}, Float] object at 0x...>
>>> pow(16, 0.5).inspect() 
4
>>> word = wf.parameter("word", wf.Str)
>>> repeats = wf.widgets.slider("repeats", min=0, max=5, step=1)
>>> repeated = (word + " ") * repeats
>>> repeat_func = wf.Function.from_object(repeated)
>>> repeat_func
<descarteslabs.workflows.types.function.function.Function[{'word': Str, 'repeats': Int}, Str] object at 0x...>
>>> repeat_func("hello", 3).inspect() 
'hello hello hello '
>>> from descarteslabs.workflows import Int, Float, Bool
>>> func_type = Function[Int, {}, Int] # function with Int arg, no named args, returning an Int
>>> func_type = Function[Int, {'x': Float}, Bool] # function with Int arg, kwarg 'x' of type Float, returning a Bool
>>> func_type = Function[{}, Int] # zero-argument function, returning a Int
>>> func = Function[Int, Int, {}, Int](lambda x, y: x + y) # function taking two Ints and adding them together
>>> func
<descarteslabs.workflows.types.function.function.Function[Int, Int, {}, Int] object at 0x...>
>>> func(3, 4).inspect() 
7

Attributes:

all_arg_types

The types of all arguments this Function takes, in positional order (arg_types + kwarg_types)

arg_types

The types of the positional-only arguments this Function takes

kwarg_types

The names and types, in order, of the named arguments this Function takes

return_type

The Proxytype returned by this Function

Methods:

compute([format, destination, file, …])

Compute a proxy object and wait for its result.

from_callable(func, *arg_types[, return_type])

Construct a Workflows Function from a Python callable.

from_object(obj)

Turn a Workflows object that depends on parameters into a Function.

inspect([format, file, cache, _ruster, …])

Quickly compute a proxy object using a low-latency, lower-reliability backend.

publish(version[, title, description, …])

Publish a proxy object as a Workflow with the given version.

compute(format='pyarrow', destination='download', file=None, timeout=None, block=True, progress_bar=None, cache=True, _ruster=None, _trace=False, client=None, num_retries=None, **arguments)

Compute a proxy object and wait for its result.

If the caller has too many outstanding compute jobs, this will raise a ResourceExhausted exception.

Parameters
  • geoctx (GeoContext, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a GeoContext, but for operations that only involve non-geospatial types, this parameter is optional.

  • format (Str or Dict, default "pyarrow") – The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes or None.

  • destination (str or dict, default "download") – The destination for the result. See the destinations documentation for more information.

  • file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them.

  • timeout (Int, optional) – The number of seconds to wait for the result, if block is True. Raises JobTimeoutError if the timeout passes.

  • block (Bool, default True) – If True (default), block until the job is completed, or timeout has passed. If False, immediately returns a Job (which has already had execute called).

  • progress_bar (Bool, default None) – Whether to draw the progress bar. If None (default), will display a progress bar in Jupyter Notebooks, but not elsewhere. Ignored if block==False.

  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters

  • num_retries (Int, optional) – The number of retries to make in the event of a request failure. If you are making numerous long-running asynchronous requests, you can use this parameter as a way to indicate that you are comfortable waiting and retrying in response to RESOURCE EXHAUSTED errors. By default, most failures will trigger a small number of retries, but if you have reached your outstanding job limit, by default, the client will not retry. This parameter is unnecessary when making synchronous compute requests (ie. block=True, the default). See the compute section of the Workflows Guide for more information.

  • **arguments (Any) – Values for all parameters that obj depends on (or arguments that obj takes, if it’s a Function). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.

Returns

result – When format="pyarrow" (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from descarteslabs.workflows.result_types. For other formats, returns raw bytes. Consider using file in that case to save the results to a file. If the destination doesn’t support retrieving results (like “email”), returns None.

Return type

Python object, bytes, or None

Raises

RetryError – Raised if there are too many failed retries. Inspect RetryError.exceptions to determine the ultimate cause of the error. If you reach your maximum number of outstanding compute jobs, there will be one or more ResourceExhausted exceptions.

classmethod from_callable(func, *arg_types, return_type=None)[source]

Construct a Workflows Function from a Python callable.

You must specify the types of arguments the function takes, either through type annotations (preferable) or directly in from_callable.

If the function has type annotations, from_callable can be used as a decorator:

@wf.Function.from_callable
def my_function(x: wf.Int, y: wf.Image) -> wf.ImageCollection:
    ...

Otherwise, the argument types must be passed to from_callable.

Parameters
  • func (Python callable or Function) –

    The function to convert.

    A function is delayed by calling it once, passing in dummy Workflows objects and seeing what operations were applied in the value it returns.

    If func is already a Workflows Function, its argument types and return type must be compatible with arg_types and return_type, if they’re given. Specifically:

    • If arg_types are given, func must take that number of arguments, and each argument type must be a superclass of the corresponding one in arg_types. Otherwise, it can take any arguments.

    • If return_type is give, func must return a subclass of return_type. Otherwise, it can return any type.

  • *arg_types (Proxytype, optional) – For each parameter of func, the type that it should accept. The number of argument types given much match the number of arguments func actually accepts. If not given, func must have type annotations for all of its arguments. If func has type annotations, but arg_types are also given explicitly, then the annotations are ignored.

  • return_type (Proxytype, optional) – The type the function should return. If not given, and there is no return type annotation, the return type will be inferred from what the function actually returns when called.

Returns

Return type

Function

Example

>>> import descarteslabs.workflows as wf
>>> def string_pow(base: wf.Str, exp: wf.Float) -> wf.Float:
...     return wf.Float(base) ** exp
>>> wf_string_pow = wf.Function.from_callable(string_pow)  # types inferred from annotations
>>> print(wf_string_pow)
<descarteslabs.workflows.types.function.function.Function[{'base': Str, 'exp': Float}, Float] object at 0x...>
>>> wf_string_pow("2", 2.0).inspect()  
4.0
>>> # or, passing Str and Float as the argument types explicitly:
>>> def string_pow(base, exp):
...     return wf.Float(base) ** exp
>>> wf_pow = Function.from_callable(string_pow, wf.Str, wf.Float, return_type=wf.Float)
>>> wf_pow
<descarteslabs.workflows.types.function.function.Function[{'base': Str, 'exp': Float}, Float] object at 0x...>
>>> wf_pow("3", 2.0).inspect()  
9.0
classmethod from_object(obj)[source]

Turn a Workflows object that depends on parameters into a Function.

Any parameters obj depends on become arguments to the Function. Calling that function essentially returns obj, with the given values applied to those parameters.

Example

>>> import descarteslabs.workflows as wf
>>> word = wf.parameter("word", wf.Str)
>>> repeats = wf.widgets.slider("repeats", min=0, max=5, step=1)
>>> repeated = (word + " ") * repeats
>>> # `repeated` depends on parameters; we have to pass values for them to compute it
>>> repeated.inspect(word="foo", repeats=3) 
'foo foo foo '
>>> # turn `repeated` into a Function that takes those parameters
>>> repeat = wf.Function.from_object(repeated)
>>> repeat
<descarteslabs.workflows.types.function.function.Function[{'word': Str, 'repeats': Int}, Str] object at 0x...>
>>> repeat("foo", 3).inspect() 
'foo foo foo '
>>> repeat("hello", 2).inspect() 
'hello hello '
Parameters

obj (Proxytype) – A Workflows proxy object.

Returns

func – A Function equivalent to obj TODO

Return type

Function

inspect(format='pyarrow', file=None, cache=True, _ruster=None, timeout=60, client=None, **arguments)

Quickly compute a proxy object using a low-latency, lower-reliability backend.

Inspect is meant for getting simple computations out of Workflows, primarily for interactive use. It’s quicker but less resilient, won’t be retried if it fails, and has no progress updates.

If you have a larger computation (longer than ~30sec), or you want to be sure the computation will succeed, use compute instead. compute creates a Job, which runs asynchronously, will be retried if it fails, and stores its results for later retrieval.

Parameters
  • geoctx (scenes.geocontext.GeoContext, GeoContext, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a GeoContext, but for operations that only involve non-geospatial types, this parameter is optional.

  • format (str or dict, default "pyarrow") –

    The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes.

  • file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them.

  • cache (bool, default True) – Whether to use the cache for this job.

  • timeout (int, optional, default 60) – The number of seconds to wait for the result. Raises JobTimeoutError if the timeout passes.

  • client (workflows.inspect.InspectClient, optional) – Allows you to use a specific InspectClient instance with non-default auth and parameters

  • **arguments (Any) – Values for all parameters that obj depends on (or arguments that obj takes, if it’s a Function). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.

Returns

result – When format="pyarrow" (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from descarteslabs.workflows.result_types. For other formats, returns raw bytes. Consider using file in that case to save the results to a file.

Return type

Python object or bytes

publish(version, title='', description='', labels=None, tags=None, docstring='', version_labels=None, viz_options=None, client=None)

Publish a proxy object as a Workflow with the given version.

If the proxy object depends on any parameters (obj.params is not empty), it’s first internally converted to a Function that takes those parameters (using Function.from_object).

Parameters
  • id (Str) – ID for the new Workflow object. This should be of the form email:workflow_name and should be globally unique. If this ID is not of the proper format, you will not be able to save the Workflow.

  • version (Str) – The version to be set, tied to the given obj. This should adhere to the semantic versioning schema.

  • title (Str, default "") – User-friendly title for the Workflow.

  • description (str, default "") – Long-form description of this Workflow. Markdown is supported.

  • labels (Dict, optional) – Key-value pair labels to add to the Workflow.

  • tags (list, optional) – A list of strings to add as tags to the Workflow.

  • docstring (Str, default "") – The docstring for this version.

  • version_labels (Dict, optional) – Key-value pair labels to add to the version.

  • client (workflows.client.Client, optional) – Allows you to use a specific client instance with non-default auth and parameters

Returns

workflow – The saved Workflow object. workflow.id contains the ID of the new Workflow.

Return type

Workflow or Function

property all_arg_types

The types of all arguments this Function takes, in positional order (arg_types + kwarg_types)

Type

tuple

property arg_types

The types of the positional-only arguments this Function takes

Type

tuple

property kwarg_types

The names and types, in order, of the named arguments this Function takes

Type

dict

property return_type

The Proxytype returned by this Function

Type

type