Function¶

Classes:

Function(function) Function[arg_type, ..., {kwarg: type, ...}, return_type]: Proxy function.

class Function(function)[source]¶

Function[arg_type, ..., {kwarg: type, ...}, return_type]: Proxy function.

You can create a Function from any Python function, usually using Function.from_callable (or proxify). You can also turn a Workflows object that depends on parameters into a Function using Function.from_object.

Functions have positional-only arguments, named arguments, and a return value of specific types. All the arguments are required. Like Python functions, the named arguments can be given positionally or by name. For example, if func is a Function[{'x': Int, 'y': Str}, Int], func(1, "hello"), func(x=1, y="hello"), and func(y="hello", x=1) are all equivalent.

If you’re creating a Function yourself, you should always use named arguments—positional-only arguments are primarily for internal use. Just use Function.from_callable or Function.from_object and it will take care of everything for you.

isinstance and issubclass have special behavior for Functions, since unlike Python, Functions are strongly typed (you know what type of arguments they take, and what type of value they return, without having to run them). In general, if x and y are both Functions, isinstance(x, type(y)) means that you can safely use x wherever you could use y—the types they accept and return are compatible. Formally, Function is contravariant in its argument types and covariant in its return type. That means that a Function[Number, ...] is considered a subtype of Function[Int, ...], because any function that can handle a Number can also handle an Int. Whereas Function[... Int] (Function that returns an Int) is a subtype of Function[..., Number], since Int is a subtype of Number.

Examples

>>> import descarteslabs.workflows as wf
>>> @wf.Function.from_callable
... def pow(base: wf.Int, exp: wf.Float) -> wf.Float:
...     return base ** exp
>>> pow
<descarteslabs.workflows.types.function.function.Function[{'base': Int, 'exp': Float}, Float] object at 0x...>
>>> pow(16, 0.5).inspect() 
4

>>> word = wf.parameter("word", wf.Str)
>>> repeats = wf.widgets.slider("repeats", min=0, max=5, step=1)
>>> repeated = (word + " ") * repeats
>>> repeat_func = wf.Function.from_object(repeated)
>>> repeat_func
<descarteslabs.workflows.types.function.function.Function[{'word': Str, 'repeats': Int}, Str] object at 0x...>
>>> repeat_func("hello", 3).inspect() 
'hello hello hello '

>>> from descarteslabs.workflows import Int, Float, Bool
>>> func_type = Function[Int, {}, Int] # function with Int arg, no named args, returning an Int
>>> func_type = Function[Int, {'x': Float}, Bool] # function with Int arg, kwarg 'x' of type Float, returning a Bool
>>> func_type = Function[{}, Int] # zero-argument function, returning a Int

>>> func = Function[Int, Int, {}, Int](lambda x, y: x + y) # function taking two Ints and adding them together
>>> func
<descarteslabs.workflows.types.function.function.Function[Int, Int, {}, Int] object at 0x...>
>>> func(3, 4).inspect() 
7

Attributes:

`all_arg_types`	The types of all arguments this `Function` takes, in positional order (`arg_types` + `kwarg_types`)
`arg_types`	The types of the positional-only arguments this `Function` takes
`kwarg_types`	The names and types, in order, of the named arguments this `Function` takes
`return_type`	The Proxytype returned by this `Function`

Methods:

`compute`([format, destination, file, …])	Compute a proxy object and wait for its result.
`from_callable`(func, *arg_types[, return_type])	Construct a Workflows Function from a Python callable.
`from_object`(obj)	Turn a Workflows object that depends on parameters into a `Function`.
`inspect`([format, file, cache, _ruster, …])	Quickly compute a proxy object using a low-latency, lower-reliability backend.
`publish`(version[, title, description, …])	Publish a proxy object as a `Workflow` with the given version.

compute(format='pyarrow', destination='download', file=None, timeout=None, block=True, progress_bar=None, cache=True, _ruster=None, _trace=False, client=None, num_retries=None, **arguments)¶

Compute a proxy object and wait for its result.

If the caller has too many outstanding compute jobs, this will raise a ResourceExhausted exception.

Parameters:	geoctx (`GeoContext`, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a `GeoContext`, but for operations that only involve non-geospatial types, this parameter is optional. format (Str or Dict, default "pyarrow") – The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes or None. destination (str or dict, default "download") – The destination for the result. See the destinations documentation for more information. file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them. timeout (Int, optional) – The number of seconds to wait for the result, if `block` is True. Raises `JobTimeoutError` if the timeout passes. block (Bool, default True) – If True (default), block until the job is completed, or `timeout` has passed. If False, immediately returns a `Job` (which has already had `execute` called). progress_bar (Bool, default None) – Whether to draw the progress bar. If `None` (default), will display a progress bar in Jupyter Notebooks, but not elsewhere. Ignored if `block==False`. client (`workflows.client.Client`, optional) – Allows you to use a specific client instance with non-default auth and parameters num_retries (Int, optional) – The number of retries to make in the event of a request failure. If you are making numerous long-running asynchronous requests, you can use this parameter as a way to indicate that you are comfortable waiting and retrying in response to RESOURCE EXHAUSTED errors. By default, most failures will trigger a small number of retries, but if you have reached your outstanding job limit, by default, the client will not retry. This parameter is unnecessary when making synchronous `compute` requests (ie. block=True, the default). See the compute section of the Workflows Guide for more information. *arguments (Any*) – Values for all parameters that `obj` depends on (or arguments that `obj` takes, if it’s a `Function`). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.
Returns:	result – When `format="pyarrow"` (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from `descarteslabs.workflows.result_types`. For other formats, returns raw bytes. Consider using `file` in that case to save the results to a file. If the destination doesn’t support retrieving results (like “email”), returns None.
Return type:	Python object, bytes, or None
Raises:	RetryError – Raised if there are too many failed retries. Inspect `RetryError.exceptions` to determine the ultimate cause of the error. If you reach your maximum number of outstanding compute jobs, there will be one or more `ResourceExhausted` exceptions.

classmethod from_callable(func, *arg_types, return_type=None)[source]¶

Construct a Workflows Function from a Python callable.

You must specify the types of arguments the function takes, either through type annotations (preferable) or directly in from_callable.

If the function has type annotations, from_callable can be used as a decorator:

@wf.Function.from_callable
def my_function(x: wf.Int, y: wf.Image) -> wf.ImageCollection:
    ...

Otherwise, the argument types must be passed to from_callable.

Parameters:	func (Python callable or Function) – The function to convert. A function is delayed by calling it once, passing in dummy Workflows objects and seeing what operations were applied in the value it returns. If `func` is already a Workflows `Function`, its argument types and return type must be compatible with `arg_types` and `return_type`, if they’re given. Specifically: If `arg_types` are given, `func` must take that number of arguments, and each argument type must be a superclass of the corresponding one in `arg_types`. Otherwise, it can take any arguments. If `return_type` is give, `func` must return a subclass of `return_type`. Otherwise, it can return any type. arg_types (Proxytype, optional) – For each parameter of `func`, the type that it should accept. The number of argument types given much match the number of arguments `func` actually accepts. If not given, `func` must have type annotations for all of its arguments. If `func` has type annotations, but `arg_types` are also given explicitly, then the annotations are ignored. return_type* (Proxytype, optional) – The type the function should return. If not given, and there is no return type annotation, the return type will be inferred from what the function actually returns when called.
Returns:
Return type:	Function

Example

>>> import descarteslabs.workflows as wf
>>> def string_pow(base: wf.Str, exp: wf.Float) -> wf.Float:
...     return wf.Float(base) ** exp
>>> wf_string_pow = wf.Function.from_callable(string_pow)  # types inferred from annotations
>>> print(wf_string_pow)
<descarteslabs.workflows.types.function.function.Function[{'base': Str, 'exp': Float}, Float] object at 0x...>
>>> wf_string_pow("2", 2.0).inspect()  
4.0

>>> # or, passing Str and Float as the argument types explicitly:
>>> def string_pow(base, exp):
...     return wf.Float(base) ** exp
>>> wf_pow = Function.from_callable(string_pow, wf.Str, wf.Float, return_type=wf.Float)
>>> wf_pow
<descarteslabs.workflows.types.function.function.Function[{'base': Str, 'exp': Float}, Float] object at 0x...>
>>> wf_pow("3", 2.0).inspect()  
9.0

classmethod from_object(obj)[source]¶

Turn a Workflows object that depends on parameters into a Function.

Any parameters obj depends on become arguments to the Function. Calling that function essentially returns obj, with the given values applied to those parameters.

Example

>>> import descarteslabs.workflows as wf
>>> word = wf.parameter("word", wf.Str)
>>> repeats = wf.widgets.slider("repeats", min=0, max=5, step=1)
>>> repeated = (word + " ") * repeats

>>> # `repeated` depends on parameters; we have to pass values for them to compute it
>>> repeated.inspect(word="foo", repeats=3) 
'foo foo foo '

>>> # turn `repeated` into a Function that takes those parameters
>>> repeat = wf.Function.from_object(repeated)
>>> repeat
<descarteslabs.workflows.types.function.function.Function[{'word': Str, 'repeats': Int}, Str] object at 0x...>
>>> repeat("foo", 3).inspect() 
'foo foo foo '
>>> repeat("hello", 2).inspect() 
'hello hello '

Parameters:	obj (Proxytype) – A Workflows proxy object.
Returns:	func – A `Function` equivalent to `obj` TODO
Return type:	Function

inspect(format='pyarrow', file=None, cache=True, _ruster=None, timeout=60, client=None, **arguments)¶

Quickly compute a proxy object using a low-latency, lower-reliability backend.

Inspect is meant for getting simple computations out of Workflows, primarily for interactive use. It’s quicker but less resilient, won’t be retried if it fails, and has no progress updates.

If you have a larger computation (longer than ~30sec), or you want to be sure the computation will succeed, use compute instead. compute creates a Job, which runs asynchronously, will be retried if it fails, and stores its results for later retrieval.

Parameters:	geoctx (`common.geo.geocontext.GeoContext`, `GeoContext`, or None) – The GeoContext parameter under which to run the computation. Almost all computations will require a `GeoContext`, but for operations that only involve non-geospatial types, this parameter is optional. format (str or dict, default "pyarrow") – The serialization format for the result. See the formats documentation for more information. If “pyarrow” (the default), returns an appropriate Python object, otherwise returns raw bytes. file (path or file-like object, optional) – If specified, writes results to the path or file instead of returning them. cache (bool, default True) – Whether to use the cache for this job. timeout (int, optional, default 60) – The number of seconds to wait for the result. Raises `JobTimeoutError` if the timeout passes. client (`workflows.inspect.InspectClient`, optional) – Allows you to use a specific InspectClient instance with non-default auth and parameters *arguments (Any*) – Values for all parameters that `obj` depends on (or arguments that `obj` takes, if it’s a `Function`). Can be given as Proxytypes, or as Python objects like numbers, lists, and dicts that can be promoted to them. These arguments cannot depend on any parameters.
Returns:	result – When `format="pyarrow"` (the default), returns an appropriate Python object representing the result, either as a plain Python type, or object from `descarteslabs.workflows.result_types`. For other formats, returns raw bytes. Consider using `file` in that case to save the results to a file.
Return type:	Python object or bytes

publish(version, title='', description='', labels=None, tags=None, docstring='', version_labels=None, viz_options=None, client=None)¶

Publish a proxy object as a Workflow with the given version.

If the proxy object depends on any parameters (obj.params is not empty), it’s first internally converted to a Function that takes those parameters (using Function.from_object).

Parameters:	id (Str) – ID for the new Workflow object. This should be of the form `email:workflow_name` and should be globally unique. If this ID is not of the proper format, you will not be able to save the Workflow. version (Str) – The version to be set, tied to the given `obj`. This should adhere to the semantic versioning schema. title (Str, default "") – User-friendly title for the `Workflow`. description (str, default "") – Long-form description of this `Workflow`. Markdown is supported. labels (Dict, optional) – Key-value pair labels to add to the `Workflow`. tags (list, optional) – A list of strings to add as tags to the `Workflow`. docstring (Str, default "") – The docstring for this version. version_labels (Dict, optional) – Key-value pair labels to add to the version. client (`workflows.client.Client`, optional) – Allows you to use a specific client instance with non-default auth and parameters
Returns:	workflow – The saved `Workflow` object. `workflow.id` contains the ID of the new Workflow.
Return type:	Workflow or Function

property all_arg_types¶

The types of all arguments this Function takes, in positional order (arg_types + kwarg_types)

Type:	tuple

property arg_types¶

The types of the positional-only arguments this Function takes

Type:	tuple

property kwarg_types¶

The names and types, in order, of the named arguments this Function takes

Type:	dict

property return_type¶

The Proxytype returned by this Function

Type:	type