Vector

Note: It is recommended to use the object oriented interface at descarteslabs.vectors.

class Vector(url=None, auth=None)[source]

Client for storing and querying vector data.

The Descartes Labs Vector service allows you store vector features (points, polygons, etc.) with associated key-value properties, and query that data by geometry or by properties.

It works best at the scale of millions of features. For small amounts of vector data that easily fit in memory, working directly with a GeoJSON file or similar may be more efficient.

Concepts:

  • “Feature”: a single geometric entity and its associated metadata (equivalent to a GeoJSON Feature).
  • “Product”: a collection of related Features, with a common id, description, and access controls.

This client currently returns data as dictionaries in JSON API format.

The parent Service class implements authentication and exponential backoff/retry. Override the url parameter to use a different instance of the backing service.

count_features(product_id)[source]

Get the count of the features in a product.

Parameters:product_id (str) – (Required) The ID of the product to count all features for.
Return type:int
Returns:The total number of features in a product.
create_feature(product_id, geometry, properties=None, fix_geometry='accept')[source]

Add a feature to an existing vector product.

Parameters:
  • product_id (str) – (Required) The ID of the Vector product to which this feature will belong.
  • geometry (dict) –

    (Required) Shape associated with this vector feature. This accepts the following types of GeoJSON geometries:

    • Points
    • MultiPoints
    • Polygons
    • MultiPolygons
    • LineStrings
    • MultiLineStrings
    • GeometryCollections
  • properties (dict) – Dictionary of arbitrary properties.
  • fix_geometry (str) – String specifying how to handle certain problem geometries, including those which do not follow counter-clockwise winding order (which is required by the GeoJSON spec but not many popular tools). Allowed values are reject (reject invalid geometries with an error), fix (correct invalid geometries if possible and use this corrected value when creating the feature), and accept (the default) which will correct the geometry for internal use but retain the original geometry in the results.
Return type:

DotDict
Returns:

Created Feature, as a JSON API resource collection. The keys are:

data: A 'DotDict' instance with the following keys:

    id:        The ID of the feature.
    type:      'feature'.
    attributes: A DotDict instance with the following keys:

        created:    Time that the task was created in ISO-8601 UTC.
        properties: A DotDict instance.  The keys are user provided.
                    Supported values are strings, numbers, the
                    value 'None'.
        geometry:   A DotDict instance with the following keys:

            type:       The type of the feature, one of 'Polygon',
                         'MultiPolygon', 'Point', 'MultiPoint',
                         'LineString', 'MultiLineString', or
                         'GeometryCollection'.
            coordinates: A list of coordinates; the exact structure
                         of potentially nesting lists depends on
                         the given type.

create_features(product_id, features, fix_geometry='accept')[source]

Add multiple features to an existing vector product.

Parameters:
  • product_id (str) – (Required) The ID of the Vector product to which these features will belong.
  • features (list(dict)) – (Required) Each feature must be a dict with a geometry and properties field. If you provide more than 100 features, they will be batched in groups of 100, but consider using upload_features() instead.
  • fix_geometry (str) – String specifying how to handle certain problem geometries, including those which do not follow counter-clockwise winding order (which is required by the GeoJSON spec but not many popular tools). Allowed values are reject (reject invalid geometries with an error), fix (correct invalid geometries if possible and use this corrected value when creating the feature), and accept (the default) which will correct the geometry for internal use but retain the original geometry in the results.
Return type:

DotDict
Returns:

The features as a JSON API resource object. The keys are:

data: A list of 'DotDict' instances.  Each instance contains
      keys as described in create_feature().

Raises:

ClientError – A variety of http-related exceptions can thrown. If more than 100 features were passed in, some of these may have been successfully inserted, others not. If this is a problem, then stick with <= 100 features.
create_product(**kwargs)[source]

Add a vector product to your catalog.

Parameters:
  • product_id (str) – (Required) A unique name for this Vector product. In the created product a namespace consisting of your user id (e.g. ae60fc891312ggadc94ade8062213b0063335a3c:) or your organization id (e.g., yourcompany:) will be prefixed to this, if it doesn’t already have one, in order to make the id globally unique.
  • title (str) – (Required) Official product title.
  • description (str) – (Required) Information about the product, why it exists, and what it provides.
  • owners (list(str)) – User, group, or organization IDs that own this product. Each ID must be prefixed with user:, group:, or org:. Defaults to [current user, current user’s org].
  • readers (list(str)) – User, group, or organization IDs that can read this product. Each ID must be prefixed with user:, group:, or org:.
  • writers (list(str)) – User, group, or organization IDs that can edit this product. Each ID must be prefixed with user:, group:, or org:.
Return type:

DotDict
Returns:

Created vector product, as a JSON API resource object. For a list of keys, please see get_product().
create_product_from_query(**kwargs)[source]

Query vector features within an existing product and create a new vector product to your catalog from the query result.

At least one of geometry, query_expr, or query_limit is required.

Parameters:
  • new_product_id (str) – (Required) A unique name for this product. In the created product a namespace consisting of your user id (e.g. ae60fc891312ggadc94ade8062213b0063335a3c:) or your organization id (e.g., yourcompany:) will be prefixed to this, if it doesn’t already have one, in order to make the id globally unique.
  • title (str) – (Required) Official product title.
  • description (str) – (Required) Information about the product, why it exists, and what it provides.
  • product_id (str) – (Required) The ID of the product within which to search.
  • owners (list(str)) – User, group, or organization IDs that own this product. Each ID must be prefixed with user:, group:, or org:. Defaults to [current user, current user’s org].
  • readers (list(str)) – User, group, or organization IDs that can read this product. Each ID must be prefixed with user:, group:, or org:.
  • writers (list(str)) – User, group, or organization IDs that can edit this product. Each ID must be prefixed with user:, group:, or org:.
  • geometry (dict) –

    Search for Features intersecting this shape. This accepts the following types of GeoJSON geometries:

    • Points
    • MultiPoints
    • Polygons
    • MultiPolygons
    • LineStrings
    • MultiLineStrings
    • GeometryCollections
  • query_expr (descarteslabs.client.common.filtering.Expression) – A rich query expression generator that represents an arbitrary tree of boolean combinations of property comparisons. Using the properties filter factory inside descarteslabs.client.services.vector as p, you can E.g query_expr=(p.temperature >= 50) & (p.hour_of_day > 18), or even more complicated expressions like query_expr=(100 > p.temperature >= 50) | ((p.month != 10) & (p.day_of_month > 14)) This expression gets serialized and applied to the properties mapping supplied with the features in the vector product. If you supply a property which doesn’t exist as part of the expression that comparison will evaluate to False.
  • query_limit (int) – Maximum number of features to return for this query, defaults to all.
Return type:

DotDict
Returns:

Created vector product, as a JSON API resource object. For a description of the keys, please see get_product().
delete_features_from_query(product_id, geometry=None, query_expr=None, **kwargs)[source]

Query an existing Vector product and delete features that match the query results.

One of geometry or query_expr is required.

Parameters:
  • product_id (str) – (Required) The ID of the product within which to search for features to delete.
  • geometry (dict) –

    Search for Features intersecting this shape. This accepts the following types of GeoJSON geometries:

    • Points
    • MultiPoints
    • Polygons
    • MultiPolygons
    • LineStrings
    • MultiLineStrings
    • GeometryCollections
  • query_expr (descarteslabs.client.common.filtering.Expression) – A rich query expression generator that represents an arbitrary tree of boolean combinations of property comparisons. Using the properties filter factory inside descarteslabs.client.services.vector as p, you can E.g query_expr=(p.temperature >= 50) & (p.hour_of_day > 18), or even more complicated expressions like query_expr=(100 > p.temperature >= 50) | ((p.month != 10) & (p.day_of_month > 14)) This expression gets serialized and applied to the properties mapping supplied with the features in the vector product. If you supply a property which doesn’t exist as part of the expression that comparison will evaluate to False.
Return type:

DotDict
Returns:

Vector product from which the features were deleted, as a JSON API resource object. For a list of keys, please see get_product().
delete_product(product_id)[source]

Remove a vector product from the catalog.

Parameters:product_id (str) – (Required) The ID of the Vector product to remove.
Raises:NotFoundError – If the product cannot be found.
export_product_from_query(product_id, key, geometry=None, query_expr=None, query_limit=None)[source]

Query vector features within an existing product and export the result to DescartesLabs Storage as type data using the given storage key.

If none of the geometry, query_expr, or query_limit are given, the full product is exported.

Note that the export is happening asynchronously and can take a while depending on the size of the product or query. You can request the status using get_export_result() or get_export_results().

Once the export is complete, you can download the file from Descartes Labs Storage using descarteslabs.client.services.storage.Storage.get_file() with the given key and a local filename.

Parameters:
  • product_id (str) – (Required) The ID of the product within which to search or which to copy.
  • key (str) – (Required) The name under which the export will be available in the Storage service. The storage_type will be data. Note that this will overwrite any existing data if the key already exists.
  • geometry (dict) –

    Search for Features intersecting this shape. This accepts the following types of GeoJSON geometries:

    • Points
    • MultiPoints
    • Polygons
    • MultiPolygons
    • LineStrings
    • MultiLineStrings
    • GeometryCollections
  • query_expr (descarteslabs.client.common.filtering.Expression) – A rich query expression generator that represents an arbitrary tree of boolean combinations of property comparisons. Using the properties filter factory inside descarteslabs.client.services.vector as p, you can E.g query_expr=(p.temperature >= 50) & (p.hour_of_day > 18), or even more complicated expressions like query_expr=(100 > p.temperature >= 50) | ((p.month != 10) & (p.day_of_month > 14)) This expression gets serialized and applied to the properties mapping supplied with the features in the vector product. If you supply a property which doesn’t exist as part of the expression that comparison will evaluate to False.
  • query_limit (int) – Maximum number of features to return for this query, defaults to all.
Return type:

str
Returns:

The export id.
get_delete_features_status(product_id)[source]

Get the status of the job deleting features from a query.

Parameters:product_id (str) – (Required) The ID of the product to get delete status for. the previously initiated deletion using create_product() or create_product_from_query().
Rtype DotDict:
Returns:A dictionary with information about the status. For a description of the keys, please see get_product_from_query_status(). Note that the type will be u'delete_job.
get_export_result(product_id, export_id)[source]

Get details about a specific export job. Included information about processing error streams, which can help debug failed uploads. Note that this information is available once the job completed (either successfully or unsuccessfully).

Parameters:
  • product_id (str) – (Required) The ID of the product for which to return the result.
  • export_id (str) – (Required) The export ID for which to return the result, as previously returned by Vector.export_product_from_query().
Return type:

DotDict
Returns:

The information about the export task once the task has completed. The keys are:

data: A single 'DotDict' instance with the following keys:

    id:         The id of the task.
    type:       export.
    attributes: A DotDict instance with the following keys:

        created:           Time that the task was created in
                           ISO-8601 UTC.
        exception_name:    The type of exception, if there is one,
                           'None' otherwise.
        failure_type:      'executable_failure' if resource limits are
                           reached, or 'exception' if an exception was
                           thrown, 'None' otherwise.
        peak_memory_usage: The amount of memory used by the task in bytes.
        runtime:           The number of CPU seconds used by the task.
        status:            'SUCCESS' or 'FAILURE'.
        labels:            A list of string labels.  The last value is
                           the key.

Raises:

NotFoundError – When the task is either not found or is pending or running but has not completed yet.
get_export_results(product_id)[source]

Get a list of the exports submitted to a vector product, and status information about each. Note that only completed tasks are shown; any tasks that are pending or running will not be listed.

Parameters:product_id (str) – (Required) The ID of the product to get results for.
Return type:Iterator of DotDict instances
Returns:An iterator returning DotDict instances containing information about each completed export task created by Vector.export_product_from_query(). See get_export_result() under data for an explanation of the information returned.
get_product(product_id)[source]

Get a product’s properties.

Parameters:product_id (str) – (Required) The ID of the Vector product to fetch.
Return type:DotDict
Returns:The vector product, as a JSON API resource object. The keys are:
data: A single 'DotDict' instance with the following keys:

    id:   The ID of the Vectort product.
    type: 'product'.
    meta: A single DotDict instance with the following keys:

        created: Time that the task was created in ISO-8601 UTC.

    attributes: A single DotDict instance with the following keys:

        title:       The title given to this product.
        description: The description given to this product.
        owners:      The owners of this product (at a minimum
                     the organization and the user who created
                     this product).
        readers:     The users, groups, or organizations that
                     can read this product.
        writers:     The users, groups, or organizations that
                     can write into this product.
get_product_from_query_status(product_id)[source]

Get the status of the job creating a new product from a query.

Parameters:product_id (str) – (Required) The ID of the product for which to to check the status. This ID must have been created by a call to create_product_from_query().
Return type:DotDict
Returns:A dictionary with information about the status. The keys are
data: A 'DotDict' instance with the following keys:

    id:         The internal ID for this job.
    type:       'copy_job'.
    attributes: A DotDict instance with the following keys:

        created: Time that the task was created in ISO-8601 UTC.
        ended:   Time that the task completed in ISO-8601 UTC
                 (when available).
        started: Time that the start stared in ISO-8601 UTC
                 (when available).
        state:   'PENDING', 'RUNNING', or 'DONE'.
get_upload_result(product_id, upload_id, pending=False)[source]

Get details about a specific upload job. Included information about processing error streams, which can help debug failed uploads.

Parameters:
  • product_id (str) – (Required) The ID of the product which to retrieve the upload result.
  • upload_id (str) – (Required) An id pertaining to this requested upload, either returned by Vector.get_upload_results() or upload_features().
  • pending (bool) – If True, include pending upload jobs in the result. Defaults to False.
Rtype DotDict:
Returns:

The result as a JSON API resource object. The keys are:

data: A DotDict instance with the following keys:

    id:         The ID of the upload task (which is not the upload
                ID, but you can use it instead of the upload ID).
    type:       'upload'.
    attributes: A DotDict instance with the following keys:

        created:           Time that the task was created in
                           ISO-8601 UTC.
        exception_name:    The type of exception, if there is one,
                           'None' otherwise.
        failure_type:      'executable_failure' if resource limits
                           are reached, or 'exception' if an exception
                           was thrown, 'None' otherwise.
        peak_memory_usage: The amount of memory used by the task
                           in bytes.
        runtime:           The number of CPU seconds used by the task.
        status:            'RUNNING', 'SUCCESS' or 'FAILURE'.
        labels:            A list of string labels.  The last value is
                           the upload ID.
        load:              A DotDict instance describing the actual
                           result of the load which continuous
                           asynchronously after the task itself
                           has completed (as indicated by the
                           'RUNNING`` status):

            errors:      How many errors the load caused.
            output_rows: The number of actual rows created during the
                         load (which may differ from the number of rows
                         given in the upload file).
            state:       'PENDING', 'RUNNING', or 'DONE'.

        result: A DotDict instance describing the result of the Upload
                Task (which pre-processes the data before it's loaded):

            errors:         A list of errors, potentially empty.
            input_features: The number of valid rows in the upload file.
            input_rows:     The number of rows that will be loaded
                            (unless there are errors, this should be
                            identical to ``output_rows`` above).
            job_id:         The internal job ID.

get_upload_results(product_id, pending=False)[source]

Get a list of the uploads submitted to a vector product, and status information about each.

Parameters:
  • product_id (str) – (Required) The ID of the Vector product for which to retrieve the upload results
  • pending (bool) – If True, include pending upload jobs in the result. Defaults to False.
Return type:

Iterator of DotDict instances
Returns:

An iterator over all upload results created with upload_features(), returning DotDict instances with the following keys:

id:         The ID of the upload task (which is not the upload
            ID, but you can use it instead of the upload ID).
type:       'upload'.
attributes: A 'DotDict' instance with the following keys
            (note that this contains less information then the
            information returned by get_upload_result()):

    created:           Time that the task was created in ISO-8601 UTC.
    exception_name:    The type of exception, if there is one,
                       'None' otherwise.
    failure_type:      'executable_failure' if resource limits are
                       reached, or 'exception' if an exception was
                       thrown, 'None' otherwise.
    peak_memory_usage: The amount of memory used by the task in bytes.
    runtime:           The number of CPU seconds used by the task.
    status:            'RUNNING', 'SUCCESS' or 'FAILURE'.
    labels:            A list of string labels.  The last value is
                       the upload ID.

list_products(page_size=100, page=1)[source]

Get all vector products that you have access using JSON API pagination. The first page (1) will always succeed but may be empty. Subsequent pages may throw NotFoundError.

Parameters:
  • page_size (int) – Maximum number of vector products to return per page; default is 100.
  • page (int) – Which page of results to fetch, if there are more results than page_size.
Return type:

DotDict
Returns:

Available vector products and their properties, as a JSON API collection. This dictionary contains the following keys:

data:  A list of 'DotDict' instances.  For the keys
       please see the data key in get_product().
links: (Optional) A single DotDict instance with the
       following keys if there is more than one page of products:

    next: (Optional) A link to the next page of products
          if available.
    prev: (Optional) A link to the previous page of products
          if available.

replace_product(**kwargs)[source]

Replace a vector product in your catalog.

Parameters:
  • product_id (str) – (Required) The ID of the Vector product to update.
  • title (str) – (Required) Official product title.
  • description (str) – (Required) Information about the product, why it exists, and what it provides.
  • owners (list(str)) – User, group, or organization IDs that own this product. Each ID must be prefixed with user:, group:, or org:. Defaults to [current user, current user’s org].
  • readers (list(str)) – User, group, or organization IDs that can read this product. Each ID must be prefixed with user:, group:, or org:.
  • writers (list(str)) – User, group, or organization IDs that can edit this product. Each ID must be prefixed with user:, group:, or org:.
  • name (str) – (Deprecated) Will be removed completely in future versions.
Return type:

DotDict
Returns:

Replaced vector product, as a JSON API resource object. For a description of the keys, please see get_product().
search_features(product_id, geometry=None, query_expr=None, query_limit=None, **kwargs)[source]

Iterate over vector features within an existing product.

At least one of geometry, query_expr, or query_limit is required.

The returned iterator has a length that indicates the size of the query.

Parameters:
  • product_id (str) – (Required) The ID of the product within which to search.
  • geometry (dict) –

    Search for Features intersecting this shape. This accepts the following types of GeoJSON geometries:

    • Points
    • MultiPoints
    • Polygons
    • MultiPolygons
    • LineStrings
    • MultiLineStrings
    • GeometryCollections
  • query_expr (descarteslabs.client.common.filtering.Expression) – A rich query expression generator that represents an arbitrary tree of boolean combinations of property comparisons. Using the properties filter factory inside descarteslabs.client.services.vector as p, you can E.g query_expr=(p.temperature >= 50) & (p.hour_of_day > 18), or even more complicated expressions like query_expr=(100 > p.temperature >= 50) | ((p.month != 10) & (p.day_of_month > 14)) This expression gets serialized and applied to the properties mapping supplied with the features in the vector product. If you supply a property which doesn’t exist as part of the expression that comparison will evaluate to False.
  • query_limit (int) – Maximum number of features to return for this query, defaults to all.
Return type:

Iterator of DotDict instances
Returns:

Features satisfying the query, as JSONAPI primary data objects. For a description of the keys, please refer to create_feature() under the data key.

len() can be used on the returned iterator to determine the query size.
update_product(**kwargs)[source]

Update a vector product in your catalog using the given parameters. If a parameter is not provided, it will not change. You cannot change the product_id.

Parameters:
  • product_id (str) – (Required) The ID of the Vector product to update.
  • name (str) – (Deprecated) Will be removed completely in future versions.
  • title (str) – Official product title.
  • description (str) – Information about the product, why it exists, and what it provides.
  • owners (list(str)) – User, group, or organization IDs that own this product. Each ID must be prefixed with user:, group:, or org:.
  • readers (list(str)) – User, group, or organization IDs that can read this product. Each ID must be prefixed with user:, group:, or org:.
  • writers (list(str)) – User, group, or organization IDs that can edit this product. Each ID must be prefixed with user:, group:, or org:.
Return type:

DotDict
Returns:

Updated vector product, as a JSON API resource object. For a description of the keys please see get_product().
upload_features(file_ish, product_id, max_errors=0, fix_geometry='accept')[source]

Asynchonously upload a file or stream of Newline Delimited JSON features. Note that the file may not contain empty lines or lines containing anything but the feature (including e.g. a trailing comma), or such lines will cause errors.

It is recommended that the IOBase object is a byte-oriented (not text-oriented) object, although Python 3 allows io.StringIO to be used.

This is an asynchronous operation and you can query for the status using Vector.get_upload_result() with the upload_id returned by this method.

Parameters:
  • file_ish (str or io.IOBase) – an open io.IOBase object, or a path to the file to upload.
  • product_id (str) – (Required) The ID of the Vector product to which these features will belong.
  • max_errors (int) – The maximum number of errors permitted before declaring failure.
  • fix_geometry (str) – String specifying how to handle certain problem geometries, including those which do not follow counter-clockwise winding order (which is required by the GeoJSON spec but not many popular tools). Allowed values are reject (reject invalid geometries with an error), fix (correct invalid geometries if possible and use this corrected value when creating the feature), and accept (the default) which will correct the geometry for internal use but retain the original geometry in the results.
Return type:

str
Returns:

The upload id.
SEARCH_PAGE_SIZE = 1000
TIMEOUT = (9.5, 60)
properties = <descarteslabs.common.property_filtering.filtering.GenericProperties object>