Vector ⥂

Vector is a catalog for vector data. It enables users to store, query, and display vector data — which includes everything from fault lines to thermal anomalies to material spectra to ML detections.

This quick-start guide provides examples to demonstrate some of the basic features of Vector. This guide’s prerequisite is a notebook environment that includes the Descartes Labs Python client and supports ipyleaflet. Vector tables consist of features, which themselves consist of a geometry and properties. Features are encoded as GeoJSON and support the following geometry types: Point, MultiPoint, Line, LineString, MultiLine, MultiLineString, Polygon, MultiPolygon, GeometryCollection.

Installation

If you are using Workbench, Vector will come already installed on your Python environment. If you are using another environment, install the Vector client from PyPI by running:

pip install descarteslabs-vector

Getting Started

Vector is available within the Descartes Labs client and can be imported into a notebook with the following:

import descarteslabs as dl
from descarteslabs import vector
%%js

window.maps = [];

L.Map.addInitHook(function () {
  window.maps.push(this);
});

Creating a Vector Table

Vector tables to which a user has read access at minimum can be listed by using the list() method.

for table in vector.Table.list():
  print(table.tid())

In the above code example, the tid() method is being called with each iteration, which will return the Vector table ID. Vector table IDs must be unique within a given organization.

As this is an example notebook and table IDs must be unique, any existing Vector table with the same table ID for this example will we will first be deleted.

orgname = dl.auth.Auth().payload["org"]
for table in vector.Table.list():
    if table.tid() == f"{orgname}:us-counties":
        print(f"Deleting {table}")
        table.delete()

In the code above, Vector tables are listed and iterated over. If a table already exists with the table ID equivalent to us-counties, it will be deleted by calling the delete() method.

With any potential duplicate tables now deleted, a Vector table can created by executing the following code:

table = vector.Table.create(
  "us-counties", # ID for the table
  "US Counties"  # Name for the table
  owners=[f"org:{orgname}"] # Table owners
)

In this example, a Vector table was created by invoking the create() method. This method requires a table ID, which will be prefixed with YOURORGNAME:, and a table name. For this example, the table will hold information pertaining to US counties. A table object will be returned upon success.

Ingesting Data

With the Vector table created, the table can be populated with features using the following code:

import requests
url = "https://gist.githubusercontent.com/sdwfrost/d1c73f91dd9d175998ed166eb216994a/raw/e89c35f308cee7e2e5a784e1d3afc5d449e9e4bb/counties.geojson"
response = requests.get(url)
feature_collection = response.json()
_ = table.add(feature_collection)

In the above code snippet, a GeoJSON feature collection was downloaded from a webpage. The feature collection was then added to the table with the add() method. In this case, the feature collection contains polygons of US counties.

Querying Data

After a table has been populated with features, the table can be queried using a combination of spatial and property filtering.

Spatial Filter

The comparison geometry can be represented as a GeoJSON feature. In the code below, an AOI is being created for the southwestern US.

aoi = {
    "type": "Polygon",
    "coordinates": [
        [
            [-109.63936670604541,33.07321249994284],
            [-99.18198027219883,33.07321249994284],
            [-99.18198027219883,39.037426282400816],
            [-109.63936670604541,39.037426282400816],
            [-109.63936670604541,33.07321249994284]
        ]
    ],
}

The AOI can then be used to perform a spatial intersection against the features in the table by using the following:

feature_search = table.features()
feature_search = feature_search.intersects(aoi)
feature_collection = feature_search.collect()
gdf = geopandas.GeoDataFrame.from_features(feature_collection.features_list())
gdf
US Counties that Intersect AOI

In the example above, invoking the features() method returns a search object. Spatial filters can be applied to the search object to construct queries. In this case, the intersects() method has been called with the AOI as input. Once th query has been constructed, the collect() method can be invoked to execute the query and fetch the results as a feature collection. This feature collection can then be converted to a GeoPandas dataframe for further scrutiny.

Property Filter

Aspatial or property filtering leverages the DL client's property filters and can be applied with the following:

p = dl.utils.Properties()
feature_search = table.features()
feature_search = feature_search.filter(p.NAME == "Santa Fe")
feature_collection = feature_search.collect()
gdf = geopandas.GeoDataFrame.from_features(feature_collection.features_list())
gdf
Santa Fe County

In this example, invoking the features() method returns a search object. To construct queries, property filters can be applied to the search object using the filter() method. Calling the collect() method will then execute the query and retreive the results as a feature collection. This feature collection can then be converted to a GeoPandas dataframe for further visualization and analysis.

Spatial and Property Filter

Spatial and property filtering can be combined to create more complex queries:

p = dl.utils.Properties()
feature_search = table.features()
feature_search = feature_search.intersects(aoi)
feature_search = feature_search.filter(p.STATEFP == 49)
feature_collection = feature_search.collect()

gdf = geopandas.GeoDataFrame.from_features(feature_collection.features_list())
gdf
US Counties that Intersect AOI and have FIPS Code Equal to 49

Similar to previous examples, invoking the features() method returns a search object. Property and spatial filters are then applied to the search object using the filter() and intersects methods. Finally, the query is executed with the collect() method, which will return a feature collection. This feature collection can then be converted to a GeoPandas dataframe for further experimentation.

Visualization

Working within a notebook, Vector tables can be displayed using the following:

m = ipyleaflet.Map(
    scroll_wheel_zoom=True,
    center=(44.5, -103)
)
m.zoom = 3
m
lyr = table.visualize("US Counties", m)
%%js
var layers = Object.values(window.maps).at(-1)._layers
var layer = Object.values(layers).at(-1)
layer.options.fetchOptions = {credentials: 'include'}
lyr.redraw()
US Counties

In this example, an ipyleaflet map has been created centered over the US. Invoking the visualize() method with a name and map will return a VectorTileLayer which has been added to the map display. The javascript code block is used to link your credentials with the VectorTileLayer. Calling lyr.redraw() will force the map layer to redraw using the proper credentials.

Filtering Tiles

Similar to the spatial and property filtering above, filters can also be supplied to ``visualize()’’:

p = dl.utils.Properties()
property_filter = p.STATEFP == 35
vector_tile_layer_styles = {
    "default": {
        "fill": "true",
        "fillColor": "#00ff00",
        "color": "#000000",
        "fillOpacity": 0.5,
    }
}
lyr = table.visualize(
    "New Mexico Counties",
    m,
    property_filter=property_filter,
    vector_tile_layer_styles=vector_tile_layer_styles,
)
%%js
var layers = Object.values(window.maps).at(-1)._layers
var layer = Object.values(layers).at(-1)
layer.options.fetchOptions = {credentials: 'include'}
lyr.redraw()
New Mexico Counties

In this example, the visualize() method has been invoked with a property filter and layer style. Instead of visualizing all US counties, only counties with a state FIPS code of 35, e.g. New Mexico. The VectorTileLayer returned will be formatted according to the vector_tile_layer_style.

Deleting Tables

To delete a table, simply invoke the delete() method on a table object.

try:
    table = Table.get(f"{orgname}:us-counties")
    table.delete()
except:
    pass

In this example, retrieving and deleting a table are encapsulated in a try/except block. When retrieving a table, if the table does not exist, an error is raised.