Metadata¶

Warning

This low-level client has been deprecated in favor of the newer object-oriented Catalog client.

class Metadata(url=None, auth=None, retries=None)[source]¶

Image Metadata Service

Any methods that take start and end timestamps accept most common date/time formats as a string. If no explicit timezone is given, the timestamp is assumed to be in UTC. For example '2012-06-01' means June 1st 2012 00:00 in UTC, '2012-06-01 00:00+02:00' means June 1st 2012 00:00 in GMT+2.

Parameters:

url (str) – URL for the metadata service. Only change this if you are being asked to use a non-default Descartes Labs catalog. If not set, then descarteslabs.config.get_settings().METADATA_URL will be used.
auth (Auth) – A custom user authentication (defaults to the user authenticated locally by token information on disk or by environment variables)
retries (urllib3.util.retry.Retry) – A custom retry configuration used for all API requests (defaults to a reasonable amount of retries)

Attributes:

`TIMEOUT`
`properties`

Methods:

`available_products`()	Get the list of product identifiers you have access to.
`bands`(products[, limit, offset, wavelength, …])	Search for imagery data bands that you have access to.
`derived_bands`([bands, require_bands, limit, …])	Search for predefined derived bands that you have access to.
`features`(products[, sat_ids, date, place, …])	Generator that efficiently scrolls through the search results.
`get`(image_id)	Get metadata of a single image.
`get_band`(band_id)	Get information about a single band.
`get_bands_by_id`(id_)	For a given image source id, return the available bands.
`get_bands_by_product`(product_id)	All bands (including derived bands) available in a product.
`get_by_ids`(ids[, fields, ignore_not_found])	Get metadata for multiple images by image id.
`get_derived_band`(derived_band_id)	Get information about a single derived band.
`get_product`(product_id)	Get information about a single product.
`ids`(products[, sat_ids, date, place, geom, …])	Search metadata given a spatio-temporal query.
`paged_search`(products[, sat_ids, date, …])	Execute a metadata query in a paged manner, with up to 10,000 items per page.
`products`([bands, limit, offset, owner, text])	Search products that are available on the platform.
`search`(products[, sat_ids, date, place, …])	Search metadata given a spatio-temporal query.
`summary`(products[, sat_ids, date, interval, …])	Get a summary of the results for the specified spatio-temporal query.

available_products()[source]¶

Get the list of product identifiers you have access to.

Returns:	List of product ids
Return type:	DotList

Example::

>>> from descarteslabs.client.services.metadata import Metadata
>>> products = Metadata().available_products()
>>> products  
['landsat:LC08:PRE:TOAR']

bands(products, limit=None, offset=None, wavelength=None, resolution=None, tags=None, bands=None, **kwargs)[source]¶

Search for imagery data bands that you have access to.

Parameters:	products (list(str)) – A list of product(s) to return bands for. May not be empty. limit (int) – Number of results to return. offset (int) – Index to start at when returning results. wavelength (float) – A wavelength in nm e.g 700 that the band sensor must measure. resolution (int) – The resolution in meters per pixel e.g 30 of the data available in this band. tags (list(str)) – A list of tags that the band must have in its own tag list.
Returns:	List of dicts containing at most limit bands. Empty if there are no bands matching query (e.g. product id not available).
Return type:	DotList(DotDict)

derived_bands(bands=None, require_bands=None, limit=None, offset=None, **kwargs)[source]¶

Search for predefined derived bands that you have access to.

Parameters:	bands (list(str)) – Limit the derived bands to ones that can be computed using this list of spectral bands. e.g [“red”, “nir”, “swir1”] require_bands (bool) – Control whether searched bands must contain all the spectral bands passed in the bands param. Defaults to False. limit (int) – Number of results to return. offset (int) – Index to start at when returning results.
Returns:	List of dicts containing at most limit bands.
Return type:	DotList(DotDict)

features(products, sat_ids=None, date='acquired', place=None, geom=None, start_datetime=None, end_datetime=None, cloud_fraction=None, cloud_fraction_0=None, fill_fraction=None, storage_state=None, q=None, fields=None, batch_size=1000, dltile=None, sort_field=None, sort_order='asc', randomize=None, **kwargs)[source]¶

Generator that efficiently scrolls through the search results.

Parameters:	batch_size (int) – Number of features to fetch per request.
Returns:	Generator of GeoJSON `Feature` objects. Empty if no matching images found.
Return type:	generator

Example:

>>> from descarteslabs.client.services.metadata import Metadata
>>> features = Metadata().features(
...     "landsat:LC08:PRE:TOAR",
...     start_datetime='2016-01-01',
...     end_datetime="2016-03-01"
... )
>>> total = 0
>>> for f in features:
...     total += 1

>>> total 
31898

get(image_id)[source]¶

Get metadata of a single image.

Parameters:	image_id (str) – Image identifier.
Returns:	A dictionary of metadata for a single image.
Return type:	DotDict
Raises:	NotFoundError – Raised if image id cannot be found.

Example:

>>> from descarteslabs.client.services.metadata import Metadata
>>> meta = Metadata().get('landsat:LC08:PRE:TOAR:meta_LC80270312016188_v1')
>>> keys = list(meta.keys())
>>> keys.sort()
>>> keys  
['acquired', 'area', 'bits_per_pixel', 'bright_fraction', 'bucket',
 'cloud_fraction', 'cloud_fraction_0', 'confidence_dlsr', 'cs_code',
 'descartes_version', 'file_md5s', 'file_sizes', 'files', 'fill_fraction',
 'geolocation_accuracy', 'geometry', 'geotrans', 'id', 'identifier', 'key',
 'modified', 'processed', 'product', 'proj4', 'projcs', 'published',
 'raster_size', 'reflectance_scale', 'roll_angle', 'sat_id',
 'solar_azimuth_angle', 'solar_elevation_angle', 'storage_state',
 'sw_version', 'terrain_correction', 'tile_id']

get_band(band_id)[source]¶

Get information about a single band.

Parameters:	band_id (str) – Band Identifier.
Returns:	A dictionary with metadata for a single band.
Return type:	DotDict
Raises:	NotFoundError – Raised if an band id cannot be found.

get_bands_by_id(id_)[source]¶

For a given image source id, return the available bands.

Parameters:	id (str) – A `Metadata` image identifier.
Returns:	A dictionary of band entries and their metadata.
Return type:	DotDict
Raises:	NotFoundError – Raised if image id cannot be found.

Example:

>>> from descarteslabs.client.services.metadata import Metadata
>>> bands = Metadata().get_bands_by_id('landsat:LC08:PRE:TOAR:meta_LC80270312016188_v1')
>>> ndvi_info = bands['derived:ndvi'] # View NDVI band information
>>> ndvi_info['physical_range']
[-1.0, 1.0]

get_bands_by_product(product_id)[source]¶

All bands (including derived bands) available in a product.

Parameters:	product_id (str) – A product identifier.
Returns:	A dictionary mapping band ids to dictionaries of their metadata. Returns empty dict if product id not found.
Return type:	DotDict

get_by_ids(ids, fields=None, ignore_not_found=True, **kwargs)[source]¶

Get metadata for multiple images by image id. The response contains list of found images in the order of the given ids.

Parameters:	ids (list(str)) – Image identifiers. fields (list(str)) – Properties to return. ignore_not_found (bool) – For image id lookups that fail: if `True`, ignore; if `False`, raise `NotFoundError`. Default is `True`.
Returns:	List of image metadata dicts.
Return type:	DotList(DotDict)
Raises:	NotFoundError – Raised if an image id cannot be found and ignore_not_found set to False (default is True)

get_derived_band(derived_band_id)[source]¶

Get information about a single derived band.

Parameters:	derived_band_id (str) – Derived band identifier.
Returns:	A dictionary with metadata for a single derived band.
Return type:	DotDict
Raises:	NotFoundError – Raised if a band id cannot be found.

get_product(product_id)[source]¶

Get information about a single product.

Parameters:	product_id (str) – Product Identifier.
Returns:	A dictionary with metadata for a single product.
Return type:	DotDict
Raises:	NotFoundError – Raised if an product id cannot be found.

ids(products, sat_ids=None, date='acquired', place=None, geom=None, start_datetime=None, end_datetime=None, cloud_fraction=None, cloud_fraction_0=None, fill_fraction=None, storage_state=None, q=None, limit=100, dltile=None, sort_field=None, sort_order=None, randomize=None, **kwargs)[source]¶

Search metadata given a spatio-temporal query. All parameters are optional.

Parameters:	products (list(str)) – Products identifier(s). May not be empty. sat_ids (list(str)) – Satellite identifier(s). date (str) – The date field to use for search (e.g. acquired). place (str) – A slug identifier to be used as a region of interest. geom (str) – A GeoJSON or WKT region of interest. start_datetime (str) – Desired starting timestamp, in any common format. end_datetime (str) – Desired ending timestamp, in any common format. cloud_fraction (float) – Maximum cloud fraction, calculated by data provider. cloud_fraction_0 (float) – Maximum cloud fraction, calculated by cloud mask pixels. fill_fraction (float) – Minimum scene fill fraction, calculated as valid/total pixels. storage_state (str) – Filter results based on storage_state value. Allowed values are “available”, “remote”, or None, which returns all results regardless of storage_state value. q (Expression) – Expression for filtering the results. See `properties`. limit (int) – Number of items to return. dltile (str) – A dltile key used to specify the search geometry, an alternative to the `geom` argument. sort_field (str) – Property to sort on. sort_order (str) – Order of sort. randomize (bool) – Randomize the results. You may also use an int or str as an explicit seed.
Returns:	List of image identifiers. Empty list if no matching images found.
Return type:	DotList(str)

Example:

>>> from descarteslabs.client.services.metadata import Metadata
>>> iowa_geom = {
...     "coordinates": [[
...         [-96.498997, 42.560832],
...         [-95.765645, 40.585208],
...         [-91.729115, 40.61364],
...         [-91.391613, 40.384038],
...         [-90.952233, 40.954047],
...         [-91.04589, 41.414085],
...         [-90.343228, 41.587833],
...         [-90.140613, 41.995999],
...         [-91.065059, 42.751338],
...         [-91.217706, 43.50055],
...         [-96.599191, 43.500456],
...         [-96.498997, 42.560832]
...     ]],
...     "type": "Polygon"
... }
>>> ids = Metadata().ids(geom=iowa_geom,
...                      products=['landsat:LC08:PRE:TOAR'],
...                      start_datetime='2016-07-01',
...                      end_datetime='2016-07-31T23:59:59')
>>> len(ids)  
2

>>> ids  
['landsat:LC08:PRE:TOAR:meta_LC80260322016197_v1', 'landsat:LC08:PRE:TOAR:meta_LC80270312016188_v1']

paged_search(products, sat_ids=None, date='acquired', place=None, geom=None, start_datetime=None, end_datetime=None, cloud_fraction=None, cloud_fraction_0=None, fill_fraction=None, storage_state=None, q=None, limit=100, fields=None, dltile=None, sort_field=None, sort_order='asc', randomize=None, continuation_token=None, **kwargs)[source]¶

Execute a metadata query in a paged manner, with up to 10,000 items per page.

Most clients should use features() instead, which batch searches into smaller requests and handles the paging for you.

Parameters:	products (list(str)) – Product Identifier(s). May not be empty. sat_ids (list(str)) – Satellite identifier(s). date (str) – The date field to use for search (default is acquired). place (str) – A slug identifier to be used as a region of interest. geom (str) – A GeoJSON or WKT region of interest or a Shapely shape object. start_datetime (str) – Desired starting timestamp, in any common format. end_datetime (str) – Desired ending timestamp, in any common format. cloud_fraction (float) – Maximum cloud fraction, calculated by data provider. cloud_fraction_0 (float) – Maximum cloud fraction, calculated by cloud mask pixels. fill_fraction (float) – Minimum scene fill fraction, calculated as valid/total pixels. storage_state (str) – Filter results based on storage_state value. Allowed values are “available”, “remote”, or None, which returns all results regardless of storage_state value. q (Expression) – Expression for filtering the results. See `properties`. limit (int) – Maximum number of items per page to return. fields (list(str)) – Properties to return. dltile (str) – A dltile key used to specify the search geometry, an alternative to the `geom` argument. sort_field (str) – Property to sort on. sort_order (str) – Order of sort. randomize (bool) – Randomize the results. You may also use an int or str as an explicit seed. continuation_token (str) – None for new query, or the properties.continuation_token value from the returned FeatureCollection from a previous invocation of this method to page through a large result set.
Returns:	GeoJSON `FeatureCollection` containing at most limit features.
Return type:	DotDict

products(bands=None, limit=None, offset=None, owner=None, text=None, **kwargs)[source]¶

Search products that are available on the platform.

Parameters:	bands (list(str)) – Band name(s) e.g [“red”, “nir”] to filter products by. Note that products must match all bands that are passed. limit (int) – Number of results to return. offset (int) – Index to start at when returning results. owner (str) – Filter products by the owner’s uuid. text (str) – Filter products by string match.
Returns:	List of dicts containing at most limit products. Empty if no matching products are found.
Return type:	DotList(DotDict)

search(products, sat_ids=None, date='acquired', place=None, geom=None, start_datetime=None, end_datetime=None, cloud_fraction=None, cloud_fraction_0=None, fill_fraction=None, storage_state=None, q=None, limit=100, fields=None, dltile=None, sort_field=None, sort_order='asc', randomize=None, **kwargs)[source]¶

Search metadata given a spatio-temporal query. All parameters are optional.

If performing a large query, consider using the iterator features() instead.

Parameters:	products (list(str)) – Product Identifier(s). May not be empty. sat_ids (list(str)) – Satellite identifier(s). date (str) – The date field to use for search (e.g. acquired). place (str) – A slug identifier to be used as a region of interest. geom (str) – A GeoJSON or WKT region of interest. start_datetime (str) – Desired starting timestamp, in any common format. end_datetime (str) – Desired ending timestamp, in any common format. cloud_fraction (float) – Maximum cloud fraction, calculated by data provider. cloud_fraction_0 (float) – Maximum cloud fraction, calculated by cloud mask pixels. fill_fraction (float) – Minimum scene fill fraction, calculated as valid/total pixels. storage_state (str) – Filter results based on storage_state value. Allowed values are “available”, “remote”, or None, which returns all results regardless of storage_state value. q (Expression) – Expression for filtering the results. See `properties`. limit (int) – Maximum number of items to return. fields (list(str)) – Properties to return. dltile (str) – A dltile key used to specify the search geometry, an alternative to the `geom` argument. sort_field (str) – Property to sort on. sort_order (str) – Order of sort. randomize (bool) – Randomize the results. You may also use an int or str as an explicit seed.
Returns:	GeoJSON `FeatureCollection`. Empty features list if no matching images found.
Return type:	DotDict

Note that as of release 0.16.0 the continuation_token token has been removed. Please use the paged_search() if you require this feature.

Example:

>>> from descarteslabs.client.services.metadata import Metadata
>>> iowa_geom = {
...     "coordinates": [[
...         [-96.498997, 42.560832],
...         [-95.765645, 40.585208],
...         [-91.729115, 40.61364],
...         [-91.391613, 40.384038],
...         [-90.952233, 40.954047],
...         [-91.04589, 41.414085],
...         [-90.343228, 41.587833],
...         [-90.140613, 41.995999],
...         [-91.065059, 42.751338],
...         [-91.217706, 43.50055],
...         [-96.599191, 43.500456],
...         [-96.498997, 42.560832]
...     ]],
...     "type": "Polygon"
... }
>>> scenes = Metadata().search(
...     geom=iowa_geom,
...     products=['landsat:LC08:PRE:TOAR'],
...     start_datetime='2016-07-01',
...     end_datetime='2016-07-31T23:59:59'
... )
>>> len(scenes['features'])  
2

summary(products, sat_ids=None, date='acquired', interval=None, place=None, geom=None, start_datetime=None, end_datetime=None, cloud_fraction=None, cloud_fraction_0=None, fill_fraction=None, storage_state=None, q=None, pixels=None, dltile=None, **kwargs)[source]¶

Get a summary of the results for the specified spatio-temporal query.

Parameters:	products (list(str)) – Product identifier(s). May not be empty. sat_ids (list(str)) – Satellite identifier(s). date (str) – The date field to use for search (e.g. acquired). interval (str) – Part of the date to aggregate over (e.g. day). The list of possibilites is: `year` or `y` `quarter` `month` or `M` `week` or `q` `day` or `d` `hour` or `h` `minute` or `m` `product` place (str) – A slug identifier to be used as a region of interest. geom (str) – A GeoJSON or WKT region of interest or a Shapely shape object. start_datetime (str) – Desired starting timestamp, in any common format. end_datetime (str) – Desired ending timestamp, in any common format. cloud_fraction (float) – Maximum cloud fraction, calculated by data provider. cloud_fraction_0 (float) – Maximum cloud fraction, calculated by cloud mask pixels. fill_fraction (float) – Minimum scene fill fraction, calculated as valid/total pixels. storage_state (str) – Filter results based on storage_state value. Allowed values are “available”, “remote”, or None, which returns all results regardless of storage_state value. q (Expression) – Expression for filtering the results. See `properties`. pixels (bool) – Whether to include pixel counts in summary calculations. dltile (str) – A dltile key used to specify the search geometry, an alternative to the `geom` argument.
Returns:	Dictionary containing summary of products that match query. Empty products list if no matching products found.
Return type:	DotDict

Example:

>>> from descarteslabs.client.services.metadata import Metadata
>>> iowa_geom = {
...     "coordinates": [[
...         [-96.498997, 42.560832],
...         [-95.765645, 40.585208],
...         [-91.729115, 40.61364],
...         [-91.391613, 40.384038],
...         [-90.952233, 40.954047],
...         [-91.04589, 41.414085],
...         [-90.343228, 41.587833],
...         [-90.140613, 41.995999],
...         [-91.065059, 42.751338],
...         [-91.217706, 43.50055],
...         [-96.599191, 43.500456],
...         [-96.498997, 42.560832]
...     ]],
...     "type": "Polygon"
... }
>>> Metadata().summary(geom=iowa_geom,
...                    products=['landsat:LC08:PRE:TOAR'],
...                    start_datetime='2016-07-06',
...                    end_datetime='2016-07-07',
...                    interval='hour',
...                    pixels=True)
{
    'bytes': 290740659,
    'count': 3,
    'items': [
      {
        'bytes': 191795912,
        'count': 2,
        'date': '2016-07-06T16:00:00.000Z',
        'pixels': 500639616,
        'timestamp': 1467820800
      },
      {
        'bytes': 98944747,
        'count': 1,
        'date': '2016-07-06T17:00:00.000Z',
        'pixels': 251142720,
        'timestamp': 1467824400
      }
    ],
    'pixels': 751782336,
    'products': ['landsat:LC08:PRE:TOAR']
}

TIMEOUT = (9.5, 120)¶

properties = <descarteslabs.common.property_filtering.filtering.Properties object>¶

Filtering¶

properties¶

Most of the metadata searching functions allow for flexible filtering on scene metadata.

>>> from descarteslabs.client.services.metadata import Metadata, properties as p
>>> metadata = Metadata()

>>> metadata.summary(products=["sentinel-2:L1C"], q=(p.acquired > "2017"))
{'bytes': 28174123929918, 'count': 330034, 'products': ['sentinel-2:L1C']}

You can pass multiple filter conditions, all of which need to be true for results to be included:

>>> metadata.summary(
...     products=["sentinel-2:L1C"],
...     q=[
...         p.acquired > "2017",
...         p.cloud_fraction < 0.5,
...         150 < p.azimuth_angle < 160
...     ]
... )
{'bytes': 747678539408, 'count': 5979, 'products': ['sentinel-2:L1C']}

You cannot use the boolean keywords and and or because of Python language limitations; use bitwise operators & and | instead to express more complex boolean logic. & stands for a boolean “and”, | stands for a boolean “or”:

>>> metadata.summary(
...     products=["sentinel-2:L1C"],
...     q=((p.acquired > "2017") & (p.acquired < "2018")) | (p.acquired > "2019")
... )
{'bytes': 310745851027609, 'count': 3922704, 'products': [u'sentinel-2:L1C']}

Supported query attributes:

absolute_orbit -

Number of orbits elapsed since the first ascending node crossing after launch.

Only present for Sentinel 2 imagery.
acquired - Acquisition date and time.
archived - Archival date and time.
area - Area of the captured scene in square kilometers.
azimuth_angle - Satellite azimuth angle in degrees.
cirrus_fraction - Fraction of pixels identified as cirrus clouds. Only applicable to Sentinel-2 data.

See https://sentinel.esa.int/web/sentinel/technical-guides/sentinel-2-msi/level-1c/cloud-masks for more information.
cloud_fraction - Fraction of cloudy pixels determined by the vendor cloud mask.
cloud_fraction_0 - Fraction of cloudy pixels supplied by the vendor.
earth_sun_distance - Earth-sun distance in astronomical units.
geolocation_accuracy -

RMSE of the geometric residuals (pixels) in both line and sample directions measured on the terrain-corrected product independently using GLS2000. Only applicable to Landsat data.
opaque_fraction - Fraction of pixels identified as dense clouds. Only applicable to Sentinel-2 data.

See https://sentinel.esa.int/web/sentinel/technical-guides/sentinel-2-msi/level-1c/cloud-masks for more information.
product - Product identifier (e.g. landsat:LC08:PRE:TOAR)
processed - Timestamp of when the scene was processed into the platform.
published - Date and time when the scene was published.
relative_orbit -

Count of orbits from 1 to the number contained in a repeat cycle. Relative orbit number 1 corresponds to the orbit whose ascending node crossing is closest to the Greenwich Meridian (eastwards).

Only present for Sentinel 1 and 2 imagery.
roll_angle - Satellite zenith angle for Landsat 8 imagery.
sat_id - Vendor-specific satellite ID.
solar_azimuth_angle - Solar azimuth angle when the scene was captured.
solar_elevation_angle - Solar elevation angle when the scene was captured.
terrain_correction - Landsat Level-1 data processing level
- L1T (Pre-Collection) / L1TP (Collection 1) - Radiometrically calibrated and orthorectified using ground control points and DEM data to correct for relief displacement. These are the highest quality Level-1 products suitable for pixel-level time series analysis.
- L1GT (Pre-Collection) / L1GT (Collection 1) - Radiometrically calibrated and with systematic geometric corrections applied using the spacecraft ephemeris data and DEM data to correct for relief displacement.
- L1G (Pre-Collection) / L1GS (Collection 1) - Radiometrically calibrated and with only systematic geometric corrections applied using the spacecraft ephemeris data.
See https://landsat.usgs.gov/landsat-processing-details/ for more information.
tile_id - Vendor-specific tile ID.
view_angle - Satellite view angle in degrees.