Metadata

class Metadata(url=None, auth=None, retries=None)[source]

Image Metadata Service

Any methods that take start and end timestamps accept most common date/time formats as a string. If no explicit timezone is given, the timestamp is assumed to be in UTC. For example '2012-06-01' means June 1st 2012 00:00 in UTC, '2012-06-01 00:00+02:00' means June 1st 2012 00:00 in GMT+2.

Parameters:
  • url (str) – A HTTP URL pointing to a version of the storage service (defaults to current version)
  • auth (Auth) – A custom user authentication (defaults to the user authenticated locally by token information on disk or by environment variables)
  • retries (urllib3.util.retry.Retry) – A custom retry configuration used for all API requests (defaults to a reasonable amount of retries)
available_products()[source]

Get the list of product identifiers you have access to.

Returns:List of product ids
Return type:DotList
Example::
>>> from descarteslabs.client.services import Metadata
>>> products = Metadata().available_products()
>>> products  # doctest: +SKIP
['landsat:LC08:PRE:TOAR']
bands(products=None, limit=None, offset=None, wavelength=None, resolution=None, tags=None, bands=None, **kwargs)[source]

Search for imagery data bands that you have access to.

Parameters:
  • products (list(str)) – A list of product(s) to return bands for.
  • limit (int) – Number of results to return.
  • offset (int) – Index to start at when returning results.
  • wavelength (float) – A wavelength in nm e.g 700 that the band sensor must measure.
  • resolution (int) – The resolution in meters per pixel e.g 30 of the data available in this band.
  • tags (list(str)) – A list of tags that the band must have in its own tag list.
Returns:

List of dicts containing at most limit bands. Empty if there are no bands matching query (e.g. product id not available).

Return type:

DotList(DotDict)

derived_bands(bands=None, require_bands=None, limit=None, offset=None, **kwargs)[source]

Search for predefined derived bands that you have access to.

Parameters:
  • bands (list(str)) – Limit the derived bands to ones that can be computed using this list of spectral bands. e.g [“red”, “nir”, “swir1”]
  • require_bands (bool) – Control whether searched bands must contain all the spectral bands passed in the bands param. Defaults to False.
  • limit (int) – Number of results to return.
  • offset (int) – Index to start at when returning results.
Returns:

List of dicts containing at most limit bands.

Return type:

DotList(DotDict)

features(products=None, sat_ids=None, date='acquired', place=None, geom=None, start_datetime=None, end_datetime=None, cloud_fraction=None, cloud_fraction_0=None, fill_fraction=None, storage_state=None, q=None, fields=None, batch_size=1000, dltile=None, sort_field=None, sort_order='asc', randomize=None, **kwargs)[source]

Generator that efficiently scrolls through the search results.

Parameters:batch_size (int) – Number of features to fetch per request.
Returns:Generator of GeoJSON Feature objects. Empty if no matching images found.
Return type:generator

Example:

>>> from descarteslabs.client.services import Metadata
>>> features = Metadata().features(
...     "landsat:LC08:PRE:TOAR",
...     start_datetime='2016-01-01',
...     end_datetime="2016-03-01"
... )
>>> total = 0
>>> for f in features:
...     total += 1

>>> total 
31898
get(image_id)[source]

Get metadata of a single image.

Parameters:image_id (str) – Image identifier.
Returns:A dictionary of metadata for a single image.
Return type:DotDict
Raises:NotFoundError – Raised if image id cannot be found.

Example:

>>> from descarteslabs.client.services import Metadata
>>> meta = Metadata().get('landsat:LC08:PRE:TOAR:meta_LC80270312016188_v1')
>>> keys = list(meta.keys())
>>> keys.sort()
>>> keys
['acquired', 'area', 'bits_per_pixel', 'bright_fraction', 'bucket', 'cloud_fraction',
 'cloud_fraction_0', 'confidence_dlsr', 'cs_code', 'descartes_version', 'file_md5s', 'file_sizes',
 'files', 'fill_fraction', 'geolocation_accuracy', 'geometry', 'geotrans', 'id', 'identifier',
 'key', 'processed', 'product', 'proj4', 'projcs', 'published', 'raster_size', 'reflectance_scale',
 'roll_angle', 'sat_id', 'solar_azimuth_angle', 'solar_elevation_angle', 'storage_state',
 'sw_version', 'terrain_correction', 'tile_id']
get_band(band_id)[source]

Get information about a single band.

Parameters:band_id (str) – Band Identifier.
Returns:A dictionary with metadata for a single band.
Return type:DotDict
Raises:NotFoundError – Raised if an band id cannot be found.
get_bands_by_id(id_)[source]

For a given image source id, return the available bands.

Parameters:id (str) – A Metadata image identifier.
Returns:A dictionary of band entries and their metadata.
Return type:DotDict
Raises:NotFoundError – Raised if image id cannot be found.

Example:

>>> from descarteslabs.client.services import Metadata
>>> bands = Metadata().get_bands_by_id('landsat:LC08:PRE:TOAR:meta_LC80270312016188_v1')
>>> ndvi_info = bands['derived:ndvi'] # View NDVI band information
>>> ndvi_info['physical_range']
[-1.0, 1.0]
get_bands_by_product(product_id)[source]

All bands (includig derived bands) available in a product.

Parameters:product_id (str) – A product identifier.
Returns:A dictionary mapping band ids to dictionaries of their metadata. Returns empty dict if product id not found.
Return type:DotDict
get_by_ids(ids, fields=None, ignore_not_found=True, **kwargs)[source]

Get metadata for multiple images by image id. The response contains list of found images in the order of the given ids.

Parameters:
  • ids (list(str)) – Image identifiers.
  • fields (list(str)) – Properties to return.
  • ignore_not_found (bool) – For image id lookups that fail: if True, ignore; if False, raise NotFoundError. Default is True.
Returns:

List of image metadata dicts.

Return type:

DotList(DotDict)

Raises:

NotFoundError – Raised if an image id cannot be found and ignore_not_found set to False (default is True)

get_derived_band(derived_band_id)[source]

Get information about a single derived band.

Parameters:derived_band_id (str) – Derived band identifier.
Returns:A dictionary with metadata for a single derived band.
Return type:DotDict
Raises:NotFoundError – Raised if an band id cannot be found.
get_product(product_id)[source]

Get information about a single product.

Parameters:product_id (str) – Product Identifier.
Returns:A dictionary with metadata for a single product.
Return type:DotDict
Raises:NotFoundError – Raised if an product id cannot be found.
ids(products=None, sat_ids=None, date='acquired', place=None, geom=None, start_datetime=None, end_datetime=None, cloud_fraction=None, cloud_fraction_0=None, fill_fraction=None, storage_state=None, q=None, limit=100, dltile=None, sort_field=None, sort_order=None, randomize=None, **kwargs)[source]

Search metadata given a spatio-temporal query. All parameters are optional.

Parameters:
  • products (list(str)) – Products identifier(s).
  • sat_ids (list(str)) – Satellite identifier(s).
  • date (str) – The date field to use for search (e.g. acquired).
  • place (str) – A slug identifier to be used as a region of interest.
  • geom (str) – A GeoJSON or WKT region of interest.
  • start_datetime (str) – Desired starting timestamp, in any common format.
  • end_datetime (str) – Desired ending timestamp, in any common format.
  • cloud_fraction (float) – Maximum cloud fraction, calculated by data provider.
  • cloud_fraction_0 (float) – Maximum cloud fraction, calculated by cloud mask pixels.
  • fill_fraction (float) – Minimum scene fill fraction, calculated as valid/total pixels.
  • storage_state (str) – Filter results based on storage_state value. Allowed values are “available”, “remote”, or None, which returns all results regardless of storage_state value.
  • q (Expression) – Expression for filtering the results. See properties.
  • limit (int) – Number of items to return.
  • dltile (str) – A dltile key used to specify the search geometry, an alternative to the geom argument.
  • sort_field (str) – Property to sort on.
  • sort_order (str) – Order of sort.
  • randomize (bool) – Randomize the results. You may also use an int or str as an explicit seed.
Returns:

List of image identifiers. Empty list if no matching images found.

Return type:

DotList(str)

Example:

>>> from descarteslabs.client.services import Metadata
>>> iowa_geom = {
...     "coordinates": [[
...         [-96.498997, 42.560832],
...         [-95.765645, 40.585208],
...         [-91.729115, 40.61364],
...         [-91.391613, 40.384038],
...         [-90.952233, 40.954047],
...         [-91.04589, 41.414085],
...         [-90.343228, 41.587833],
...         [-90.140613, 41.995999],
...         [-91.065059, 42.751338],
...         [-91.217706, 43.50055],
...         [-96.599191, 43.500456],
...         [-96.498997, 42.560832]
...     ]],
...     "type": "Polygon"
... }
>>> ids = Metadata().ids(geom=iowa_geom,
...                      products=['landsat:LC08:PRE:TOAR'],
...                      start_datetime='2016-07-01',
...                      end_datetime='2016-07-31T23:59:59')
>>> len(ids)  
2

>>> ids  
['landsat:LC08:PRE:TOAR:meta_LC80260322016197_v1', 'landsat:LC08:PRE:TOAR:meta_LC80270312016188_v1']

Execute a metadata query in a paged manner, with up to 10,000 items per page.

Most clients should use features() instead, which batch searches into smaller requests and handles the paging for you.

Parameters:
  • products (list(str)) – Product Identifier(s).
  • sat_ids (list(str)) – Satellite identifier(s).
  • date (str) – The date field to use for search (default is acquired).
  • place (str) – A slug identifier to be used as a region of interest.
  • geom (str) – A GeoJSON or WKT region of interest or a Shapely shape object.
  • start_datetime (str) – Desired starting timestamp, in any common format.
  • end_datetime (str) – Desired ending timestamp, in any common format.
  • cloud_fraction (float) – Maximum cloud fraction, calculated by data provider.
  • cloud_fraction_0 (float) – Maximum cloud fraction, calculated by cloud mask pixels.
  • fill_fraction (float) – Minimum scene fill fraction, calculated as valid/total pixels.
  • storage_state (str) – Filter results based on storage_state value. Allowed values are “available”, “remote”, or None, which returns all results regardless of storage_state value.
  • q (Expression) – Expression for filtering the results. See properties.
  • limit (int) – Maximum number of items per page to return.
  • fields (list(str)) – Properties to return.
  • dltile (str) – A dltile key used to specify the search geometry, an alternative to the geom argument.
  • sort_field (str) – Property to sort on.
  • sort_order (str) – Order of sort.
  • randomize (bool) – Randomize the results. You may also use an int or str as an explicit seed.
  • continuation_token (str) – None for new query, or the properties.continuation_token value from the returned FeatureCollection from a previous invocation of this method to page through a large result set.
Returns:

GeoJSON FeatureCollection containing at most limit features.

Return type:

DotDict

products(bands=None, limit=None, offset=None, owner=None, text=None, **kwargs)[source]

Search products that are available on the platform.

Parameters:
  • bands (list(str)) – Band name(s) e.g [“red”, “nir”] to filter products by. Note that products must match all bands that are passed.
  • limit (int) – Number of results to return.
  • offset (int) – Index to start at when returning results.
  • owner (str) – Filter products by the owner’s uuid.
  • text (str) – Filter products by string match.
Returns:

List of dicts containing at most limit products. Empty if no matching products are found.

Return type:

DotList(DotDict)

search(products=None, sat_ids=None, date='acquired', place=None, geom=None, start_datetime=None, end_datetime=None, cloud_fraction=None, cloud_fraction_0=None, fill_fraction=None, storage_state=None, q=None, limit=100, fields=None, dltile=None, sort_field=None, sort_order='asc', randomize=None, **kwargs)[source]

Search metadata given a spatio-temporal query. All parameters are optional.

If performing a large query, consider using the iterator features() instead.

Parameters:
  • products (list(str)) – Product Identifier(s).
  • sat_ids (list(str)) – Satellite identifier(s).
  • date (str) – The date field to use for search (e.g. acquired).
  • place (str) – A slug identifier to be used as a region of interest.
  • geom (str) – A GeoJSON or WKT region of interest.
  • start_datetime (str) – Desired starting timestamp, in any common format.
  • end_datetime (str) – Desired ending timestamp, in any common format.
  • cloud_fraction (float) – Maximum cloud fraction, calculated by data provider.
  • cloud_fraction_0 (float) – Maximum cloud fraction, calculated by cloud mask pixels.
  • fill_fraction (float) – Minimum scene fill fraction, calculated as valid/total pixels.
  • storage_state (str) – Filter results based on storage_state value. Allowed values are “available”, “remote”, or None, which returns all results regardless of storage_state value.
  • q (Expression) – Expression for filtering the results. See properties.
  • limit (int) – Maximum number of items to return.
  • fields (list(str)) – Properties to return.
  • dltile (str) – A dltile key used to specify the search geometry, an alternative to the geom argument.
  • sort_field (str) – Property to sort on.
  • sort_order (str) – Order of sort.
  • randomize (bool) – Randomize the results. You may also use an int or str as an explicit seed.
Returns:

GeoJSON FeatureCollection. Empty features list if no matching images found.

Return type:

DotDict

Note that as of release 0.16.0 the continuation_token token has been removed. Please use the paged_search() if you require this feature.

Example:

>>> from descarteslabs.client.services import Metadata
>>> iowa_geom = {
...     "coordinates": [[
...         [-96.498997, 42.560832],
...         [-95.765645, 40.585208],
...         [-91.729115, 40.61364],
...         [-91.391613, 40.384038],
...         [-90.952233, 40.954047],
...         [-91.04589, 41.414085],
...         [-90.343228, 41.587833],
...         [-90.140613, 41.995999],
...         [-91.065059, 42.751338],
...         [-91.217706, 43.50055],
...         [-96.599191, 43.500456],
...         [-96.498997, 42.560832]
...     ]],
...     "type": "Polygon"
... }
>>> scenes = Metadata().search(
...     geom=iowa_geom,
...     products=['landsat:LC08:PRE:TOAR'],
...     start_datetime='2016-07-01',
...     end_datetime='2016-07-31T23:59:59'
... )
>>> len(scenes['features'])  
2
summary(products=None, sat_ids=None, date='acquired', interval=None, place=None, geom=None, start_datetime=None, end_datetime=None, cloud_fraction=None, cloud_fraction_0=None, fill_fraction=None, storage_state=None, q=None, pixels=None, dltile=None, **kwargs)[source]

Get a summary of the results for the specified spatio-temporal query.

Parameters:
  • products (list(str)) – Product identifier(s).
  • sat_ids (list(str)) – Satellite identifier(s).
  • date (str) – The date field to use for search (e.g. acquired).
  • interval (str) –

    Part of the date to aggregate over (e.g. day). The list of possibilites is:

    • year or y
    • quarter
    • month or M
    • week or q
    • day or d
    • hour or h
    • minute or m
    • product
  • place (str) – A slug identifier to be used as a region of interest.
  • geom (str) – A GeoJSON or WKT region of interest or a Shapely shape object.
  • start_datetime (str) – Desired starting timestamp, in any common format.
  • end_datetime (str) – Desired ending timestamp, in any common format.
  • cloud_fraction (float) – Maximum cloud fraction, calculated by data provider.
  • cloud_fraction_0 (float) – Maximum cloud fraction, calculated by cloud mask pixels.
  • fill_fraction (float) – Minimum scene fill fraction, calculated as valid/total pixels.
  • storage_state (str) – Filter results based on storage_state value. Allowed values are “available”, “remote”, or None, which returns all results regardless of storage_state value.
  • q (Expression) – Expression for filtering the results. See properties.
  • pixels (bool) – Whether to include pixel counts in summary calculations.
  • dltile (str) – A dltile key used to specify the search geometry, an alternative to the geom argument.
Returns:

Dictionary containing summary of products that match query. Empty products list if no matching products found.

Return type:

DotDict

Example:

>>> from descarteslabs.client.services import Metadata
>>> iowa_geom = {
...     "coordinates": [[
...         [-96.498997, 42.560832],
...         [-95.765645, 40.585208],
...         [-91.729115, 40.61364],
...         [-91.391613, 40.384038],
...         [-90.952233, 40.954047],
...         [-91.04589, 41.414085],
...         [-90.343228, 41.587833],
...         [-90.140613, 41.995999],
...         [-91.065059, 42.751338],
...         [-91.217706, 43.50055],
...         [-96.599191, 43.500456],
...         [-96.498997, 42.560832]
...     ]],
...     "type": "Polygon"
... }
>>> Metadata().summary(geom=iowa_geom,
...                    products=['landsat:LC08:PRE:TOAR'],
...                    start_datetime='2016-07-06',
...                    end_datetime='2016-07-07',
...                    interval='hour',
...                    pixels=True)
{
  'bytes': 93298309,
  'count': 1,
  'items': [
    {
      'bytes': 93298309,
      'count': 1,
      'date': '2016-07-06T16:00:00.000Z',
      'pixels': 250508160,
      'timestamp': 1467820800
    }
  ],
  'pixels': 250508160,
  'products': ['landsat:LC08:PRE:TOAR']
}

Filtering

properties

Most of the metadata searching functions allow for flexible filtering on scene metadata.

>>> from descarteslabs.client.services.metadata import Metadata, properties as p
>>> metadata = Metadata()
>>> metadata.summary(products=["sentinel-2:L1C"], q=(p.acquired > "2017"))
{'bytes': 28174123929918, 'count': 330034, 'products': ['sentinel-2:L1C']}

You can pass multiple filter conditions, all of which need to be true for results to be included:

>>> metadata.summary(
...     products=["sentinel-2:L1C"],
...     q=[
...         p.acquired > "2017",
...         p.cloud_fraction < 0.5,
...         150 < p.azimuth_angle < 160
...     ]
... )
{'bytes': 747678539408, 'count': 5979, 'products': ['sentinel-2:L1C']}

You cannot use the boolean keywords and and or because of Python language limitations; use bitwise operators & and | instead to express more complex boolean logic. & stands for a boolean “and”, | stands for a boolean “or”:

>>> metadata.summary(
...     products=["sentinel-2:L1C"],
...     q=((p.acquired > "2017") & (p.acquired < "2018")) | (p.acquired > "2019")
... )
{'bytes': 310745851027609, 'count': 3922704, 'products': [u'sentinel-2:L1C']}

Supported query attributes:

  • absolute_orbit -

    Number of orbits elapsed since the first ascending node crossing after launch.

    Only present for Sentinel 2 imagery.

  • acquired - Acquisition date and time.

  • archived - Archival date and time.

  • area - Area of the captured scene in square kilometers.

  • azimuth_angle - Satellite azimuth angle in degrees.

  • cirrus_fraction - Fraction of pixels identified as cirrus clouds. Only applicable to Sentinel-2 data.

  • cloud_fraction - Fraction of cloudy pixels determined by the vendor cloud mask.

  • cloud_fraction_0 - Fraction of cloudy pixels supplied by the vendor.

  • earth_sun_distance - Earth-sun distance in astronomical units.

  • geolocation_accuracy -

    RMSE of the geometric residuals (pixels) in both line and sample directions measured on the terrain-corrected product independently using GLS2000. Only applicable to Landsat data.

  • opaque_fraction - Fraction of pixels identified as dense clouds. Only applicable to Sentinel-2 data.

  • product - Product identifier (e.g. landsat:LC08:PRE:TOAR)

  • processed - Timestamp of when the scene was processed into the platform.

  • published - Date and time when the scene was published.

  • relative_orbit -

    Count of orbits from 1 to the number contained in a repeat cycle. Relative orbit number 1 corresponds to the orbit whose ascending node crossing is closest to the Greenwich Meridian (eastwards).

    Only present for Sentinel 1 and 2 imagery.

  • roll_angle - Satellite zenith angle for Landsat 8 imagery.

  • sat_id - Vendor-specific satellite ID.

  • solar_azimuth_angle - Solar azimuth angle when the scene was captured.

  • solar_elevation_angle - Solar elevation angle when the scene was captured.

  • terrain_correction - Landsat Level-1 data processing level

    • L1T (Pre-Collection) / L1TP (Collection 1) - Radiometrically calibrated and orthorectified using ground control points and DEM data to correct for relief displacement. These are the highest quality Level-1 products suitable for pixel-level time series analysis.
    • L1GT (Pre-Collection) / L1GT (Collection 1) - Radiometrically calibrated and with systematic geometric corrections applied using the spacecraft ephemeris data and DEM data to correct for relief displacement.
    • L1G (Pre-Collection) / L1GS (Collection 1) - Radiometrically calibrated and with only systematic geometric corrections applied using the spacecraft ephemeris data.

    See https://landsat.usgs.gov/landsat-processing-details/ for more information.

  • tile_id - Vendor-specific tile ID.

  • view_angle - Satellite view angle in degrees.