Metadata (deprecated)

class Metadata(url=None, auth=None, retries=None)[source]

Image Metadata Service

Any methods that take start and end timestamps accept most common date/time formats as a string. If no explicit timezone is given, the timestamp is assumed to be in UTC. For example '2012-06-01' means June 1st 2012 00:00 in UTC, '2012-06-01 00:00+02:00' means June 1st 2012 00:00 in GMT+2.

Parameters
  • url (str) – A HTTP URL pointing to a version of the storage service (defaults to current version)

  • auth (Auth) – A custom user authentication (defaults to the user authenticated locally by token information on disk or by environment variables)

  • retries (urllib3.util.retry.Retry) – A custom retry configuration used for all API requests (defaults to a reasonable amount of retries)

available_products()[source]

Get the list of product identifiers you have access to.

Returns

List of product ids

Return type

DotList

Example::
>>> from descarteslabs.client.services import Metadata
>>> products = Metadata().available_products()
>>> products  
['landsat:LC08:PRE:TOAR']
bands(products=None, limit=None, offset=None, wavelength=None, resolution=None, tags=None, bands=None, **kwargs)[source]

Search for imagery data bands that you have access to.

Parameters
  • products (list(str)) – A list of product(s) to return bands for.

  • limit (int) – Number of results to return.

  • offset (int) – Index to start at when returning results.

  • wavelength (float) – A wavelength in nm e.g 700 that the band sensor must measure.

  • resolution (int) – The resolution in meters per pixel e.g 30 of the data available in this band.

  • tags (list(str)) – A list of tags that the band must have in its own tag list.

Returns

List of dicts containing at most limit bands. Empty if there are no bands matching query (e.g. product id not available).

Return type

DotList(DotDict)

derived_bands(bands=None, require_bands=None, limit=None, offset=None, **kwargs)[source]

Search for predefined derived bands that you have access to.

Parameters
  • bands (list(str)) – Limit the derived bands to ones that can be computed using this list of spectral bands. e.g [“red”, “nir”, “swir1”]

  • require_bands (bool) – Control whether searched bands must contain all the spectral bands passed in the bands param. Defaults to False.

  • limit (int) – Number of results to return.

  • offset (int) – Index to start at when returning results.

Returns

List of dicts containing at most limit bands.

Return type

DotList(DotDict)

features(products=None, sat_ids=None, date='acquired', place=None, geom=None, start_datetime=None, end_datetime=None, cloud_fraction=None, cloud_fraction_0=None, fill_fraction=None, storage_state=None, q=None, fields=None, batch_size=1000, dltile=None, sort_field=None, sort_order='asc', randomize=None, **kwargs)[source]

Generator that efficiently scrolls through the search results.

Parameters

batch_size (int) – Number of features to fetch per request.

Returns

Generator of GeoJSON Feature objects. Empty if no matching images found.

Return type

generator

Example:

>>> from descarteslabs.client.services import Metadata
>>> features = Metadata().features(
...     "landsat:LC08:PRE:TOAR",
...     start_datetime='2016-01-01',
...     end_datetime="2016-03-01"
... )
>>> total = 0
>>> for f in features:
...     total += 1

>>> total 
31898
get(image_id)[source]

Get metadata of a single image.

Parameters

image_id (str) – Image identifier.

Returns

A dictionary of metadata for a single image.

Return type

DotDict

Raises

NotFoundError – Raised if image id cannot be found.

Example:

>>> from descarteslabs.client.services import Metadata
>>> meta = Metadata().get('landsat:LC08:PRE:TOAR:meta_LC80270312016188_v1')
>>> keys = list(meta.keys())
>>> keys.sort()
>>> keys
['acquired', 'area', 'bits_per_pixel', 'bright_fraction', 'bucket', 'cloud_fraction',
 'cloud_fraction_0', 'confidence_dlsr', 'cs_code', 'descartes_version', 'file_md5s', 'file_sizes',
 'files', 'fill_fraction', 'geolocation_accuracy', 'geometry', 'geotrans', 'id', 'identifier',
 'key', 'processed', 'product', 'proj4', 'projcs', 'published', 'raster_size', 'reflectance_scale',
 'roll_angle', 'sat_id', 'solar_azimuth_angle', 'solar_elevation_angle', 'storage_state',
 'sw_version', 'terrain_correction', 'tile_id']
get_band(band_id)[source]

Get information about a single band.

Parameters

band_id (str) – Band Identifier.

Returns

A dictionary with metadata for a single band.

Return type

DotDict

Raises

NotFoundError – Raised if an band id cannot be found.

get_bands_by_id(id_)[source]

For a given image source id, return the available bands.

Parameters

id (str) – A Metadata image identifier.

Returns

A dictionary of band entries and their metadata.

Return type

DotDict

Raises

NotFoundError – Raised if image id cannot be found.

Example:

>>> from descarteslabs.client.services import Metadata
>>> bands = Metadata().get_bands_by_id('landsat:LC08:PRE:TOAR:meta_LC80270312016188_v1')
>>> ndvi_info = bands['derived:ndvi'] # View NDVI band information
>>> ndvi_info['physical_range']
[-1.0, 1.0]
get_bands_by_product(product_id)[source]

All bands (includig derived bands) available in a product.

Parameters

product_id (str) – A product identifier.

Returns

A dictionary mapping band ids to dictionaries of their metadata. Returns empty dict if product id not found.

Return type

DotDict

get_by_ids(ids, fields=None, ignore_not_found=True, **kwargs)[source]

Get metadata for multiple images by image id. The response contains list of found images in the order of the given ids.

Parameters
  • ids (list(str)) – Image identifiers.

  • fields (list(str)) – Properties to return.

  • ignore_not_found (bool) – For image id lookups that fail: if True, ignore; if False, raise NotFoundError. Default is True.

Returns

List of image metadata dicts.

Return type

DotList(DotDict)

Raises

NotFoundError – Raised if an image id cannot be found and ignore_not_found set to False (default is True)

get_derived_band(derived_band_id)[source]

Get information about a single derived band.

Parameters

derived_band_id (str) – Derived band identifier.

Returns

A dictionary with metadata for a single derived band.

Return type

DotDict

Raises

NotFoundError – Raised if an band id cannot be found.

get_product(product_id)[source]

Get information about a single product.

Parameters

product_id (str) – Product Identifier.

Returns

A dictionary with metadata for a single product.

Return type

DotDict

Raises

NotFoundError – Raised if an product id cannot be found.

ids(products=None, sat_ids=None, date='acquired', place=None, geom=None, start_datetime=None, end_datetime=None, cloud_fraction=None, cloud_fraction_0=None, fill_fraction=None, storage_state=None, q=None, limit=100, dltile=None, sort_field=None, sort_order=None, randomize=None, **kwargs)[source]

Search metadata given a spatio-temporal query. All parameters are optional.

Parameters
  • products (list(str)) – Products identifier(s).

  • sat_ids (list(str)) – Satellite identifier(s).

  • date (str) – The date field to use for search (e.g. acquired).

  • place (str) – A slug identifier to be used as a region of interest.

  • geom (str) – A GeoJSON or WKT region of interest.

  • start_datetime (str) – Desired starting timestamp, in any common format.

  • end_datetime (str) – Desired ending timestamp, in any common format.

  • cloud_fraction (float) – Maximum cloud fraction, calculated by data provider.

  • cloud_fraction_0 (float) – Maximum cloud fraction, calculated by cloud mask pixels.

  • fill_fraction (float) – Minimum scene fill fraction, calculated as valid/total pixels.

  • storage_state (str) – Filter results based on storage_state value. Allowed values are “available”, “remote”, or None, which returns all results regardless of storage_state value.

  • q (Expression) – Expression for filtering the results. See properties.

  • limit (int) – Number of items to return.

  • dltile (str) – A dltile key used to specify the search geometry, an alternative to the geom argument.

  • sort_field (str) – Property to sort on.

  • sort_order (str) – Order of sort.

  • randomize (bool) – Randomize the results. You may also use an int or str as an explicit seed.

Returns

List of image identifiers. Empty list if no matching images found.

Return type

DotList(str)

Example:

>>> from descarteslabs.client.services import Metadata
>>> iowa_geom = {
...     "coordinates": [[
...         [-96.498997, 42.560832],
...         [-95.765645, 40.585208],
...         [-91.729115, 40.61364],
...         [-91.391613, 40.384038],
...         [-90.952233, 40.954047],
...         [-91.04589, 41.414085],
...         [-90.343228, 41.587833],
...         [-90.140613, 41.995999],
...         [-91.065059, 42.751338],
...         [-91.217706, 43.50055],
...         [-96.599191, 43.500456],
...         [-96.498997, 42.560832]
...     ]],
...     "type": "Polygon"
... }
>>> ids = Metadata().ids(geom=iowa_geom,
...                      products=['landsat:LC08:PRE:TOAR'],
...                      start_datetime='2016-07-01',
...                      end_datetime='2016-07-31T23:59:59')
>>> len(ids)  
2

>>> ids  
['landsat:LC08:PRE:TOAR:meta_LC80260322016197_v1', 'landsat:LC08:PRE:TOAR:meta_LC80270312016188_v1']

Execute a metadata query in a paged manner, with up to 10,000 items per page.

Most clients should use features() instead, which batch searches into smaller requests and handles the paging for you.

Parameters
  • products (list(str)) – Product Identifier(s).

  • sat_ids (list(str)) – Satellite identifier(s).

  • date (str) – The date field to use for search (default is acquired).

  • place (str) – A slug identifier to be used as a region of interest.

  • geom (str) – A GeoJSON or WKT region of interest or a Shapely shape object.

  • start_datetime (str) – Desired starting timestamp, in any common format.

  • end_datetime (str) – Desired ending timestamp, in any common format.

  • cloud_fraction (float) – Maximum cloud fraction, calculated by data provider.

  • cloud_fraction_0 (float) – Maximum cloud fraction, calculated by cloud mask pixels.

  • fill_fraction (float) – Minimum scene fill fraction, calculated as valid/total pixels.

  • storage_state (str) – Filter results based on storage_state value. Allowed values are “available”, “remote”, or None, which returns all results regardless of storage_state value.

  • q (Expression) – Expression for filtering the results. See properties.

  • limit (int) – Maximum number of items per page to return.

  • fields (list(str)) – Properties to return.

  • dltile (str) – A dltile key used to specify the search geometry, an alternative to the geom argument.

  • sort_field (str) – Property to sort on.

  • sort_order (str) – Order of sort.

  • randomize (bool) – Randomize the results. You may also use an int or str as an explicit seed.

  • continuation_token (str) – None for new query, or the properties.continuation_token value from the returned FeatureCollection from a previous invocation of this method to page through a large result set.

Returns

GeoJSON FeatureCollection containing at most limit features.

Return type

DotDict

products(bands=None, limit=None, offset=None, owner=None, text=None, **kwargs)[source]

Search products that are available on the platform.

Parameters
  • bands (list(str)) – Band name(s) e.g [“red”, “nir”] to filter products by. Note that products must match all bands that are passed.

  • limit (int) – Number of results to return.

  • offset (int) – Index to start at when returning results.

  • owner (str) – Filter products by the owner’s uuid.

  • text (str) – Filter products by string match.

Returns

List of dicts containing at most limit products. Empty if no matching products are found.

Return type

DotList(DotDict)

search(products=None, sat_ids=None, date='acquired', place=None, geom=None, start_datetime=None, end_datetime=None, cloud_fraction=None, cloud_fraction_0=None, fill_fraction=None, storage_state=None, q=None, limit=100, fields=None, dltile=None, sort_field=None, sort_order='asc', randomize=None, **kwargs)[source]

Search metadata given a spatio-temporal query. All parameters are optional.

If performing a large query, consider using the iterator features() instead.

Parameters
  • products (list(str)) – Product Identifier(s).

  • sat_ids (list(str)) – Satellite identifier(s).

  • date (str) – The date field to use for search (e.g. acquired).

  • place (str) – A slug identifier to be used as a region of interest.

  • geom (str) – A GeoJSON or WKT region of interest.

  • start_datetime (str) – Desired starting timestamp, in any common format.

  • end_datetime (str) – Desired ending timestamp, in any common format.

  • cloud_fraction (float) – Maximum cloud fraction, calculated by data provider.

  • cloud_fraction_0 (float) – Maximum cloud fraction, calculated by cloud mask pixels.

  • fill_fraction (float) – Minimum scene fill fraction, calculated as valid/total pixels.

  • storage_state (str) – Filter results based on storage_state value. Allowed values are “available”, “remote”, or None, which returns all results regardless of storage_state value.

  • q (Expression) – Expression for filtering the results. See properties.

  • limit (int) – Maximum number of items to return.

  • fields (list(str)) – Properties to return.

  • dltile (str) – A dltile key used to specify the search geometry, an alternative to the geom argument.

  • sort_field (str) – Property to sort on.

  • sort_order (str) – Order of sort.

  • randomize (bool) – Randomize the results. You may also use an int or str as an explicit seed.

Returns

GeoJSON FeatureCollection. Empty features list if no matching images found.

Return type

DotDict

Note that as of release 0.16.0 the continuation_token token has been removed. Please use the paged_search() if you require this feature.

Example:

>>> from descarteslabs.client.services import Metadata
>>> iowa_geom = {
...     "coordinates": [[
...         [-96.498997, 42.560832],
...         [-95.765645, 40.585208],
...         [-91.729115, 40.61364],
...         [-91.391613, 40.384038],
...         [-90.952233, 40.954047],
...         [-91.04589, 41.414085],
...         [-90.343228, 41.587833],
...         [-90.140613, 41.995999],
...         [-91.065059, 42.751338],
...         [-91.217706, 43.50055],
...         [-96.599191, 43.500456],
...         [-96.498997, 42.560832]
...     ]],
...     "type": "Polygon"
... }
>>> scenes = Metadata().search(
...     geom=iowa_geom,
...     products=['landsat:LC08:PRE:TOAR'],
...     start_datetime='2016-07-01',
...     end_datetime='2016-07-31T23:59:59'
... )
>>> len(scenes['features'])  
2
summary(products=None, sat_ids=None, date='acquired', interval=None, place=None, geom=None, start_datetime=None, end_datetime=None, cloud_fraction=None, cloud_fraction_0=None, fill_fraction=None, storage_state=None, q=None, pixels=None, dltile=None, **kwargs)[source]

Get a summary of the results for the specified spatio-temporal query.

Parameters
  • products (list(str)) – Product identifier(s).

  • sat_ids (list(str)) – Satellite identifier(s).

  • date (str) – The date field to use for search (e.g. acquired).

  • interval (str) –

    Part of the date to aggregate over (e.g. day). The list of possibilites is:

    • year or y

    • quarter

    • month or M

    • week or q

    • day or d

    • hour or h

    • minute or m

    • product

  • place (str) – A slug identifier to be used as a region of interest.

  • geom (str) – A GeoJSON or WKT region of interest or a Shapely shape object.

  • start_datetime (str) – Desired starting timestamp, in any common format.

  • end_datetime (str) – Desired ending timestamp, in any common format.

  • cloud_fraction (float) – Maximum cloud fraction, calculated by data provider.

  • cloud_fraction_0 (float) – Maximum cloud fraction, calculated by cloud mask pixels.

  • fill_fraction (float) – Minimum scene fill fraction, calculated as valid/total pixels.

  • storage_state (str) – Filter results based on storage_state value. Allowed values are “available”, “remote”, or None, which returns all results regardless of storage_state value.

  • q (Expression) – Expression for filtering the results. See properties.

  • pixels (bool) – Whether to include pixel counts in summary calculations.

  • dltile (str) – A dltile key used to specify the search geometry, an alternative to the geom argument.

Returns

Dictionary containing summary of products that match query. Empty products list if no matching products found.

Return type

DotDict

Example:

>>> from descarteslabs.client.services import Metadata
>>> iowa_geom = {
...     "coordinates": [[
...         [-96.498997, 42.560832],
...         [-95.765645, 40.585208],
...         [-91.729115, 40.61364],
...         [-91.391613, 40.384038],
...         [-90.952233, 40.954047],
...         [-91.04589, 41.414085],
...         [-90.343228, 41.587833],
...         [-90.140613, 41.995999],
...         [-91.065059, 42.751338],
...         [-91.217706, 43.50055],
...         [-96.599191, 43.500456],
...         [-96.498997, 42.560832]
...     ]],
...     "type": "Polygon"
... }
>>> Metadata().summary(geom=iowa_geom,
...                    products=['landsat:LC08:PRE:TOAR'],
...                    start_datetime='2016-07-06',
...                    end_datetime='2016-07-07',
...                    interval='hour',
...                    pixels=True)
{
  'bytes': 93298309,
  'count': 1,
  'items': [
    {
      'bytes': 93298309,
      'count': 1,
      'date': '2016-07-06T16:00:00.000Z',
      'pixels': 250508160,
      'timestamp': 1467820800
    }
  ],
  'pixels': 250508160,
  'products': ['landsat:LC08:PRE:TOAR']
}

Filtering

properties

Most of the metadata searching functions allow for flexible filtering on scene metadata.

>>> from descarteslabs.client.services.metadata import Metadata, properties as p
>>> metadata = Metadata()
>>> metadata.summary(products=["sentinel-2:L1C"], q=(p.acquired > "2017"))
{'bytes': 28174123929918, 'count': 330034, 'products': ['sentinel-2:L1C']}

You can pass multiple filter conditions, all of which need to be true for results to be included:

>>> metadata.summary(
...     products=["sentinel-2:L1C"],
...     q=[
...         p.acquired > "2017",
...         p.cloud_fraction < 0.5,
...         150 < p.azimuth_angle < 160
...     ]
... )
{'bytes': 747678539408, 'count': 5979, 'products': ['sentinel-2:L1C']}

You cannot use the boolean keywords and and or because of Python language limitations; use bitwise operators & and | instead to express more complex boolean logic. & stands for a boolean “and”, | stands for a boolean “or”:

>>> metadata.summary(
...     products=["sentinel-2:L1C"],
...     q=((p.acquired > "2017") & (p.acquired < "2018")) | (p.acquired > "2019")
... )
{'bytes': 310745851027609, 'count': 3922704, 'products': [u'sentinel-2:L1C']}

Supported query attributes:

  • absolute_orbit -

    Number of orbits elapsed since the first ascending node crossing after launch.

    Only present for Sentinel 2 imagery.

  • acquired - Acquisition date and time.

  • archived - Archival date and time.

  • area - Area of the captured scene in square kilometers.

  • azimuth_angle - Satellite azimuth angle in degrees.

  • cirrus_fraction - Fraction of pixels identified as cirrus clouds. Only applicable to Sentinel-2 data.

  • cloud_fraction - Fraction of cloudy pixels determined by the vendor cloud mask.

  • cloud_fraction_0 - Fraction of cloudy pixels supplied by the vendor.

  • earth_sun_distance - Earth-sun distance in astronomical units.

  • geolocation_accuracy -

    RMSE of the geometric residuals (pixels) in both line and sample directions measured on the terrain-corrected product independently using GLS2000. Only applicable to Landsat data.

  • opaque_fraction - Fraction of pixels identified as dense clouds. Only applicable to Sentinel-2 data.

  • product - Product identifier (e.g. landsat:LC08:PRE:TOAR)

  • processed - Timestamp of when the scene was processed into the platform.

  • published - Date and time when the scene was published.

  • relative_orbit -

    Count of orbits from 1 to the number contained in a repeat cycle. Relative orbit number 1 corresponds to the orbit whose ascending node crossing is closest to the Greenwich Meridian (eastwards).

    Only present for Sentinel 1 and 2 imagery.

  • roll_angle - Satellite zenith angle for Landsat 8 imagery.

  • sat_id - Vendor-specific satellite ID.

  • solar_azimuth_angle - Solar azimuth angle when the scene was captured.

  • solar_elevation_angle - Solar elevation angle when the scene was captured.

  • terrain_correction - Landsat Level-1 data processing level

    • L1T (Pre-Collection) / L1TP (Collection 1) - Radiometrically calibrated and orthorectified using ground control points and DEM data to correct for relief displacement. These are the highest quality Level-1 products suitable for pixel-level time series analysis.

    • L1GT (Pre-Collection) / L1GT (Collection 1) - Radiometrically calibrated and with systematic geometric corrections applied using the spacecraft ephemeris data and DEM data to correct for relief displacement.

    • L1G (Pre-Collection) / L1GS (Collection 1) - Radiometrically calibrated and with only systematic geometric corrections applied using the spacecraft ephemeris data.

    See https://landsat.usgs.gov/landsat-processing-details/ for more information.

  • tile_id - Vendor-specific tile ID.

  • view_angle - Satellite view angle in degrees.