Skip to content

How to check whether all ARD products exist? #189

@johntruckenbrodt

Description

@johntruckenbrodt

For ARD generation, the processor.py script executes the following main steps:

  • select all scenes based on the user-defined search criteria
  • group the scenes by acquisition time (data take)
  • for each group:
    • select all overlapping MGRS tiles
      • for each tile:
        • start function ard.format

Inside ard.format, the function ard.get_datasets is called, which collects all relevant processed files and performs a check whether the part of the file overlapping with the MGRS tile contains any valid data. If it doesn't, the processed product is ignored and its files not returned by the function (because they are not relevant for creating the current ARD product).
As important as this is, it leads to the situation that the full list of ARD files that should exist can only be known if all processed SAR products exist. it would be good to know whether all ARD products exist from the source scene selection alone. Otherwise some scenes whose processing output had already been deleted are re-processed because the processor thinks some tiles are missing.
I currently see two options:

  • create dummy product folders with some note saying that the product was not created because the source products did not overlap with it
  • maintain some database/text file in ard_dir listing those products that were never created

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions