Skip to content

Task graph schema validation #130

@amitschang

Description

@amitschang

Attaching a schema which describes necessary and produced columns of each task could aid in pre-execution validation. For example:

  • Task A takes in "id" column, produces "data" column
  • Task B takes in "id" and "data", produces "measurement"
  • Task C takes "measurement", produces "result"
  • Task D takes "measurement", "data", produces "another_result"

These could be executed in order [A, B, C, D] or [A, B, D, C], but not [A, D].

Since the aim is to validate a possible graph and enable plugging tasks in varying arrangements, we cannot define a full schema for tasks, only the necessary input columns and the columns produced, so existing schema definitions and validation tools might not fit.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions