-
Notifications
You must be signed in to change notification settings - Fork 54
Open
Description
Overview
A research team within the Purdue College of Agriculture has started to use Frictionless for its data pipelines, and we would like to write custom Steps that receive configuration options from the Pipeline YAML file. For example:
steps:
# Standard Step -- ensure ID is an integer
- type: field-update
name: id
descriptor:
name: farm_animal_id
type: integer
# Custom Step -- normalize funky date formats
# Currently, in our testing, values (below) such as field_name or descriptor.name
# are not getting passed to the custom step, and it appears they are not kept by
# the custom Step mechanism. We want to capture them and pass them into the
# custom Step initialization.
- type: date-normalize
field_name: Date
descriptor:
name: Date
type: string
Metadata
Metadata
Assignees
Labels
No labels