-
Notifications
You must be signed in to change notification settings - Fork 154
Open
Description
Describe the bug
While using the cell_convert
step during validation, I encountered an issue where the same error is reported across multiple resources (tables), even when the specified column does not exist in one of them.
To Reproduce
I was validating multiple CSV files (screening_profile_data
and screening_observation_data
) using a pipeline step like:
{
"name": "screening_profile_data",
"type": "table",
"valid": false,
"place": "<memory>",
"labels": [],
"stats": {
"errors": 1,
"warnings": 0,
"seconds": 0.197
},
"warnings": [],
"errors": [
{
"type": "step-error",
"title": "Step Error",
"description": "Provided step is not valid",
"message": "Step is not valid: \"cell_convert\" raises \"selection is not a field or valid field index: 'SCREENING_ENTITY_ID_CODE_SYSTEM'\"",
"tags": [],
"note": "\"cell_convert\" raises \"selection is not a field or valid field index: 'SCREENING_ENTITY_ID_CODE_SYSTEM'\""
}
]
},
{
"name": "screening_observation_data",
"type": "table",
"valid": false,
"place": "nyher-fhir-ig-example/SDOH_ScreeningObs_CareRidgeSCN_testcase1_20250312040214.csv",
"labels": [],
"stats": {
"errors": 1,
"warnings": 0,
"seconds": 0.21,
"md5": "38f272992a5cdcf1bf85f896e1192297",
"sha256": "60b88259afdcdc3ef94400aca63a9643c4c2f00c38224c0e681753140145d6f7",
"bytes": 14676,
"fields": 22
},
"warnings": [],
"errors": [
{
"type": "step-error",
"title": "Step Error",
"description": "Provided step is not valid",
"message": "Step is not valid: \"cell_convert\" raises \"selection is not a field or valid field index: 'SCREENING_ENTITY_ID_CODE_SYSTEM'\"",
"tags": [],
"note": "\"cell_convert\" raises \"selection is not a field or valid field index: 'SCREENING_ENTITY_ID_CODE_SYSTEM'\""
}
]
},
Observed Behavior
Both files report the following error even though only one of them actually includes the SCREENING_ENTITY_ID_CODE_SYSTEM
column:
"message": "Step is not valid: \"cell_convert\" raises \"selection is not a field or valid field index: 'SCREENING_ENTITY_ID_CODE_SYSTEM'\""
Metadata
Metadata
Assignees
Labels
No labels