Refactor CSV import errors and validation #3325
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
thomasleese
force-pushed
the
csv-parser-errors
branch
from
2f725f3 to
aeb288d
Compare
thomasleese
force-pushed
the
csv-parser-errors
branch
from
aeb288d to
8ea3713
Compare
thomasleese
force-pushed
the
csv-parser-errors
branch
from
8ea3713 to
61695d1
Compare
thomasleese
force-pushed
the
csv-parser-errors
branch
from
61695d1 to
bc84536
Compare
thomasleese
force-pushed
the
csv-parser-errors
branch
from
bc84536 to
a2e93d0
Compare
thomasleese
force-pushed
the
csv-parser-errors
branch
from
a2e93d0 to
6d8704c
Compare
thomasleese
force-pushed
the
csv-parser-field
branch
from
83e9022 to
0f80890
Compare
thomasleese
force-pushed
the
csv-parser-errors
branch
from
6d8704c to
1bd4eac
Compare
thomasleese
force-pushed
the
csv-parser-field
branch
2 times, most recently
from
1b4aad1 to
8503e8e
Compare
thomasleese
force-pushed
the
csv-parser-errors
branch
from
1bd4eac to
cbf9da3
Compare
thomasleese
force-pushed
the
csv-parser-errors
branch
from
cbf9da3 to
90f5db8
Compare
This updates the validators to support passing a `nil` value, with `allow_blank` set and accept that `nil` is a blank value and therefore don't raise a validation error.
This refactors the validations in preparation for updating the error handling to present the user with the original header and the cell that was that invalid.
This renames it to `REASONS_NOT_ADMINISTERED` to make it clear what the reason is referring to.
misaka
approved these changes
Apr 8, 2025
This refactors the validations in preparation for updating the error handling to present the user with the original header and the cell that was that invalid.
This updates the error handling on the class and cohort imports to use the header values that were provided by the user in the CSV file to ensure that what we're showing to the user matches exactly what they uploaded.
thomasleese
force-pushed
the
csv-parser-errors
branch
from
90f5db8 to
6b60c8b
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
This updates the error handling on the immunisation imports to use the header values that were provided by the user in the CSV file to ensure that what we're showing to the user matches exactly what they uploaded.
thomasleese
force-pushed
the
csv-parser-errors
branch
from
6b60c8b to
72f65e3
Compare
|
thomasleese
added a commit
that referenced
this pull request
Apr 8, 2025
This updates the import process to allow users to upload a file in the SystmOne format which has a different set of fields to the existing format. We haven't got a real example of this type of file for testing, so this is a best guess at how the format should be parsed. We haven't updated any parts of the UI in this change, we might want to do this in a follow up PR to describe the two different formats. To build this, the import mechanism required a substantial refactor, which is covered by: - #3312 - #3317 - #3318 - #3325 - #3326
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This makes quite a significant refactor to the way we do error handling and validation when importing CSV files, which has no user-facing changes, but allows us to make further enhancements in the future.
Specifically, we can now link errors to specific columns and reference their row, column and cell (e.g.
A1) in the error messages. There's no design for this at the moment, but if we want to we could build this easily now.It also allows us to easily support different column names while making sure that the user is presented with the column name they uploaded, for example
NHS Numbervsnhs-number.There's definitely scope for further refactoring and enhancements, but this PR was already big enough:
:basefor each rowNHS_NUMBER(A1).