Suggested addition: add list of suggested data type terms and their definitions

[regnans]

Submitter: Kim Ely

I suggest the following changes: on the page https://github.com/ess-dive-community/essdive-file-level-metadata/blob/master/CSV_dd/CSV_dd_quick_guide.md#data-type

It would be really useful to have some defined terms here. Terms like "text" and "string", and "numeric" and "integer" (and other number types) are often confused and (incorrectly) used interchangeably. A defined list would be really useful. Some other types to consider could be logical, percent, fraction.

[regnans]

Some specific questions/examples:
What is the unit for a Date field? Is it the "format" of the date, i.e. YYYY-MM-DD, or is it the smallest unit, i.e. day? Or something else?
Should a numeric ratio have a unit or N/A?

[wavingtowaves]

@tvelliquette Wanted to bring you attention to this.
@regnans and Terri do you think it would be helpful to have these definitions within the metadata "element" called data-type or elsewhere?

[regnans]

I see the simplest solution would be to have this information within the "Standard definition" row of the Data type table. Or it could be added as another table immediately below. (I realize that this request is not a quick fix, but as a data contributor I would find it really useful, as I struggle with using data type terms consistently).

[kristinboye]

Looking through the currently uploaded (in various stages) reporting formats there is no consistency with e.g. date and time reporting format (even though each reporting format requires a specific way of reporting date and time...), ways of constructing "terminology files/data dictionary files" etc. So building on Kim's and others' previous comments, I think we need to harmonize the terminology, formatting (when appropriate), and requirements across the ESS-DIVE reporting formats to simplify for the average data producer/archiver.

[regnans]

An update on "data type" terms. A BNL we have settled (for now), on assigning the following data types in our data dictionaries: integer, floating point, string, date, time, date-time.

[wavingtowaves]

Sounds good, Kim. Thanks for posting the BNL updates to data types here.

…

On Mon, Mar 8, 2021 at 10:50 AM Kim Ely ***@***.***> wrote: An update on "data type" terms. A BNL we have settled (for now), on assigning the following data types in our data dictionaries: integer, floating point, string, date, time, date-time. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#6 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADTX7SNTREGXGE4IRGAG3WDTCUL7BANCNFSM4WPA4BYA> .

-- Rob Crystal-Ornelas, PhD Postdoctoral Scholar Lawrence Berkeley National Lab | ESS-DIVE Pronouns: he/him/his

[regnans]

To clarify, my post was an FYI, not necessarily how it should be done. Very happy to take advice from data scientists here! I have a lot to learn about the meanings and implications of using different sorts of data types.

Also, with our current usage of these terms we are not using "numeric" or "text". (Although perhaps "text" is appropriate for a field that included multiple sentences of text.)

[wavingtowaves]

@vchendrix As Terri and the ORNL team are finishing up the File-level metadata reporting format, Terri sent over a question about whether or not having a place for users of this format to report which data type they are providing in their data sheets would be useful.

Notes from Terri:

• Is [data type information] useful for machine-readability?
• It is used in other reporting formats.
• How do they vary and what is the scope?
• I would think this field would need a defined list.
• Maybe a change in the name to “Field Types”
• Resources with optional lists:
  1.  https://teachcomputerscience.com/database-data-types/
  2. https://www.progenygenetics.com/knowledgebase/index.php?/Knowledgebase/Article/View/21/22/database-field-types-and-database-fields

[regnans]

This issue is still relevant (see publication recommendations made for EDDOI-9773). Please include a list of standard Data_Type terms in the GitHub reporting format documentation.