Vendor specific fields

[naveedn]

From the meeting on October 4, 2022:

• How can vendors (e.g. Shoobx, Carta, LTSE) provide vendor specific additional properties in a way that does not pollute the standard, but enables flexibility of adoption?
• If we assume that multiple vendors can operate on one file, how do we namespace vendor-fields?

Previous Agreements

• From @pjohnmeyer, we should limit the fields so there are no vendor fields

Discussion Points

• from @strichter, we should encourage flexibility and allow vendors to add additional properties to encourage adoption and sharing between vendors
• from @robwise, if vendors just stuff all the information into vendor properties, the standard loses its durability and consistency.

Potential Solutions if we decide to do vendor properties

• allowing namespacing of vendor fields using dynamic keys, which seems to be supported from jsonschema. Something could look like:

"vendor_carta": { ...carta properties },
"vendor_shareworks": { ...shareworks properties}

• we could have multiple properties such as vendor_name and vendor_properties, that could be either a singular value if there is 1 vendor, or an array of values for multiple vendors. The indexes would correlate between vendor_name and vendor properties.

{
  "vendor_name": ["shoobx", "carta"]
  "vendor_properties": [{..shoobx properties}, {..carta properties}]
}

• We could alternatively support a top-level vendor-properties object, in which each kv pair within would be a vendor name and additional properties.

"vendor_properties": {
    "carta": {},
    "shoobx": {},
     "ltse": {}
     "shareworks": {}
}

[naveedn]

If i missed anything or said something incorrectly, please correct me!

[strichter]

We can provide some historical examples here, where permitting the extension of a standard through custom/vendor data made the standard popular/usable:

1. XML/XHTML - The entire reason that XHTML took off in comparison to HTML 3 was because it allowed namespaced tags and attributes. That allowed tag- and property-based templating languages (Cold Fusion, TAL/METAL, etc.) to insert their dynamic evaluation. Note that HTML 5 came around to make syntax less strict but explicitly allowed custom tags for arbitrary extension.

2. OpenAPI/Swagger allowed for x- properties to be added that act as vendor-specific extensions allowing for insertion of additional metadata.

3. CSS allows for custom styles that led to the (in)famous -<vendor>- rules. Eventually, many vendor-specific rules were standardized. I consider this a big success from a standards-development point of view.

[strichter]

I think we could also provide a concrete example where vendor-specific extensions might be useful. I know that all companies involved in the technical working group are working on secondaries. While OCF does not support secondaries during execution, vendors might want to communicate this information. Would it be better for those vendors to come up with a separate serialization format or reuse OCF with some extensions?

[pjohnmeyer]

Thanks for recording these notes @naveedn.

I'm still not convinced we want or need this capability, because I feel it encourages us to continue operating in a world of everybody supporting N data models instead of 1 data model. I do see the point about adoption, but the tension @robwise mentioned is strong -- once people come to rely on vendor-specific properties, pretty much nobody is going to want to stop and do maintenance to pull those into the standard and deprecate them.

IF we are to allow embed vendor properties on objects, I like your last option best (support a top-level vendor-properties object on each OCF Object). I would suggest two small tweaks to your example: Add a "wart" to the key as a visual reminder that it's not a part of the schema proper, and use domain names for vendor keys. e.g.:

"_vendor": {
  "carta.com": {  },
  "ltse.com": {  }
}

I have another suggestion that I think is less harmful to the schema, and that is to add a new File to the package. This file would contain only vendor specific data, and would use object references to do so. For example, let's say Example, Inc wants to include more data about the VALUATION object in https://github.com/Open-Cap-Table-Coalition/Open-Cap-Format-OCF/blob/616d972f4ce99f33ba2729203e56ea5c0acb2017/samples/Valuations.ocf.json. They could do so by emitting an OCF_VENDOR_FILE that looks something like this:

{
  "file_type": "OCF_VENDOR_FILE",
  "vendor": "example.com",
  "items": [
    {
      "id": "892e8b19-ed26-4049-9ada-a5cf816376fd",
      "object_type": "VALUATION"
      "phase_of_moon": "WAXING_GIBBOUS",
      "the_answer": 42
    }
  ]
}

The benefits here are:

• a consumer of the package can completely ignore this file entirely at parse time if only interested in "pure" OCF data
• the vendor_properties or _vendor keys do not make their way into every object in the schema
• the vendor could more easily create their own JSON-schema to define their vendor-specific fields without having to extend OCF object types

Thoughts?

[strichter]

While the file suggestion is less invasive, it is also much more onerous. Just imagine if XML would have required a separate file for every namespace it allowed. Nobody would use XML namespaces or XML for that matter.

[pjohnmeyer]

I don't think XML is a good analog here, as XML is general purpose, whereas OCF is not. HL7 might be a more interesting comparison, especially since it has extensibility in its schema? (I'm not particularly familiar with HL7, just know of it.)

I'd like to understand more about your opinion that the file approach would be more onerous; my opinion at the moment is a weakly held one.

[strichter]

I personally like the last option "vendor_properties" or just short "vendor". I would not specify anything for the key other than it has to be a string.

[JSv4]

What about something like a vendor_data field that can hold a list of custom metadata objects like this on the Object (so it's composed into final objects)? Like this:

{
    "type": "TOTES_A_REAL_OBJ",
    ...,
    "vendor_data": [
        {
            "vendor": "The Best Cap Platform Ever",
            "data": { ... any dict }
        }
    ]
}

I think this is similar to one of the options @naveedn proposed above (thanks for the great write-up, @naveedn)

[JSv4]

Per TWG Discussion 10/18/22:

• We do see the value of this. It has lots of hypothetical use-cases, but no one has an immediate use-case for it.
• Let's revisit once we have a concrete application for this field (and if someone has one now, speak up ;-) )