Add the ability to create an arbitrary schema for rendering elsewhere

README.md

@@ -10,42 +10,45 @@ and display are supported.
 
 Table of contents:
 
-1. [Requirements](#requirements)
-2. [Installation](#installation)
-3. [Configuration](#configuration)
-   - [Schema Types](#schema-types)
-   - [Example Schemas](#example-schemas)
-   - [Common Schema Keys](#common-schema-keys)
-     - [`about_url`](#about_url)
-     - [`before_validators`, `after_validators`](#before_validators-after_validators)
-   - [Dataset Schema Keys](#dataset-schema-keys)
-     - [`dataset_type`](#dataset_type)
-     - [`dataset_fields`, `resource_fields`](#dataset_fields-resource_fields)
-     - [`draft_fields_required`](#draft_fields_required)
-   - [Group / Organization Schema Keys](#group--organization-schema-keys)
-     - [`group_type`](#group_type)
-     - [`organization_type`](#organization_type)
-     - [`fields`](#fields)
-   - [Field Keys](#field-keys)
-     - [`field_name`](#field_name)
-     - [`label`](#label)
-     - [`repeating_subfields`](#repeating_subfields)
-     - [`start_form_page`](#start_form_page)
-     - [`required`](#required)
-     - [`choices`](#choices)
-     - [`choices_helper`](#choices_helper)
-     - [`default`](#default)
-     - [`default_jinja2`](#default_jinja2)
-     - [`preset`](#preset)
-     - [`form_snippet`](#form_snippet)
-     - [`display_snippet`](#display_snippet)
-     - [`display_property`](#display_property)
-     - [`validators`](#validators)
-     - [`output_validators`](#output_validators)
-     - [`create_validators`](#create_validators)
-     - [`help_text`](#help_text)
-4. [Action API Endpoints](#action-api-endpoints)
-5. [Running the Tests](#running-the-tests)
+- [ckanext-scheming](#ckanext-scheming)
+- [Requirements](#requirements)
+- [Installation](#installation)
+- [Configuration](#configuration)
+  - [Schema Types](#schema-types)
+  - [Example Schemas](#example-schemas)
+  - [Common Schema Keys](#common-schema-keys)
+    - [`about_url`](#about_url)
+  - [Dataset Schema Keys](#dataset-schema-keys)
+    - [`dataset_type`](#dataset_type)
+    - [`dataset_fields`, `resource_fields`](#dataset_fields-resource_fields)
+    - [`before_validators`, `after_validators`](#before_validators-after_validators)
+    - [`draft_fields_required`](#draft_fields_required)
+  - [Group / Organization Schema Keys](#group--organization-schema-keys)
+    - [`group_type`](#group_type)
+    - [`organization_type`](#organization_type)
+    - [`fields`](#fields)
+  - [Arbitrary Schema Keys](#arbitrary-schema-keys)
+    - [`schema_id`](#schema_id)
+  - [Field Keys](#field-keys)
+    - [`field_name`](#field_name)
+    - [`label`](#label)
+    - [`repeating_subfields`](#repeating_subfields)
+    - [`start_form_page`](#start_form_page)
+    - [`required`](#required)
+    - [`choices`](#choices)
+    - [`choices_helper`](#choices_helper)
+    - [`default`](#default)
+    - [`default_jinja2`](#default_jinja2)
+    - [`preset`](#preset)
+    - [`form_snippet`](#form_snippet)
+    - [`display_snippet`](#display_snippet)
+    - [`display_property`](#display_property)
+    - [`validators`](#validators)
+    - [`output_validators`](#output_validators)
+    - [`create_validators`](#create_validators)
+    - [`help_text`](#help_text)
+- [Action API Endpoints](#action-api-endpoints)
+- [Running the Tests](#running-the-tests)
 
 
 
@@ -75,7 +78,7 @@ Set the schemas you want to use with configuration options:
 ```ini
 
 # Each of the plugins is optional depending on your use
-ckan.plugins = scheming_datasets scheming_groups scheming_organizations
+ckan.plugins = scheming_datasets scheming_groups scheming_organizations scheming_arbitrary
 
 #   module-path:file to schemas being used
 scheming.dataset_schemas = ckanext.spatialx:spatialx_schema.yaml
@@ -88,6 +91,7 @@ scheming.group_schemas = ckanext.scheming:group_with_bookface.json
                          ckanext.myplugin:/etc/ckan/default/group_with_custom_fields.json
 scheming.organization_schemas = ckanext.scheming:org_with_dept_id.json
                                 ckanext.myplugin:org_with_custom_fields.json
+scheming.arbitrary_schemas = ckanext.scheming:arbitrary_schema_example.yaml
 #
 #   URLs may also be used, e.g:
 #
@@ -103,6 +107,9 @@ scheming.dataset_fallback = false
 ## Schema Types
 With this plugin, you can customize the group, organization, and dataset entities in CKAN. Adding and enabling a schema will modify the forms used to update and create each entity, indicated by the respective `type` property at the root level. Such as `group_type`, `organization_type`, and `dataset_type`. Non-default types are supported properly as is indicated throughout the examples.
 
+Moreover, `scheming_arbitrary` enables the definition and rendering of a custom form without being tied to a particular entity type.
+The handling of a form submission must be implemented by the developer separately.
+
 
 ## Example Schemas
 
@@ -130,7 +137,9 @@ Organization schemas:
 * [Default organization schema with field modifications](ckanext/scheming/org_with_dept_id.json)
 * [Organization with custom type](ckanext/scheming/custom_org_with_address.json)
 
+Arbitrary schemas:
 
+* [Arbitrary schema example](ckanext/scheming/arbitrary_schema_example.yaml)
 
 ## Common Schema Keys
 
@@ -257,6 +266,16 @@ fields:
 A single `fields` list replaces the `dataset_fields` and `resource_fields` schema properties in dataset schemas.
 
 
+## Arbitrary Schema Keys
+
+It closely resembles the group/organization schema, with the exception of a single field - `schema_id`.
+
+### `schema_id`
+
+The `schema_id` field serves as a unique identifier for any arbitrary schema, which is utilized within the codebase for retrieving the schema.
+
+
+----------------
 
 ## Field Keys
 ### `field_name`

ckanext/scheming/arbitrary_schema_example.yaml

@@ -0,0 +1,27 @@
+scheming_version: 2
+schema_id: ckanext_notifier
+about: An example of a config schema for a fictional extension
+
+fields:
+  - field_name: ckanext.ckanext_notifier.enable_notifications
+    label: Enable notifications
+    validators: default(true) boolean_validator
+    preset: select
+    required: true
+    choices:
+      - value: true
+        label: Enable
+      - value: false
+        label: Disable
+
+  - field_name: ckanext.ckanext_notifier.notify_to_email
+    label: Notification email
+    validators: unicode_safe email_validator
+    required: true
+    help_text: Specify the email address to which the notification will be sent
+
+  - field_name: ckanext.ckanext_notifier.frequency
+    label: Notification frequency in seconds
+    validators: default(3600) int_validator
+    required: true
+    input_type: number

ckanext/scheming/helpers.py

@@ -443,6 +443,31 @@ def scheming_flatten_subfield(subfield, data):
     return flat
 
 
+@helper
+def scheming_arbitrary_schemas(expanded=True):
+    """
+    Return the dict of arbitrary schemas. Or if scheming_arbitrary
+    plugin is not loaded return None.
+    """
+    from ckanext.scheming.plugins import SchemingArbitraryPlugin as plugin
+
+    if plugin.instance:
+        if expanded:
+            return plugin.instance._expanded_schemas
+        return plugin.instance._schemas
+
+    return {}
+
+
+@helper
+def scheming_get_arbitrary_schema(schema_id, expanded=True):
+    """
+    Return the schema for the schema_id passed or None if
+    no schema is defined for that schema_id.
+    """
+    return scheming_arbitrary_schemas(expanded).get(schema_id)
+
+
 @helper
 def scheming_missing_required_fields(pages, data=None, package_id=None):
     if package_id:

ckanext/scheming/plugins.py

@@ -489,6 +489,17 @@ def get_actions(self):
                 logic.scheming_organization_schema_show,
         }
 
+class SchemingArbitraryPlugin(p.SingletonPlugin, _SchemingMixin):
+    p.implements(p.IConfigurer)
+
+    SCHEMA_OPTION = "scheming.arbitrary_schemas"
+    FALLBACK_OPTION = 'scheming.arbitrary_fallback'
+    SCHEMA_TYPE_FIELD = "schema_id"
+
+    @classmethod
+    def _store_instance(cls, self):
+        SchemingArbitraryPlugin.instance = self
+
 
 class SchemingNerfIndexPlugin(p.SingletonPlugin):
     """

ckanext/scheming/templates/scheming/group/group_form.html

@@ -20,13 +20,7 @@
     {% endblock %}
     {{ h.csrf_input() if 'csrf_input' in h }}
     {%- set schema = h.scheming_get_group_schema(group_type) -%}
-    {%- for field in schema['fields'] -%}
-        {%- if field.form_snippet is not none -%}
-          {%- snippet 'scheming/snippets/form_field.html',
-          field=field, data=data, errors=errors, licenses=licenses,
-          entity_type='group', object_type=group_type -%}
-        {%- endif -%}
-    {%- endfor -%}
+    {% snippet 'scheming/snippets/render_fields.html', fields=schema.fields, data=data, errors=errors, entity_type='group', object_type=group_type %}
 
     <div class="form-actions">
         {% block delete_button %}

ckanext/scheming/templates/scheming/organization/group_form.html

@@ -22,13 +22,7 @@
     {% endblock %}
     {{ h.csrf_input() if 'csrf_input' in h }}
     {%- set schema = h.scheming_get_organization_schema(group_type) -%}
-    {%- for field in schema['fields'] -%}
-        {%- if field.form_snippet is not none -%}
-          {%- snippet 'scheming/snippets/form_field.html',
-          field=field, data=data, errors=errors, licenses=licenses,
-          entity_type='organization', object_type=group_type -%}
-        {%- endif -%}
-    {%- endfor -%}
+    {% snippet 'scheming/snippets/render_fields.html', fields=schema.fields, data=data, errors=errors, entity_type='organization', object_type=group_type %}
 
     {{ form.required_message() }}
 

ckanext/scheming/templates/scheming/package/snippets/package_form.html

@@ -79,30 +79,8 @@
   {%- else -%}
     {%- set fields = schema.dataset_fields -%}
   {%- endif -%}
-  {%- for field in fields -%}
-    {%- if field.form_snippet is not none -%}
-      {%- if field.field_name not in data %}
-        {# Set the field default value before rendering but only if
-        it doesn't already exist in data which would mean the form
-        has been submitted. #}
-        {% if field.default_jinja2 %}
-          {% do data.__setitem__(
-            field.field_name,
-            h.scheming_render_from_string(field.default_jinja2)) %}
-        {% elif field.default %}
-          {% do data.__setitem__(field.field_name, field.default) %}
-        {% endif %}
-      {% endif -%}
-      {%- snippet 'scheming/snippets/form_field.html',
-        field=field,
-        data=data,
-        errors=errors,
-        licenses=c.licenses,
-        entity_type='dataset',
-        object_type=dataset_type
-      -%}
-    {%- endif -%}
-  {%- endfor -%}
+
+  {% snippet 'scheming/snippets/render_fields.html', fields=schema.dataset_fields, data=data, errors=errors, licenses=c.licenses, entity_type='dataset', object_type=dataset_type, set_fields_defaults=true %}
 
   {%- if pages -%}
     <input type="hidden" name="_ckan_phase" value="{{ active_page }}" />

ckanext/scheming/templates/scheming/package/snippets/resource_form.html

@@ -58,34 +58,7 @@
   {%- endif -%}
 
   {%- set schema = h.scheming_get_dataset_schema(dataset_type) -%}
-  {%- for field in schema.resource_fields -%}
-    {%- if field.form_snippet is not none -%}
-      {%- if field.field_name not in data %}
-        {# Set the field default value before rendering but only if
-        it doesn't already exist in data which would mean the form
-        has been submitted. #}
-        {% if field.default_jinja2 %}
-          {% do data.__setitem__(
-            field.field_name,
-            h.scheming_render_from_string(field.default_jinja2)) %}
-        {% elif field.default %}
-          {% do data.__setitem__(field.field_name, field.default) %}
-        {% endif %}
-      {% endif -%}
-      {# We pass pkg_name as the package_id because that's the only
-      variable available in this snippet #}
-      {%- snippet 'scheming/snippets/form_field.html',
-        field=field,
-        data=data,
-        errors=errors,
-        licenses=c.licenses,
-        entity_type='dataset',
-        object_type=dataset_type,
-        package_id=pkg_name
-      -%}
-    {%- endif -%}
-  {%- endfor -%}
-
+  {% snippet 'scheming/snippets/render_fields.html', fields=schema.resource_fields, data=data, errors=errors, entity_type='dataset', object_type=dataset_type, set_fields_defaults=true %}
 {% endblock %}
 
 

ckanext/scheming/templates/scheming/snippets/render_fields.html

@@ -0,0 +1,30 @@
+{#
+fields - a list of scheming field dictionaries
+data - form data fields
+errors - A dict of errors for the fields
+entity_type - entity type
+object_type - object type
+set_fields_defaults - flag to set the default field values
+{#}
+
+{% for field in fields if field.form_snippet is not none %}
+  {% if field.field_name not in data and set_fields_defaults %}
+    {# Set the field default value before rendering but only if
+    it doesn't already exist in data which would mean the form
+    has been submitted. #}
+    {% if field.default_jinja2 %}
+      {% do data.__setitem__(field.field_name, h.scheming_render_from_string(field.default_jinja2)) %}
+    {% elif field.default %}
+      {% do data.__setitem__(field.field_name, field.default) %}
+    {% endif %}
+  {% endif %}
+
+  {% snippet 'scheming/snippets/form_field.html',
+    field=field,
+    data=data,
+    errors=errors,
+    licenses=licenses,
+    entity_type=entity_type,
+    object_type=object_type
+  %}
+{% endfor %}

ckanext/scheming/tests/test_arbitrary_schema.py

@@ -0,0 +1,47 @@
+import pytest
+from flask import render_template
+from bs4 import BeautifulSoup
+
+import ckan.plugins.toolkit as tk
+
+
+class TestArbitrarySchema:
+    def test_arbitrary_schema_structure(self):
+        schema = tk.h.scheming_get_arbitrary_schema("ckanext_notifier")
+
+        assert schema["scheming_version"]
+        assert schema["schema_id"] == "ckanext_notifier"
+        assert schema["about"]
+        assert isinstance(schema["fields"], list)
+
+    @pytest.mark.usefixtures("with_request_context")
+    def test_render_arbitrary_schema(self, app):
+        schema = tk.h.scheming_get_arbitrary_schema("ckanext_notifier")
+
+        result = render_template(
+            "scheming/snippets/render_fields.html",
+            fields=schema["fields"],
+            data={},
+            errors={},
+        )
+
+        soup = BeautifulSoup(result)
+
+        assert len(soup.select("div.form-group")) == 3
+
+
+@pytest.mark.usefixtures("with_plugins")
+class TestGetArbitrarySchemaHelper:
+    def test_get_all_arbitrary_schemas(self):
+        assert tk.h.scheming_arbitrary_schemas()
+
+    @pytest.mark.ckan_config("scheming.arbitrary_schemas", "")
+    def test_get_all_arbitrary_schemas_if_none(self):
+        assert not tk.h.scheming_arbitrary_schemas()
+
+    def test_get_specific_schema(self):
+        assert tk.h.scheming_get_arbitrary_schema("ckanext_notifier")
+
+    @pytest.mark.ckan_config("scheming.arbitrary_schemas", "")
+    def test_get_specific_schema_if_none(self):
+        assert not tk.h.scheming_get_arbitrary_schema("ckanext_notifier")

setup.py

@@ -36,6 +36,7 @@
     scheming_datasets=ckanext.scheming.plugins:SchemingDatasetsPlugin
     scheming_groups=ckanext.scheming.plugins:SchemingGroupsPlugin
     scheming_organizations=ckanext.scheming.plugins:SchemingOrganizationsPlugin
+    scheming_arbitrary=ckanext.scheming.plugins:SchemingArbitraryPlugin
     scheming_nerf_index=ckanext.scheming.plugins:SchemingNerfIndexPlugin
     scheming_test_subclass=ckanext.scheming.tests.plugins:SchemingTestSubclass
     scheming_test_plugin=ckanext.scheming.tests.plugins:SchemingTestSchemaPlugin