Skip to content

Config setting to add custom composite separator #295

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Config setting to add custom composite separator #295

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
7cd401d
Merge pull request #1 from ckan/master
fostermh Apr 28, 2021
c49e8eb
fostermh Aug 20, 2021
3c7bb74
fostermh Aug 25, 2021
833ad64
fostermh Nov 1, 2021
66975d2
fostermh Nov 1, 2021
4d69945
make subfield multiple_text work again. Document new option
ccancellieri Nov 1, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion ckanext/scheming/helpers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ def lang():
from ckantoolkit import h
return h.lang()

@helper
def scheming_composite_separator():
return config.get('scheming.composite.separator','|')
Copy link
Contributor

@ccancellieri ccancellieri Nov 1, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably it would be better to keep this setting as the default '-' and override if needed.
Could this affect existing schemas? If so we should move this to the scheming yaml / json instead providing a per type setting. What do you think?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point about the default separator character. I have made that change. I don't think this will affect existing schemas, now that it is using the correct default separator char. A per field override is an interesting idea though. It would be a bit of work as everywhere the separator is used we would also need to check which field we are handling and check the schema. So far, my experience is that a global setting is meeting all our needs in cioos.


@helper
def scheming_language_text(text, prefer_lang=None):
Expand Down Expand Up @@ -421,14 +424,16 @@ def scheming_flatten_subfield(subfield, data):
after a validation error) then they are returned as-is.
"""
flat = dict(data)
sep = toolkit.h.scheming_composite_separator()

if subfield['field_name'] not in data:
return flat

for i, record in enumerate(data[subfield['field_name']]):
prefix = '{field_name}-{index}-'.format(
prefix = '{field_name}{sep}{index}{sep}'.format(
field_name=subfield['field_name'],
index=i,
sep=sep,
)
for k in record:
flat[prefix + k] = record[k]
Expand Down
9 changes: 5 additions & 4 deletions ckanext/scheming/plugins.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -331,15 +331,16 @@ def expand_form_composite(data, fieldnames):
when submitting dataset/resource form composite fields look like
"field-0-subfield..." convert these to lists of dicts
"""
sep = p.toolkit.h.scheming_composite_separator()
# if "field" exists, don't look for "field-0-subfield"
fieldnames -= set(data)
if not fieldnames:
return
indexes = {}
for key in sorted(data):
if '-' not in key:
if sep not in key:
continue
parts = key.split('-')
parts = key.split(sep)
if parts[0] not in fieldnames:
continue
if parts[1] not in indexes:
Expand All @@ -348,11 +349,11 @@ def expand_form_composite(data, fieldnames):
parts[1] = indexes[parts[1]]
try:
try:
comp[int(parts[1])]['-'.join(parts[2:])] = data[key]
comp[int(parts[1])][sep.join(parts[2:])] = data[key]
del data[key]
except IndexError:
comp.append({})
comp[int(parts[1])]['-'.join(parts[2:])] = data[key]
comp[int(parts[1])][sep.join(parts[2:])] = data[key]
del data[key]
except (IndexError, ValueError):
pass # best-effort only
Expand Down
2 changes: 1 addition & 1 deletion ckanext/scheming/templates/scheming/form_snippets/repeating_subfields.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
{% for subfield in field.repeating_subfields %}
{% set sf = dict(
subfield,
field_name=field.field_name ~ '-' ~ index ~ '-' ~ subfield.field_name)
field_name=field.field_name ~ h.scheming_composite_separator() ~ index ~ h.scheming_composite_separator() ~ subfield.field_name)
%}
{%- snippet 'scheming/snippets/form_field.html',
field=sf,
Expand Down