Musings over a standard composition file format

[kneasle]

Why would a standard data format be useful?

There are many computer programs that all want to be able to handle compositions, and very few of these share an interchangeable data format. Therefore, currently transferring a composition between any of these programs requires the whole composition to be rewritten by hand, which is incredibly tedious and error prone.

Existing programs include:

• Generators of compositions: for example BYROC, Elf, SMC32 (can't find a link), Eril as well as much younger programs like Bellmetal
• Provers of compositions: for example Siril and all its implementations/derivatives, JSProve, PealProver
• Libraries of compositions: for example Composition Library, Don Morrison's ringing.org and even compositions attached to BellBoard performances
• 'Bot' Ringers for online and offline ringing programs: for example Abel, Wheatley for Ringing Room, Handbell Stadium and Bob for Ding (can't find a link)

There is also a 5th class of program that I believe doesn't exist yet, which my 3rd year project will consist of building...

• Visualisers of compositions: Programs which aid composers by providing instant truth checking, visualisations of music distribution, etc. but are not really designed with brute forcing in mind.

In some cases, we can piggy-back on other programs (e.g. Wheatley and Handbell Stadium use CompLib's row and call generation), but this is less than ideal for programs like composition generators, who's bread and butter is to be able to generate compositions as fast as possible. Therefore, calling out to CompLib's server is not probably not acceptable for both the program and CompLib's server.

My vision is that you should be able to be composing a composition with (for example) BYROC, and then when you are satisfied with the composition, you can immediately take the save file of that program and upload it directly to CompLib or import it directly into Abel or Wheatley to see how it feels to ring. All without ever having to transcribe a single bob or single.

Features of an ideal composition data format:

• Complete: Everything that is considered change ringing should be expressible in the format.
• Expressive: Simple compositions should produce simple and short files.
• Flexible: The format should be able to handle things like partial/incomplete compositions, mixed stage splicing, etc.
• Portable: It should be easy to use this format to send data between programs written in different languages. Ideally this should be built on top of an existing data format to aid portablity (e.g. JSON or XML).
• Extensible: Programs should be able to add their own metadata to the any part of the data without impacting its ability to be parsed by other programs.
• Readable: A computer-savvy human should be able to read, debug and modify files written in the format, though not necessarily write them from scratch.
• Specified: Pricking rows and calls from a file given in this format should be entirely specified according to an unambiguous specification - no single implementation should be considered canonical.

More details

Completeness

'Everything' is defined according to the Framework for Change Ringing.

Specification

Having said that no implementation should be canonical, it would be massively advantageous for the adoption of a such a data format if 'standard' libraries for parsing it are written in most major languages (preferably open sourced) and published on the standard package repositories for those languages (e.g. nuget for .NET, npm for JavaScript, pip for python, etc.). If this were done, then using this data format would hopefully be more convenient and compatible than making more 'roll-your-own' implementations.

This is I think a list of language families to cover:

• C, callable from C++, Rust, Python, etc.
• Any .NET language, callable from C#, F#, etc.
• Any JVM language, callable to Java, Scala, Kotlin, etc.
• JavaScript or any language compilable to WebAssembly for client-side web applications
• PHP?

MicroSiril

Currently the only format that could possibly be called standard is microSiril, and usually when I ask people whether a new standard is needed the common answer seems to be 'but we have microSiril'. However, microSiril satisfies only a few of the above features that I believe an ideal composition format should have. It is also a language rather than a data format - it is designed to be used by humans to represent a composition with the sole purpose of determining the truth of that composition and nothing else (and for this job it works excellently).

As a result, microSiril works very well for provers and is a passable output format for generators, but it is not remotely convenient for the other two use cases (libraries and bot ringers) - since given a microSiril file it is not possible to algorithmically determine a human-readable representation (as is needed for libraries), or to generate the locations and names of the calls (as is needed for bot ringers). microSiril is also unable to represent concepts like partial compositions, which I believe should be included in such a format.

I therefore believe that microSiril is not the answer to this data format problem, though it does embody many ideas that are worth taking inspiration from.