Footnotes and Endnotes

[coemgenuslcp]

I feel like I had already posted on this last year, but maybe I just had never hit the "Start discussion" button.

How to implement footnotes, today and in the longterm? The nicest way to the user, i.e., the longterm, might be like below,

Here is a sentence with a footnote[footnote
    Here is a footnote.
] and an endnote[endnote
    Here is an endnote.
].

where the renderer places the footnotes at the bottom of the section or of the document, or of the page when in print view. It would handle the anchors and numbering automatically (perhaps with a config override for numbering and format options — Bible verse numbers and numbered footnotes don't go well together without adjusting one or the other visually.)

Shortterm — is that kind of dynamicity even possible yet? A UDN is simple enough to make for the content and anchors, and holding onto variables during the conversion would enable number incrementing, but I don't see how to render the footnote div in another place (bottom of the section/page) than where its definition lies in the code.

[tajmone]

Footnotes are going to be an extremely important key feature for PML, so I'm really looking forward for it. I believe it should be a natively supported feature.

As for how they could be implemented, I'd like to share some thoughts about it and look at how other markup syntaxes went about it.

• AsciiDoc Language Documentation » Footnotes
• Pandoc Manual » Footnotes

For the sake of simplicity I'll be using footnotes to indicate both footnotes and endnotes, unless otherwise specified.

General Considerations

• Wikipedia » Note (typography)

When we speak of "footnotes" we are really speaking of different types of book-notes which might coexist within a same publications:

• Footnotes — shown at the bottom of the page.
• Endnotes — shown at the end of the chapter or book.
• Editor notes — additional notes added by the editor, which were not part of the original publication.

In good academic publication you can often find all the above three types being used at once. In order to distinguish between them, independent number systems are adopted:

• Arabic numbers (1, 2, 3, 4, 5, etc.)
• Roman numbers (i, ii, iii, iv, v, etc.)
• Typographic Symbols (*, †, ‡, §, ‖, ¶, **, ††, ‡‡, §§, ‖‖, ¶¶, etc.) — in cycling order, with an extra copy of the symbol added at each cycle iteration.

Ideally, PML should be able to support all three of the above, which will make it a champion syntax in terms of being up to editorial standards, especially in terms of reprinting books, where it's important to distinguish between editor notes and original notes by the author.

Although its rare to find both footnotes and endnotes within a publication, it's not unheard of. When both are used it's usually the case that the former are employed for short notes which the reader might want to consult immediately, without interrupting the main reading, whereas the latter are longer notes, placed at the chapter/book end for post-reading research.

I haven't yet seen any markup syntax support multiple notes types, so in order to support all of them we'd need to come up with some clever syntax device to distinguish between them and yet keep the notation concise.

If supporting all three is impossible (or impractical), we should at least support two of them:

1. Footnotes/Endnotes
2. Editor notes

In that case, users should be able to control (1) via some setting that determines where the notes will be shown (end of page, chapter, or book), and to control which numerals to use for both (1) and (2).

Footnotes Syntax

Pandoc supports both in-line footnotes (i.e. the footnote text is defined within the flow of the main text) or via labels and a dedicated definition block. AsciiDoc only supports in-line definitions, but footnote definition-blocks can be simulated by externalizing footnotes.

They also allow arbitrary footnote IDs, since the IDs will be replaced with ordered numbers at conversion time. This is practical since it allows footnotes to have easy to remember IDs, instead of having to look up their number. Also, it spare having to deal with renumbering the footnotes when a new footnote is added in the middle of the sequence.

Pandoc Footnotes Examples

• Pandoc Manual » Footnotes

Here is a footnote reference,[^1] and another.[^longnote]

[^1]: Here is the footnote.

[^longnote]: Here's one with multiple blocks.

and this is an inline-footnote:

Here is an inline note.^[Inline notes are easier to write, since
you don't have to pick an identifier and move down to type the
note.]

AsciiDoc Footnotes Examples

• AsciiDoc Language Documentation » Footnotes

The text of the footnote is defined between the square brackets of the footnote macro (footnote:[text]). The footnote macro accepts an optional ID using the target of the macro (footnote:id[text]). Specifying an ID allows you to refer to that same footnote from multiple locations in the document.

The hail-and-rainbow protocol can be initiated at five levels:

. doublefootnote:[The double hail-and-rainbow level makes my toes tingle.]
. tertiary
. supernumerary
. supermassive
. apocalyptic

A bold statement!footnote:disclaimer[Opinions are my own.]

Another outrageous statement.footnote:disclaimer[]

Unfortunately in AsciiDoc to prevent spaces between a word and the footnote reference you have to write the word and the footnote macro side-by-side, which looks odd without a syntax highlighter: doublefootnote:[ = double+footnote:[.

Externalizing Footnotes

• AsciiDoc Language Documentation » Externalizing a footnote

Since footnotes are defined using an inline macro, the footnote content must be inserted alongside the text it’s annotating. This requirement can make the text harder to read. You can solve this problem by externalizing your footnotes using document attributes.

AsciiDoc requires this hack to achieve what in pandoc is a natively supported feature. Document attributes have some limitations which require further hacks to enable formatting within externalized footnotes, due to how the preprocessor works, which makes this approach not so practical, especially for those who don't know AsciiDoc very well.

:fn-hail-and-rainbow: footnote:[The double hail-and-rainbow level makes my toes tingle.]
:fn-disclaimer: footnote:disclaimer[Opinions are my own.]

The hail-and-rainbow protocol can be initiated at five levels:

. double{fn-hail-and-rainbow}
. tertiary
. supernumerary
. supermassive
. apocalyptic

A bold statement!{fn-disclaimer}

Another outrageous statement.{fn-disclaimer}

The pandoc approach and its syntax is definitely much better than the AsciiDoc way of handling it.

Footnotes Reusability

Can a same footnote be referenced more than once within the text? and if yes, how can the backlink from the footnote to its reference within the text be handled?

Usually, when footnotes are rendered in hypertext documents (HTML, PDF, etc.), the in-text footnote reference is rendered as a link pointing to the footnotes contents, and within the footnote the note number is a backlink to text where the reference is placed — but this only works if each footnote "is unique", i.e. there's a single reference for each footnote, which is why in most markup languages you can't have multiple references to a same footnote.

It's quite common in real life to have a same footnote referenced multiple times in the text, e.g. a link to a specific article being mentioned in various places in a book. Ideally, one shouldn't have to create duplicate footnote entries which are identical.

In good printed books a same footnote is reference multiple times, instead of having to create duplicates of the same footnote, but since the digital publishing era we're now seeing more and more publishers take the approach of creating redundant footnotes only because of editing software limitations.

But I've seen examples of HTML documentation where a same footnotes is reused multiple times, and the way backlinks were handled is that within the footnote contents, next to the note number (which backlinks to the position of the first reference occurrence), additional backlinks are added in an ordered number fashion, like this:

[37][2][3]. Footnote contents text. Lorem ipsum dolor sit amet, consectetur
            adipisicing elit, sed do eiusmod tempor incididunt ut labore et
            dolore magna aliqua.

where [2][3] are backlinks to the 2nd and 3rd occurrences, etc., and usually styled in superscript or some other styling to distinguish from [37] which is the actual footnote number.

[coemgenuslcp]

Wow, some of those macro usages are downright ugly!

Ideally, PML should be able to support all three of the above. If supporting all three is impossible (or impractical), we should at least support two of them.

Correct me if I'm missing something, but it seems obvious to me the anchor ID should always be decoupled from the visible number (or letter — I would add alphabetic (a, b, ... y, z, aa, bb, ...) to the options as well).

Otherwise, the visible numbers/letters are then hard-coded strings, susceptible to being in no particular order, since the anchor IDs are self-standing unique strings with no concern for their relationship besides their uniqueness. I cannot think of a case where footnotes would ever be referenced with individually-specified characters instead of the aforesaid norm of one or more character sequences — numeric, Roman numeric, alphabetic, Lotharingian majuscule, Gothic runes, emojis, etc.

[tajmone]

Wow, some of those macro usages are downright ugly!

Indeed. AsciiDoc macros don't look sweet at all, and also their use tends to be complicated by the fact that you need to be aware of the pre-processor and processor order in which they are handled, and when substitutions occur — if you get the processing priority wrong you might get unexpected results. Many macros have also a syntactic sugar equivalent, which makes them look nicer in the source, but footnotes don't have one.

The pandoc approach looks cleaner to me and doesn't require any brainstorming about pre- or post-processor ordering. The problem is how to translate that to the PML way of representing elements as square-brackets delimited nodes. Furthermore, supporting multiple note-types requires some extra thoughts.

For the in-line occurrence of the footnote reference we'll need an inline node, whereas for the footnote contents definition we'll most likely need a block node, so it allows multi-paragraph contents and using block nodes inside footnotes. Probably something like this:

Just a paragraph.[fn:1]

[fn:1
My footnotes text. Styles [b allowed]! And even block-nodes:

    [quote (source="Albert Einstein, physicist")
    Two things are infinite: the universe and human stupidity; and I'm not sure about the universe.
    ]
]

Since footnotes never appear at the beginning of a paragraph, and because the in-line note reference can only contain the note-ID and have form [fn:<id>], PML should be able to distinguish between the inline-node(s) for references and the block-node for their definitions. But I'm not sure if this plays well with the design of PML nodes, since you'd end up with an inline node and a block node having almost identical syntaxes.

Maybe to distinguish between the two we could substitute the : with another symbol in one of the two (e.g. [fn 1] vs [fn:1]).

As for multi-note types support, I thought that fn could stand for footnote, en for endnote, and pn for publisher/editor note, which would keep the syntax for all three types similar and easy to remember.

Correct me if I'm missing something, but it seems obvious to me the anchor ID should always be decoupled from the visible number (or letter —

Yes. The custom ID is just an arbitrary reference. It could be a number or letter (1, 1b, i, etc.) or a mnemonic of the note (e.g. license-note) for easy of use and reuse. The main point is that authors don't have to worry about fixing the notes ordering number in the source when they add intermediate notes, since in the output they will be numbered according to appearance order.

So, let's say I assign footnotes ordered number IDs in the source, and then need to add a new note between notes 3 and 4. Instead of renumbering all the notes from 4 upward, I could just give the new note the ID 3b — i.e. if having ordered notes in the source was important for some reason. A good editor should provide functionality to rename all footnotes according to their order of appearance, which will make their IDs match those in the final output.

But often it's useful to give footnote mnemonic IDs, especially when the footnote contents are defined in an external file. Some people like to keep the footnotes definitions next to the paragraphs in which they occur, while others prefer to externalize them all in a dedicated source file; neither method is better than the other, it's up to one's personal preferences.

I would add alphabetic (a, b, ... y, z, aa, bb, ...) to the options as well).

In that case, we'll need to expose project settings to override the default "numerals" (symbols, etc.) of each notes-type. Some markup languages use worded-settings (e.g. arabic, romans, etc.) while others just require a sample of the desired ordering symbols (e.g. 1, 2, 3, i, ii, iii, a, b, c, A, B, C, *, †, ‡). IMO, a full-worded setting is more practical, because typing-in typographic symbols like † and ‡ requires alt-codes (on Windows) or some other keyboard acrobatics or external tools, since these are not present on most keyboards. Furthermore, these symbols might not be supported by many monospace fonts, and might be displayed as gibberish in the shell and output of various CLI tools.

[pml-lang]

Thanks a lot for the suggestions and useful info, @coemgenuslcp and @tajmone.

I will have a look at this and post a suggestion in the coming days.

[pml-lang]

First Draft

Here are my first draft thoughts on how to add footnotes in PML.
These are just incomplete and non-final ideas to be discussed.
Source code examples might contain minor syntax errors, because I didn't test them.

Footnotes need to be natively supported in PML, because they need special handling that cannot be achieved with scripting or user-defined-nodes.

Node name

A first obvious idea would be to add nodes footnote, endnote and editornote to cover those three types of notes.
However, later we might want to add more types of notes like authornote, publishernote, etc.
To avoid a proliferation of standard PML nodes I suggest to use the same node name for all types of notes, and differentiate them by an attribute, as explained later.
We can't use note, because that name is taken already in PML.
I suggest to use lnote (the l standing for linked and/or located, as it is a linked note at a specific location)

Node lnote

We'll add a new inline node with name lnote.
lnote can appear anywhere an inline node is allowed.
lnote contains text.

Example:

doc text [lnote text of footnote] more doc text 

Standard whitespace handling rules apply. Hence the above code can also be written like this:

doc text [lnote
    text of footnote
] more doc text 

lnote can contain inline nodes to format/style linked notes, e.g.:

[lnote text of footnote with [b bold] and [i italic] words.]

Automatic Note Identifier

All notes are numbered automatically, starting at 1 (NOT 0!).

Rendering (to be discussed)

By default, a note is rendered as in Wikipedia.

In the target document, the note is a link to the text of the note later in the document.
The link is surrounded by a pair of square brackets.
The link's text is the note's number.
The link is styled with superscript.

Example:

text ^[7]

All notes are displayed as a list at the end of the chapter, document, etc. as in this Wikipedia example, using the ^ character in the backlink.

Note References

The same note can be used several times in the text.
In that case, the user must assign a nid attribute (not id because that attribute is used already to identify block nodes) in the first occurrence of the note, e.g.:

[lnote (nid=my_id) note text]

Then the same note can be used again and again in the text by using an lnote_ref with a mandatory id attribute:

[lnote_ref (nid=my_id)]

Note that user-defined ids and incremental numbers assigned by PML are different properties.
A note with a user-defined id still has an automatically assigned number.

The list of all backlinks could be rendered as suggested by @tajmone:

the way backlinks were handled is that within the footnote contents, next to the note number (which backlinks to the position of the first reference occurrence), additional backlinks are added in an ordered number fashion, like this:

[37][2][3]. Footnote contents text. Lorem ipsum dolor sit amet, consectetur
            adipisicing elit, sed do eiusmod tempor incididunt ut labore et
            dolore magna aliqua.

If we use the Wikipedia-style ^ for the main backlink then [37][2][3] in the above example would be displayed as ^ [2][3].

Note Types

The type of note is specified by attribute type.
Default value is end for HTML documents.
An editornote would be inserted with the following syntax:

[lnote (type=editor) note text]

We need to define the list of note types supported (e.g. foot, end, editor, etc.)

Note Locations

Attribute loc is used to specify where the note's content is displayed.
Possible values could be: page, chapter, end, inline.
The default could be end, or depend on the type and target media (e.g. page for printed documents, end for HTML documents, etc.)
To specify that a note should appear at the end of a chapter we would write:

[lnote (loc=chapter) note text]

If a note is displayed at the end of a chapter, the user might want to specify in which chapter the note is to be displayed (e.g. the chapter in which the note is defined (default), a chapter with a specified level, or a specified id)

Maybe we should also add attribute hidden (default value = no), to allow temporarily or conditionally hiding of one or more notes.

Note Numbering

Notes displayed in the list of notes are numbered automatically.
By default arabic numbers (1, 2, 3, ...) are used.
We could provide options to support other numbering styles such as alphabetic (a, b, c, ...) roman numbers (i, ii, iii, ...) or typographic symbols.

Note Definitions

Notes can be defined inline (as shown in the previous examples), or they can be defined together in a block at the start of the document.
This can be done with u:set / u:get nodes.
All notes are defined at the start of the document, like this:

[u:set
    fn_1="[lnote text of footnote 1]"
    fn_2="[lnote text of footnote 2]"
]

Then they can be inserted in the text as follows:

doc text [u:get fn_1] doc text [u:get fn_2] doc text

A better approach would be to have a dedicated PML node, named lnote_def to define note content.
This would also enable users to use block nodes for the content of their notes.

Here is an example of defining a note with lnote_def, and then using it in the document:

[note_def (nid=my_id)
    text
    [image source=violin.png]
    [table
        ...
    ]
]

...

[lnote_ref (nid=my_id)]

In both cases (using u:set/get or lnote_def) the notes can also be imported from one or more files instead of being embedded in the document, e.g.

[u:ins_file path=../../shared/footnotes/common_footnotes.pml]
[u:ins_file path=config/footnotes/specific_footnotes.pml]

Customized, Succinct Syntax

By default all notes are endnotes displayed at the end of the target HTML document (or at the bottom of the page for printed documents).
The syntax is simple for these notes:

[lnote note text]

However, suppose a user wants to insert only editor notes displayed at the end of the chapter.
He/she would have to write:

[lnote (type=editor, loc=chapter) note text]

... which is verbose, error-prone and frustrating.

There are at least three solutions.
The first one exists already, the other two should be added in a future version, because they are very useful in other contexts too, not just for footnotes.

Scripting

A user could define the following script in the document (or import it from an external file):

[s:def
    function ed_note ( content ) {
        return `[lnote (type=editor, loc=chapter) ${content}]`
    }
]

Then an editornote can be inserted as follows:

doc text [s:exp ed_note("note text")] doc text

This is less verbose, but not ideal.

Overriding Default Values For Attributes (Idea)

The options node at the beginning of the document could be used to explicitly specify default values for attributes, e.g.

[doc
    options [
        attribute_defaults [
            lnote [
                [type editor]
                [loc chapter]
            ]
        ]
    ]        
    ...
]

As explained in chapter Options of the PML User Manual, options can be defined in the PML document, in a shared config file, or on the command line.
Hence, this would allow very flexible customization of default attribute values (adding default values or overriding hard-coded PML default values).
For example, the above options could also be defined in the CLI, and (as an example) we could also specify that all images should have a border as follows:

pmlc article.pml ^
    --attribute_defaults.lnote.type editor ^
    --attribute_defaults.lnote.loc chapter ^
    --attribute_defaults.image.border yes

With these default values, the code

[lnote (type=editor, loc=chapter) note text]

... can then simple be written like this:

[lnote note text]

While appropriate in some cases, the above method has two inconveniences:

• It works only if all notes in the document are editornotes. Other notes need to use attributes (including standard endnotes).

• A user who is not aware of the redefinition of attribute default values interprets [lnote note text] as a standard endnote, while it is actually an editornote.

Function Nodes (Idea)

Here is another idea I've been thinking about since some time, and I would really really like to implement it:

Consider the following Javascript function definition in a PML/PDML document:

[s:def
    function foo ( i1, i2 ) {
        // body
    }
]

This function could do anything useful, such as inserting text into the PML document, writing to a file, calling an external program and inserting its output to STDOUT into the PML document, etc.
We would call this function in a document as follows:

[s:script foo ( "value 1", "value 2" )]

If Javascript had named parameters, we would write:

[s:script foo ( i1 = "value 1", i2 = "value 2" )]

If you look at the syntax for the function call (foo ( i1 = "value 1", i2 = "value 2" )), it's easy to see that this syntax is exactly the same as the syntax for a PDML/PML node with attributes, except for the comma separator.
If we add a dedicated namespace (for example f for function call), the PDML parser could interpret the following standard PDML/PML code as a function call:

[f:foo ( i1 = "value 1" i2 = "value 2" )]

Lenient attribute parsing can be applied, because the node has only attributes, no other content:

[f:foo a1 = "value 1" a2 = "value 2"]

If PDML/PML later supports positional attributes (already implemented in the base library, but not yet supported in PDML), we could write:

[f:foo "value 1" "value 2"]

If a function has only one input argument we could even allow this:

[f:foo value 1]

Alternatively we could say that if an f node has text content, then this text is assigned to the last input argument of the called function. We could even say that f nodes are of type raw_text, so that the three syntax variations of raw_text could be used for the last input argument. This could be useful, for example, to define JS source code for a function that takes a callback function as last argument.

Now image we defined the following function:

[s:def
    function ed_note ( text ) {
        doc.insertText ( `[lnote (type=editor, loc=chapter) ${content}]` )
    }
]

Then the user could insert an editor node as follows:

doc text [f:ed_note text of note] doc text

This is what the user really wants!

As demonstrated by this example, function nodes could be used to define user-defined-nodes that inject PML code into the document.
The namespace f nicely separates such nodes from standard PML nodes, so that function nodes can quickly be recognized, searched for, etc.
Moreover the namespace eliminates the risk for name clashes with future native PML nodes.

Function nodes would of course be part of PDML extensions, so that they could not only be used in PML - they could be used in all PDML documents that support (optional) extensions.

@coemgenuslcp put it like this in a response to a great comment by @tajmone:

developers could derive their own one-use or reusable 'dialects' of PML, whether with mere syntactic candies like [bi or with a full domain-specific vocabulary of custom elements.

Implementation

I would suggest to start with a basic implementation supporting:

• node lnote without attributes
• automatic note identifier (1, 2 ,3 ...)
• rendering as described above

The following features could be added later, depending on user feedback:

• Note references (node note_ref)
• Types
• Locations
• Customized numbering
• Note definitions (node note_def).

I also suggest to implement Function Nodes in the near future, because they are very powerful and useful in other contexts.

More Options

There are more knobs we could add to make notes even more super-flexible and super-powerful.
However, we must also be careful to always KISS and not forget YAGNI.

What are your thoughts?

[tajmone]

On Syntax Strictness

this is not your typical PML notation

True! And that's a big issue.

I'm starting to realize that with its evolution course PML has now embraced the idea of having a standard/formalized way of parsing the syntax. With the very early versions of PML this didn't seem an issue, then came pXML, which later evolved into PDML, which know is at the core of PML.

Originally PML was about a practical markup language, and pXML about a practical XML representation, but since PDML the two are now double bind to each other, although they originally served two different goals.

Most likely, the notion of having to stick to a parsing formal syntax will affect the "practical" side of PML on the long run. While the use of square brackets and tags to represent nodes in a brief way works great for XML-like constructs, some aspects of lightweight markup syntaxes have historically been bound to dedicated syntax notations in the name of practicality. E.g. lists, which are usually represented either via * or - for bullet entries, or via numerals followed by a dot for ordered lists (context being they must be at the beginning of the line); whereas the PML approach becomes more verbose in this case (and lists are a highly demanded feature).

As far as I know, none of the well known lightweight markup languages can be parsed entirely via a formal system — e.g. even pandoc, which uses PEG for Markdown, had to rely on custom code to handle many edge cases. I believe this is the natural result of trying to simplify the complexity of documents formatting via a simple and human-friendly notation based on ASCII characters — i.e. you have to rely on context and sets of rules to cover all needs.

But my main argument here would be that there's a need to abstract away formatting concepts via clever solutions which are (1) easy to type, (2) easy to read, and possibly (3) resemble the desired final typographic effect.

If we start adding special (non-standard) syntax to keep the code as short as possible, we'll face serious complications and disadvantages, such as:

• [...]
• PML becomes more difficult to learn and explain.

I doubt that, especially if we keep close to the notations used by other markup syntaxes; on the other hand, the risk here is that end users will have to memorize tons of attributes and their values (some of them are not mnemonic nor intuitive either, like lnote).

PML doesn't need to relinquish the PDML parsing system, it only needs to add an abstraction layer that allows briefer representations of some elements, like an extra scanner that can convert some of these constructs to the PDML AST. Sure, it's extra work and complications, but that's true for anything that makes formatting more human friendly.

IMO doc notes must only use standard PDML syntax.
We need to agree on the list of nodes used for doc notes, and for each node we must define:

• the node name
• the list of attributes (name, allowed values, and sensible default value)

Of course, this will lead to a more verbose syntax, and power-users will not like that.

I don't think the problem is the power-users — on the contrary, more likely the average users, i.e. those who need to pick a new markup syntax. Personally, when I look at new syntaxes, the first thing I check is how lists and tables are represented, since these are among the most common elements, and at the same time those which need to be kept tidy and human readable in the source text. Authors basically want something that is not straining on the eye to work with (like XML is, with all its </>s), and as close to a WYSIWYG experience — if you look at the history of new features in markup syntaxes you can clearly notice that trend.

Attributes are a real problem, they basically can nullify all the benefits of the the shorter tags notation, and they make the source text less readable, and more similar to programming source code.

I'm wondering if it could be possible to add an extra dot notation, similar to namespaces but to indicate node subtypes. E.g. [adm.note, [adm.warn, etc. to distinguish between the different type of admonition blocks (allowing just [adm == [adm.note). This by itself would be a much shorter solution than having to use a key-value pair like [adm (type=note).

In any case, I'm getting the feeling that there's a need for some new notation to counter the growing verbosity. In PML v1 parameters could be displayed via the <<pname>> notation, now we need [u:get:pname] which is longer and makes the text less readable — the original notation testifies that the intuitive ways rarely stick to formal ways.

When it comes to shorthands, special symbols are always briefer than named entities, and also have the benefit of contrasting with text contents, which means less interference.

Therefore, a crucial PML feature is to allow power-users to easily define their own, less verbose, customized syntax for their PML documents.

I'm aware of that, but here we are talking of core editing features, which I believe should be standardized. Custom solutions come at the price of proliferation of alternative ways of achieving a same thing, and might also incur in maintenance problems, so it's also a matter of guarantees.

Editing Work and Needs

... function nodes would allow a user (for example) to replace:

In the past I used to work with pandoc + custom macros for my publishing work, but then switched to AsciiDoc. Working with custom nodes is similar ot using pandoc + custom macros, in many ways, except that pandoc markdown has a highly abstracted syntax which is very close to WYSIWYG (only the macros would add verbosity usually, although in some uses they would reduce it).

The reason I ended up switching to AsciiDoc was twofold:

1. Native features — AsciiDoc tables allowing custom spans; admonitions; and others.
2. Custom macros affected portability across multiple formats, and hindered integrating different projects together.

When I speak of "working with" either syntax what I mean is daily work and tasks like reprinting a 600 pages book from scratch (usually OCR of an old paper print, or plain-text exported from a PDF) in two days. The work would consist in going over the unformatted raw text and quickly rebuilding the lost styles (formatting, sections structures, lists, tables, footnotes, everything).

In such big projects you start to feel the burden of verbose syntaxes, especially if the syntax lacks good editor support (like AsciiDoc, were syntax highlighting usually broke after a dozen paragraphs), and in such projects it all boils down to time when it comes to productivity. Working with pandoc was much faster, not only when applying formatting, but especially when proofreading, since the syntax is very light and doesn't stand much in the way of text (but the macros did).

Some projects simply needed AsciiDoc because of advanced features, but I could realize that the working time stretched compared to pandoc markdown — a two days project could easily become a 4 or 5 days project due to syntax differences. I might add, that working with DocBook source (XML) would take even longer, due to strain on the eyes which made reading very slow and demanded to take breaks quite often.

So, ultimately, when we talk about formatting notation we need to consider the practical aspects of editing work — at the professional level, it's quite stressful, much more than coding, since you have to keep track of styling notations on top of textual contents, and correct punctuation, etc.

For these reasons, anything that can help keeping the notation as light as possible within the main text is a blessing — moving attributes within the footnotes block is an example of such an approach, since the main text has to proofread as a continuos stream, whereas footnotes are just asides that stand on their own, and once they have been checked out they rarely need revisiting.

Delivering errors-free texts is the top priority for any editor, so if a syntax notation makes it harder to follow the main text and confuses the proof-reader, leading to uncaught typos or bad punctuation, this can be a high price to pay.

It would be interesting at this point to have some sample documents, containing average formatting-features examples, as a means to keep tack of how their source files look like in real usage, and being able to compare the same docs in other markups too (markdown, ADoc, etc.) in order to weight the pros and cons of each approach.

Footnotes: Chapter Dislocation

I'm not sure I can think of a practical example why anyone would need to define a note in Ch.X but show it in another chapter.

I was thinking of a use case like this:

• Chapter 4 in a book contains nested sub-chapters.
• Some (or all) nested sub-chapters contain doc notes.
• All doc notes contained in the chapter 4 tree should be displayed at the end of chapter 4.

There is no such thing as sub-chapters — a chapter is a basic unit in breaking down publications: Volume » Book » Part » Chapter » Section.

There are sub-sections though, and sub-sub-sections, etc., although there are no special means to track their sublevels except mentioning their level explicitly. Since cross references are chapter based, when using numbered sections it's easy to stick to the notation §<Ch>.<Sec>.<Sub-Sec>.<Sub-Sec>., which delivers in a clear manner the sectioning level at hand.

In theory, an author can decide to put doc-notes arbitrarily wherever he thinks is most appropriate — at the end of the Book, the Chapter, or even the Section. Similarly, is some multi-volume publications, for example, the Index is in a volume of it's own, while in others every volume has its own Index, it ultimately boils down to what works best.

Having said that, I think that doc-notes settings should specify clearly at which document segment the notes are displayed — whether it's the end of the book, chapter or section. The problem here is that PML doesn't yet provide semantics for book-partitioning — there's only the concept of a document and its chapters (where "chapter" really stands for "section", in editorial terminology).

Without dedicated book-partitioning semantics, the only way to tell PML where to "dump" the notes is via some dedicated placeholder (a node, a macro, whatever). E.g. authors cold manually add at the end of a chapter something like:

[footnotes]

and PMLC would write all the footnotes cumulated so far. This system doesn't even require the parser to track book partitioning: it simply keeps tracking each notes-types, until their placeholders are found, in which case all the notes for that note-type are written, and the process starts over again. Authors could use these placeholders as they please, regardless of book partitioning (e.g. place the footnotes once at chapter end, the next at book end, and so on).

The topic of footnotes is strictly tied to that of book-partitioning, the latter being something that should be native to the syntax (IMO) and not left to custom nodes or functions, otherwise we'll end up with a "Babylonian library", where every user has its custom book-partitioning scheme. I'm not even sure it's all that easy to implement a book partitioning scheme either.

Book partitions are, in turn, strictly connected to section numbering and cross references, as well as with the Index. This is why I stress that some features need a strong native implementation, because they are the backbone of serious editing work.

I hope these thoughts and sideways excursions can help better contextualize the potentials and limitations when it comes to thinking of a publishing-grade lightweight syntax.

[pml-lang]

Good food for thought. Thank you.

I have a few ideas about how we could natively support a more succinct syntax (without the need for function nodes etc.), and maybe also support other syntaxes, without relinquishing the PDML parser. But right now I'm busy and I need some time to first ponder about it, before replying.

[pml-lang]

I'm starting to realize that with its evolution course PML has now embraced the idea of having a standard/formalized way of parsing the syntax. With the very early versions of PML this didn't seem an issue, then came pXML, which later evolved into PDML, which know is at the core of PML.

Data/markup formats like JSON, PDML, and XML all have the following important advantages:

• There are no edge cases that lead to unexpected/undefined/undesired behavior.
• They work well for big and complex documents (no theoretical limits).
• They are easy to learn.
• They are (relatively) easy to parse.

Exactly the opposite is often the case for languages like AsciiDoc, Markdown, reStructuredText, YAML (only for data) etc. that use special symbols and/or whitespace rules to define structure:

• There are edge cases.
• They don't work well in the case of complex documents (e.g. nested lists with complex content are problematic)
• They are often more difficult to learn and error-prone (many rules, exceptions, different implementations, etc.)
• They are challenging to parse (which also makes it difficult to develop tools and editor plugins that work well)
  For example, look at the sheer size of the CommonMark, and YAML specifications. As for AsciiDoc, I don't know if an official specification exists.

In the case of PML, the goal was to get rid of these inconveniences, as explained in my article We Need a New Document Markup Language - Here is Why.

The rules in PML should be simple, there should be no edge cases, and PML should be well suited for big and complex documents.
To achieve this, all PML documents must be valid PDML documents.
This is also required to make PDML features available in PML too (e.g. scripting, convert to JSON, XML, standalone, etc.)

This is a deliberate choice, but it comes at the expense of a standard syntax that is more verbose.

However, there are many options to support shorter syntaxes too (besides the standard PDML notation), without relinquishing the advantages of PDML.

I presented a few options already in my previous comment. For example, power-users could do a lot with function nodes and user-defined-nodes to customize PML.

I will now suggest a few more options that could be supported [i natively] in PML. These are just quick ideas. Some of them might be bad ideas, and there are no plans to implement them in the near future. Some could be added later as an experimental feature first (to get feedback from the community). I just want to mention them here to show that in the future we will be able to support more succinct syntaxes if users ask for them.

Consider the following standard (verbose) PML code:

[image source=resources/tree.png width=200 height=100 border=yes]

There are at least three alternatives to support a more succinct syntax in native PML.

Positional Attributes

Positional attributes (supported in addition to standard named attributes) would allow us to write:

[image resources/tree.png, 200, 100, yes]

However, positional attributes have disadvantages:

• More error-prone. While named attributes can appear in any order, the order of attributes is now prescribed. And if the order changes in the formal node (e.g. an attribute moves to another position, a new attribute is inserted, etc.), then all PML code must be adapted.
• Less readable. 200, 100 is less readable (and more error-prone) than width=200 height=100.
• More difficult to parse, and more rules needed if positional and named attributes can be mixed in a single node (concerns PMLC and external tools, including editor plugins).

Abbreviated Names

Instead of writing image, width, and height we could allow users to just write im, w, and h.
The user would need to provide the minimum number of characters needed to avoid ambiguities with other names:

[im s=resources/tree.png w=200 h=100 b=yes]

The disadvantage is that name abbreviations can become invalid when nodes or attributes are added later in PML.
In that case PMLC would report an error and the user must fix the PML code.

Single Attribute For Compact Syntax

For some nodes (especially those with many attributes) PML could support a special _ attribute to define all values with a short syntax.
Contrived example:

[im _=resources/tree.png<200^100,y]

Support For Other Syntaxes in Dedicated Nodes

We could support PML nodes whose content is written using another syntax. This could be used, for example, to support:

• Markdown within a PML document
• AsciiDoc syntax for tables
• Latex for mathematical formulas

Markdown embedded in a PML raw_text node would then look like this:

[markdown
    ~~~
    A markdown list:

    - item 1
    - item 2
    - item 3 with _italic_ and *bold* text
    ~~~
]

Preprocessor

We could provide a PML preprocessor to support more compact syntax.

For example, the preprocessor could convert <<name>> to [u:get name]

Java API

PML's Java API allows software developers in big enterprise projects to create very tailored solutions for all kinds of exotic needs. For example, they could transform documents using a very succinct, domain-specific syntax into a valid PML document, then use the PDML parser to create a PML AST, change the AST programmatically (e.g. add, remove, change nodes), and then use PMLC to produce HTML documents.

Other Remarks

lists, which are usually represented either via * or - for bullet entries, or via numerals followed by a dot for ordered lists (context being they must be at the beginning of the line); whereas the PML approach becomes more verbose in this case (and lists are a highly demanded feature).

Yes, but - and * lists only work well for simple lists. On the other hand, PML lists (like HTML lists) always work well, even if they are deeply nested and have complex content.

I'm wondering if it could be possible to add an extra dot notation, similar to namespaces but to indicate node subtypes. E.g. [adm.note, [adm.warn,

That's an interesting idea, indeed.

This by itself would be a much shorter solution than having to use a key-value pair like [adm (type=note).

The advantage of using an attribute is that the user can freely specify any text for the admonition's label, e.g.:

[admon (label=Wichtig!)
    Deutscher text ...
]

In any case, I'm getting the feeling that there's a need for some new notation to counter the growing verbosity. In PML v1 parameters could be displayed via the <> notation, now we need [u:get:pname] which is longer and makes the text less readable

Yes, but the <<pname>> notation required characters < and > to be escaped in normal text - a real nuisance for some users, especially those who don't use variables in their PML documents.

PML 1 also allowed to write:

[ch id=my_id Title of Chapter
    ...
]

... instead of:

[ch (id=my_id) [title Title of Chapter]
    ...
]

The initial syntax was of course more convenient, but it was difficult to parse, needed a character reader with lookahead capability, there were edge cases leading to wrong output, it was not possible to use inline nodes in chapter titles, and it would have been very challenging to develop good editor plugins for such syntax alternatives.

Custom solutions come at the price of proliferation of alternative ways of achieving a same thing, and might also incur in maintenance problems

True. That's why we might need to add some native support for a more compact PML syntax (see examples above).

There is no such thing as sub-chapters — a chapter is a basic unit in breaking down publications: Volume » Book » Part » Chapter » Section. There are sub-sections though

I was not aware of that. Thanks for clarifying.

Without dedicated book-partitioning semantics, the only way to tell PML where to "dump" the notes is via some dedicated placeholder

Yes. Good idea.

[tajmone]

Data/markup formats vs Markdown, ADoc, etc.

I agree on the disadvantages of markdown, AsciiDoc and other symbol-leveraging syntax being that:

• There are edge cases.
• They are often more difficult to learn and error-prone (many rules, exceptions, different implementations, etc.)
• They are challenging to parse (which also makes it difficult to develop tools and editor plugins that work well)
• For example, look at the sheer size of the CommonMark, and YAML specifications. As for AsciiDoc, I don't know if an official specification exists.

Well, the whole idea behind those specs is to place the burden on developers in order to make editing life easier for end users, which makes sense. CommonMark in particular has managed to put an end to long standing nightmare of the wild proliferation of Markdown variants and undefined behaviours.

But then, this comparison only goes so far with PML, since CommonMark and YAML provide official implementations in portable C libraries, which means they can be integrated in other tools, even if written in other languages (as long as they provide a decent FFI), whereas PDML/PML being in Java can't provide the same portability (i.e. other tools will have to re-implement PML from scratch).

NOTE — I believe an AsciiDoc spec in under work, but it's still a WIP open proposal, and different implementation might still differ in some details; I think the goal is to move toward a unified format in the long term.

But I disagree with the following point:

• They don't work well in the case of complex documents (e.g. nested lists with complex content are problematic)

AsciiDoc works marvellously with complex lists — nested lists with code blocks, sub-lists of different nature (bulleted, numbered, task-list, definition list, Q&A, etc.). I personally believe that the solutions that AsciiDoc came up with to handle complex lists are quite clever (and not invasive at all in terms of interfering with reading the main text). Sure, mastering ADoc lists requires time, but it's not that hard either.

Unlike Markdown and other syntaxes which rely on indentation to track sub-list elements, AsciiDoc simply augments the number of list-entry symbols according to level. E.g. for bullet lists:

* Top level
** Nested list.
*** Level three list.
* Back to top level.

I.e. it sacrifices the WYSIWYG feeling of markdown (where indentation is a visual cue to the nesting levels) in exchange for a system where by looking at the symbols at the line-start you can clearly track the nesting level.

The rules in PML should be simple, there should be no edge cases, and PML should be well suited for big and complex documents.

While it's true that a formal syntax notation will reduce syntax-related edge cases, when it comes to complex documents there will always be edge cases to deal with — footnotes within footnotes, tables within table cells, and so on (the list is quite long).

In order to support big and complex documents you need to include document-partitioning information in a project (Vol., Book, Ch., Sec.) and account for special sections (App., Preface, Intro., Index, Glossary, Colophon, etc.) — which, of course, requires to compromise on some opinionated choices, since the development of editorial standards differs e.g. in Europe and the Americas, although not all that hugely, but then different types of publications do have different editorial traditions and standards (e.g. juridical vs medical, and so on). Again, all this has to be natively supported in order for PML to qualify for the task.

The introduction of document-partitioning into the language/converter is no small deal, and might affect a number of the existing nodes in unexpected ways.

Footnotes is an example of a feature that really needs to rely on document structure to work well.

Verbosity Solutions

I will now suggest a few more options that could be supported [i natively] in PML. These are just quick ideas. Some of them might be bad ideas, and there are no plans to implement them in the near future.

I can see how each one has its pros and cons. Whereas custom user solution via scripts, custom nodes, functions, etc. are all viable since they don't affect the idiomatic PML way, all other proposal tend to come at a price in one way or another.

I was hoping more to find some additional formal syntax solutions that could be applied uniformly across the syntax to reduce verbosity, as an alternative to key-value pairs attributes. The dot notation could work in some contexts, e.g. to qualify specialized sub-nodes without attributes. But probably there other possible solutions, which only need to be discovered.

The problem with the attributes solution is that IMO it ends up creating a situation where all you have is a hammer, therefore you treat everything as a nail. IMO attributes are well suited for lateral definitions, but don't work well for downward branching (e.g. subnode specialization).

The advantage of using an attribute is that the user can freely specify any text for the admonition's label, e.g.:

Going back to the admonitions example, I believe that note, warning, etc., are sub-types of admonitions, so they shouldn't rely on attributes key-pairs but on the dot sub-type notation (which is shorter). You can still retain the custom label by creating a custom sub-admonition (e.g. admon.custom) which in this case will require the label attribute to specify the custom label:

[admon.warning
    Beware of side effect!
]

[admon.custom (label=Wichtig!)
    Deutscher text ...
]

But the predefined admonitions shouldn't be attribute-dependent but rather native nodes (i.e. subnodes), especially since the expectation is that they will be covered by stylesheets/templates in terms of dedicated colors, icons, etc.

Furthermore, we could take inspiration from HTML and how it introduced classes to eliminate excessive inline CSS styles and HTML attributes, which eventually lead to "separation of concerns". What I'm saying is that IMO attributes should be used only for essentials details that can't be done without, whereas alternative solutions that externalize the details should be preferable when possible, which would allow end users to override some defaults if they want to.

E.g. section numbering is worth externalizing, since some end users might want to tweak similar aspects when rendering a document, but if this setting is handled via inline attributes it's harder (if not impossible) to override it. On the other hand, the language of a code block is not "tweakable", it's an objective data, so it needs to be specified via an attribute.

A possible solution in PML would be to allow a class-like construct where a node tag can optionally be followed by #<class-id> (before the attributes group) so that PMLC can apply all the settings associated to that class, pretty much like in CSS. Example:

[quote #xyz (source="Albert Einstein, physicist")
    Everything should be as simple as possible, but not simpler.
]

where #xyz could control a number of properties of this node (along with others sharing the same ID) without cluttering the source, since it's defined elsewhere.

I have the feeling that eventually we'll find ways of preserving a uniform syntax notation while reducing verbosity — we only need to think outside of the box, to reason where previous system might have failed by stopping to look for alternative ways to abstract away the nature of documents. After all, isn't this the challenge?

[pml-lang]

PDML/PML being in Java can't provide the same portability (i.e. other tools will have to re-implement PML from scratch).

As said already in another discussion or email, it is now possible (since version 3.0.0) to Build a Native Shared Library for PMLC (dll file on Windows; so file on Linux), besides creating standalone executables.
It's just a matter of adding the --shared flag to GraalVM's native-image tool.
However, I didn't do that yet. Only standalone executables for Windows and Linux are currently available on PML's website.
Hence, other tools will NOT have to re-implement PML from scratch. And all JVM languages can simply use the PMLC .jar files.

AsciiDoc works marvellously with complex lists ... Unlike Markdown and other syntaxes which rely on indentation to track sub-list elements, AsciiDoc simply augments the number of list-entry symbols according to level.

The * ** syntax for lists has the same disadvantages as the # ## syntax for chapters, and they have already been explained in chapters Nested Chapters and Lists.
Edge cases, again!
And special rules and exceptions to learn and apply in Complex List Items.
These problems do NOT exist in HTML and PML lists.
Therefore, I do not think at all that AsciiDoc works 'marvellously' with complex lists.
I also know this from my own experience, because AsciiDoc's edge cases in nested lists were among the main reasons I started to ponder about PML.

While it's true that a formal syntax notation will reduce syntax-related edge cases, when it comes to complex documents there will always be edge cases to deal with — footnotes within footnotes, tables within table cells, and so on (the list is quite long).

In PML, 'footnotes within footnotes' will not be allowed, using 'Semantic node validation', a new feature announced already here:
"Semantic node validation is now implemented (but not yet applied to all nodes).
For example, list nodes can only contain el nodes, and each el node must be a child of a list node."

As for 'tables within table cells', they work well in HTML and PML who use a similar syntax (node table in PML). There is no theoretical limit for a table cell's complexity.

For other edge cases that might exist in PML, I suggest to simply open an issue and provide a concrete example. We'll then try to find the best possible solution.

eventually we'll find ways of preserving a uniform syntax notation while reducing verbosity

Yes.

I suggest to stop discussing 'succinct syntax' in this thread. If there is more to say we should create a new discussion dedicated to that topic.

This discussion was created to implement footnotes. I suggest to focus on that topic now.

We need to agree on the nodes used for footnotes, endnotes, etc., using the standard PDML notation, knowing that alternative syntaxes will be supported later, and taking into account the above suggestions from @coemgenuslcp, @tajmone and myself.

For each node we need to define:

• the node name
• how the node is used
• the list of attributes (names, valid values, and default values)

[pml-lang]

Draft 2

This is the second draft, taking into account the above comments and suggestions (especially @tajmone's great idea to use a placeholder node to define the footnotes' location(s) in the final document).
Note that this is just a draft/ideas to be discussed, not a final specification.

Inline Footnotes

Inline footnotes provide an easy way to define simple footnotes.

Node fnote

Node fnote is an inline node used to define a footnote.
It can appear anywhere an inline node is allowed.
It can contain other inline nodes, but not block nodes.

Examples:

doc text [fnote text of footnote 1] more doc text

doc text [fnote text of footnote 2 with [i italic] and [b bold] words] more doc text

doc text [fnote text of footnote 3]
more doc text

doc text [fnote
    text of footnote 4
] more doc text

Note: Instead of fnote (footnote) we could also use dnote or doc_note. I am open for other suggestions for all node/atrribute names.

Node fnotes

Node fnotes is a block node used as a placeholder for printing/displaying footnotes.

Every fnotes node must be preceded by one or more fnote nodes that have not yet been printed.
A document can have any number of fnotes nodes.`

This allows different sets of footnotes to be printed at different locations. For example, the footnotes contained in each chapter could be printed at the end of each chapter.

Examples:

All footnotes printed at the end of the document:

[doc
    [ch [title Chapter 1]
        text [fnote footnote 1] text
        text [fnote footnote 2] text
    ]

    [ch [title Chapter 2]
        text [fnote footnote 3] text
        text [fnote footnote 4] text
    ]

    [header Footnotes]
    [fnotes]
]

Footnotes printed at the end of each chapter (without header):

[doc
    [ch [title Chapter 1]
        text [fnote footnote 1] text
        text [fnote footnote 2] text
        [fnotes]
    ]

    [ch [title Chapter 2]
        text [fnote footnote 3] text
        text [fnote footnote 4] text
        [fnotes]
    ]
]

An error should be generated if:

• A document has an fnote node without a subsequent fnotes node.
• A document has an fnotes node without any preceding fnote nodes.

Note: Standard PML constant can be used to define an inline footnote's content elsewhere:

[u:set fn_1="[fnote long text of footnote ...]"]
...
text [u:get fn_1] text

Block Footnotes

Block footnotes have the following advantages:

• The content of a footnote can be defined anywhere in the document. For example, the content of all footnotes could be defined at the beginning or the end of the document. Or, footnotes could be defined at the end of the paragraph in which they are used.
• Block footnotes can contain block nodes (besides inline nodes), for example to add paragraphs, lists, tables, etc.
• Several references to the same block footnote can exist in the document.

Disadvantages:

• Block footnotes are more verbose. They require two nodes per footnote.
• An identifier must be specified manually for each footnote.

Block footnotes also use an fnotes node to define the location of the footnotes rendered in the final document.

Inline and block footnotes can be mixed in a document.

Node fnote_def

Node fnote_def is a block node that defines the content/text of a footnote.
It's id attribute (which exists for each block node) is required and must be defined manually.

Example:

[fnote_def (id=fn_77)
    Text of [i footnote]
    [table
        ...
    ]
]

Node fnote_ref

Node fnote_ref is an inline node used to insert a reference to an fnote_def node.
It has a required attribute named ref whose value is the id of an fnote_def node.

Example:

text [fnote_ref ref=fn_77]

Example of a document containing a mixture of inline and block footnotes:

[doc
    [ch [title Chapter 1]
        text [fnote footnote 1] text
        text [fnote_ref ref=fn_foo] text
    ]

    [ch [title Chapter 2]
        text [fnote_ref ref=fn_bar] text
        text [fnote footnote 4] text
    ]

    [fnote_def (id=fn_foo)
        text
        [table ...
        ]
    ]

    [fnote_def (id=fn_bar)
        [video ...]
    ]

    [header Footnotes]
    [fnotes]
]

Types And Numbers

Note: This is an advanced feature and presented here just as a preliminary idea to be discussed.

The same document might contain different types of footnotes that must be printed and numbered separately.

Example:
A document contains author notes, editor notes, and publisher notes.
Author notes are numbered numerically (1,2,3,...) and printed at the end of each chapter.
Editor notes should not be displayed in the final document.
Publisher notes are numbered alphabetically (a,b,c,...) and printed at the end of the document.

To achieve this we need to add attribute type to nodes fnote, fnote_def and fnotes.
In nodes fnote and fnote_def this attribute defines the type of footnote, e.g.:

doc text [fnote (type=editor) text by editor] doc text

Default value is author.

In node fnotes this attribute serves as a filter, to define which type of footnotes should be rendered, e.g.:

[ch [title Publisher Notes]
    [fnotes type=publisher]
]

Default value could be * to render all types.

To support different numbering schemes, we could add document options that allow users to specify their preferred numbering rules.

Rendering

Rendering should be done as stated in the first draft (i.e. similar to footnotes in Wikipedia).

Succinct Syntax

As discussed in previous comments, power-users will later be able to define their own, shorter syntax, using upcoming function nodes, user-defined nodes, and maybe additional native PML features we might add in the future.

Hence, a user who often needs to write code like this:

text [fnote_ref type=editor ref=ed-fn_12] text

... will be able to simply write (for example):

text [ed_note 12] text

Another interesting idea submitted by @tajmone (in the context of admonitions) would be to generally support the idea of node sub-types, and specify a footnote's type as a node name suffix, using a dot as separator.
For example, this would allow to write:

[fnote.editor footnote text]

... instead of:

[fnote (type=editor) footnote text]

Implementation

I suggest to proceed as follows:

• Implement and publish inline footnotes (i.e. nodes fnote and fnotes).
• Re-evaluate and implement block footnotes (based on feedback from users).
• Later: design and implement footnote types and different numbering schemes.

[tajmone]

I like your proposal because I think it take the best of both worlds — i.e. inline definitions being the quick solution for those who don't have big needs for their notes, and the block definitions covering advanced use at the price of some added verbosity.

Furthermore, the placeholder to decide where to "unload" all the cumulated notes so far seems a flexible solution; not only it mirrors the natural use case, but it also allows room for special cases if needed.

Here are a couple of personal considerations:

An error should be generated if:

• A document has an fnote node without a subsequent fnotes node.
• A document has an fnotes node without any preceding fnote nodes.

I think a warning would be sufficient in this case, because it would allow users to publish a document without notes, if they wish to. In general, if an "error" could possible be intentional then it should be treated as a warning, and there should be an extra option to allow PMLC failure at warning level, which would allow to ensure that such cases are caught when non intentional.

To achieve this we need to add attribute type to nodes fnote, fnote_def and fnotes.

I was wondering if instead of introducing this attribute it wouldn't be possible to instead just cover two notes types (footnotes and editor notes) and have a similar set of nodes for both — i.e. enote, enote_def and enotes along with fnote, fnote_def and fnotes. Although three types of doc-notes would cover every need, even having two different types would be already a huge leap in the world of lightweight markup syntaxes, and end users could still rely on custom nodes, functions and scripts to implement the third type in the rare case they are needed.

Although this solution introduces the redundancy of similar nodes, this shouldn't be a problem in terms of the parser, and in any case most formatting nodes are variations of each other (just like [b and [i are very similar). In terms of learning curve, there's only a single letter difference in these nodes: footnotes begin with f and endnotes with e, and the rest of these nodes names are very intuitive and easy to remember: the [*note singular form for inline definitions), the plural [*notes for their "dumping" placeholder, etc.

As for the numbering system of each type, I'm sure that document and project setting can be easily used to customize their look and feel, in cases in which the default values need to be changed (which most likely is not case).

In any case, I see that these proposal are starting to narrowing down on the solution, as we've already seen quite an evolution in terms of how footnotes could be implemented without violating the formal parsing paradigm. So I'm quite optimistic.

[pml-lang]

I think a warning would be sufficient in this case

Yes, I agree! An error is too harsh, indeed.

For example, it would inhibit a user to generate the target document if he intentionally wants to omit the footnotes.

I was wondering if instead of introducing this attribute it wouldn't be possible to instead just cover two notes types (footnotes and editor notes) and have a similar set of nodes for both

Yes, that might be useful for some people.

Generally speaking, there's a lot we could add to PML, but I'm always worried about adding too much too quickly, or introducing unnecessary complexity.

For example, if we later decide to support note types, then e notes might become obsolete, or confuse new users.

Therefore I suggest to start with fnote and fnotes. Adding e notes later if users ask for it will be very easy to to, as it's just a variation of an existing node.

[tajmone]

For example, if we later decide to support note types, then e notes might become obsolete, or confuse new users.

I agree 100%, there should be only one way to achieve this, and it's better to avoid introducing something than changing it again.

Therefore I suggest to start with fnote and fnotes. Adding e notes later if users ask for it will be very easy to to, as it's just a variation of an existing node.

Now that we have a better view of where all this is going it's safe to start with a simple one-notes system, and later on when we have a better picture we'll see which path is better (e.g. notes types via the . notation, or whatever other method to specify types).

Generally speaking, there's a lot we could add to PML, but I'm always worried about adding too much too quickly, or introducing unnecessary complexity.

I know, it's hard to draw the line where "necessary" features end and "esoteric" ones begin. I'd probably go as far as to state that from PML we could expect a bit more in terms of core features in terms of covering editorial needs. I.e. if AsciiDoc is richer than markdown in terms of advanced core features, then maybe PML should add more, especially in terms of those features that might be viewed as limited or problematic to handle in other formats.

[pml-lang]

A first version of footnotes has been added in the new public PML version 3.1.0.

Inline footnotes as well as block footnotes are now implemented.
I have also added an optional attribute text to node fnote_ref. This attribute allows to explicitly specify the inline text displayed in a footnote reference. By default the index in the list of footnotes is displayed.
Types And Numbers is not implemented yet.

For more information please visit:

• Changelog
• Inline Footnote fnote
• Footnotes Placeholder fnotes
• Footnote Definition fnote_def
• Footnote Referencefnote_ref

The source code changes can be seen here.

Feedback is very welcome.

PS: I was thinking about (optionally) adding tooltips/popups to inline footnote references (displayed on mouse hovering). The content of the tooltip/popup would be the content of the footnote. This would eliminate the need to go forward and then backward to read a footnote. However, we would have to cover the case of big footnotes (including tables, etc.) - too big to be displayed as a tooltip/popup.

[tajmone]

Very nice! Also quite cool the text attribute addition to override the default footnote symbol with any custom text — could be quite useful for quick bibliography-style references pointing to a footnote, e.g. text="W.Smith 1984", combining a bibliography reference and a footnote together.