Early or lazy term/goal expansion.

[dougransom]

When watching @triska's recent video on Term and Goal Expansion, I had some thoughts. Would a suitable time to perform an expansion include:

• installation, when a package is installed by backage or similar? If an expansion only needs to be done once, that seems like it might be a good time. Of course expanded code needs to be saved somewhere.
• lazily? maybe a good time to perform expansions is the first time a term is required?

[jjtolton]

I think it's very different for package developers and application developers, or that is to say, there is a different relationship between intra-team and inter-team term/goal expansion techniques.

The general rule should be, if your package is going to be an upstream dependency, goal/term expansion should be an implementation detail of your library and must be carefully constructed so as to not interfere with any other term/goal expansions or have ordering effects. The onus is on the library writer to ensure the term/goal expansion does not leak in ways that would be surprising. This also puts pretty severe restraints on how you write your expansions -- which in some ways makes it easier to write them. I think clpz is an excellent example of this sort of work.

When writing code that has no downstream consumer, such as application code, you can be more aggressive about what your expansions do -- to the point of effectively creating a DSL for your application (or parts of the application). My advice would be, as much as possibly, to centralize your expansion definitions. For instance, put them all in the same file, and try to maximize localization and coherence between expansions that have complementary effects. I would probably also put in some automated runtime checks to confirm assumptions about the expansions -- Prolog can be overly forgiving about "cat walking across the keyboard" and the effects of a subtly misbehaving expansion can be very difficult to debug.

I spent two weeks dealing with such a bug before finally getting a helpful code review from @triska who was able to identify that I had written some throwaway "scratch" code which created an interfering expansion and caused very strange errors in clpz!

• installation, when a package is installed by backage or similar? If an expansion only needs to be done once, that seems like it might be a good time. Of course expanded code needs to be saved somewhere.
• lazily? maybe a good time to perform expansions is the first time a term is required?

To respond to these questions more directly, the good news is that these are already handled for you -- although the way in which it happens is not so obvious (and may change over time!)

In my talk last year I catalogued the lifecycle and positionality of term_expansion/2 and goal_expansion/2.

tl;dr, to clarify on your bullet points, the term_expansion/2 is currently applied to all positionally subsequent terms after the term expansion is defined, so from a user perspective, all your code will be potentially affected by a term expansion when you do :- use_module(...)..

goal_expansion/2 always happens at "evaluation time".

• installation, when a package is installed by backage or similar? If an expansion only needs to be done once, that seems like it might be a good time. Of course expanded code needs to be saved somewhere.

For the sake of completeness and clarification, this would require a different technique than expansion -- this would actually be compiling (or transpiling) rather than expanding the code. They are not unrelated, you could use term_expansion/2 to write out code to a file as you expand terms, although this is perhaps an off-label use of expansion!