The (.) macro's design is broken

[tabatkins]

In the README, you imply that (. a 3) compiles to a[3] (good), but then state that (. a b) compiles to a.b (in other words, for all but the first argument atoms are converted to symbols/strings). This means it's impossible to access into a data structure with a variable; there's no way to write the equivalent of JS's a[b].

To do this properly, named property access needs to always be done with strings, like (. a 'b'), or at least with self-evaluating symbols, like (. a :b). This way everything that looks like variables is actually variables.

[anko]

The computed member expression macro is called get.

(get a b)→ a[b];
(. a b)→ a.b;

It's mentioned toward the end of the tutorial section about arrays and objects. I realise the distinction can be confusing though, and the docs should communicate it better.

[anko]

To do this properly, named property access needs to always be done with strings, like (. a 'b'), or at least with self-evaluating symbols, like (. a :b). This way everything that looks like variables is actually variables.

In JavaScript, a.b does the same as a["b"], so I think it's logical that (. a b) and (. a "b") should do the same also—right?

[tabatkins]

Ah, kk. I disagree with the design, but at least it can be worked around.

In JavaScript, a.b does the same as a["b"], so I think it's logical that (. a b) and (. a "b") should do the same also—right?

The difference is that in a.b the "b" part syntactically cannot be a variable, and you quickly learn the difference between a.foo and a[foo] (along with what other things can't be expressed via dot notation, like a[3] or a[b+"foo"]).

The (.) macro, though, eliminates the syntax distinction. You can put whatever literal value you want in there, whether it matches the JS identififer syntax or not - (. a "foo") and (. a 3) are equally valid. It is a general principle that you can always replace a literal with a variable containing that literal, but here you can't - switching to (block (= b "foo") (. a b)) does the wrong thing.

Similarly, if I assume that (. a (+ b "foo")) is valid (it looks like it is, per the (. a b (. c d)) example in the docs), then you further violate expectations when a seemingly-trivial edit - removing the concat - drastically changes which property is accessed (it switches from retrieving the value of b and concatting "foo" to it, to just retrieving the "b" property).

It's common in Lisp APIs to use self-evaluating symbols (:foo) when you want "lightweight strings" in some API. This is why I suggested (. a :b) as the syntax to desugar to a.b. Self-evaluating symbols are detectable at parse-time (in Lisps they're implemented via a reader macro), so it should hopefully be reasonably simple with your approach.

(Another inconsistency in the current approach - (. a b) desugars to a.b, but (. a foo-bar) cannot desugar to a.foo-bar, as that means something totally different in JS. You might restrict dashes in variable names, I'm not sure, but if you don't, this already needs to desugar differently, to a["foo-bar"], so the direct-mapping metaphor breaks down even for relatively simple cases.)

[anko]

Thanks so much for your thoughtful analysis. I think I see what you mean now.

What you suggest is that if the . macro behaved such that identifiers in it always implied computed member expressions (e.g. (. a b) → a[b]) and strings always non-computed member expressions (e.g. (. a "b") → a.b), then we could discard the get macro altogether and eliminate this confusion.

Test cases:

(> (. a b) 1) → a[b] > 1
(> (. a "b") 1) → a.b > 1 (or equivalently, a['b'] > 1)
(. a (+ b "foo")) → a[b + "foo]

I think it's a great idea.

I could implement it tomorrow. Can I ping you for a review?

---

Eslisp checks the final AST being compiled to JS for illegal identifier names (e.g. foo-bar) and logs errors if it finds them, so they are no problem.

If you prefer to use dash-separated variables, eslisp-camelify can automatically turn them into camelCase. This is pretty convenient with the --transform flag that wraps the whole program in a macro, as used e.g. here.

[tabatkins]

Yeah, I'm happy to do a review. ^_^

[Gonzih]

Why not use . for method invogation and get for property access? Previous and new . semantics feel very confusing to me.

[anko]

@Gonzih I don't understand. Could you give an example?

[Gonzih]

It might be a bit far from current implementation and ideas. Also might be influenced by clojure heavily.

What I don't like is that property access and function invocation are separate things.

It's to verbose to get function and then invoke it. For me it feels much more stricts to use get to access properties, but (. a b c) would alsay compile to a.b(c). But when I think about it it might be different from original design :)

[anko]

My idea is to keep the core language very plain (so e.g. . means exactly a member expression, nothing else), then let user-defined macros add sugar, and make user macros as easy as possible to write and start using.

(. a b c)→a.b(c); sounds like it would work great as a user macro though. The eslisp-fancy-function macro is an analogous thing for the function macro. (If you want to write one and want tips or get stuck, do ask. 😄)

[Gonzih]

ok, I understand. I will try to play with user defined macros for my needs. Thanks!

[Gonzih]

Then I would say that removing get function makes sense :)

[anko]

@Gonzih It's not quite just a matter of removing get—there are some downsides too. I've summarised the points for and against it here. Do you think the tradeoff is worth it?

[Gonzih]

Ugh i forgot about names that collide with things like catch/for etc. If it's only case is it possible to make . macro a bit smarter and if name collides with reserved keyword convent this call in to ["for"]?

(. a "for") -> a["for"] instead of a.for

of course then you kind of tie macro implementation to the underlying js implementation without any easy way to extend it.

Removing get feels like removing a bit of possible flexibility (have no idea if this flexibility is actually needed though).

[anko]

@Gonzih The flexibility is sometimes needed. Quoting @lhorie from chat:

Oh and another use case I just remembered is promises, which have a catch method. Identifiers that have the same name as keywords must be written in bracket format if you want to support old IEs (i.e. foo.catch() throws a syntax error in old IEs, so you need to write foo["catch"]() instead)

for closure compiler, the difference between the two is that advanced mode can minify foo.bar into a.b, but will minify foo["bar"] into a["bar"] (which is useful if you're exporting symbols from a library for consumption)

---

All three of a.b, a["b"] and a[b] must be expressible, but we only have two text-containing AST node types to work with (identifiers and strings). So to distinguish computed access from static access, we really do need 2 separate macros.

With that, I'm closing this and #14 (my implementation PR) as not an issue.

Many thanks to all involved in discussing this, especially to @tabatkins for raising it and taking the time to explain so thoroughly, @lhorie for the above insights and @whacked for an overview of analogous syntax in wisp and cljs.

[tabatkins]

So to distinguish computed access from static access, we really do need 2 separate macros.

No we don't, and the current design already conflates them heavily anyway; the distinction is artificial and confusing. (. a 1) can tell that it needs to desugar to a[1], and (. a (+ "foo" "bar")) knows that it needs to desugar to a["foo"+"bar"]. On the other hand, the current design incorrectly assumes that (. a foo-bar) should desugar to a.foo-bar, and ends up throwing an error when verifying the AST.

I described in the PR what needed to be done to maximize use of the a.b form but still use a[b] form when necessary. It's not difficult, you just need a slightly smarter check than "is it a string". I'm not sure why you concluded the changes couldn't be done in the PR.

In general, the only place where a lisp macro treats a symbol literally (rather than assuming it's a variable/function) is when it's a definition macro, like (let), and the symbol in question is the name being defined. I don't know of any macros that use symbols as strings, with the exception of self-evaluating symbols like :foo, which are semi-intended for this very purpose.

(Symbols-as-strings did happen in very old Lisp, long before Common Lisp and the string type, but that's so old to be irrelevant to our discussion.)

[anko]

@tabatkins In that case, I must have completely misunderstood.

To be maximally clear, what forms do you think should compile to a.b, a[b] and a["b"]?

[tabatkins]

(. a b) should compile to a[b] at all times - you're clearly invoking the variable a and the variable b, and any other interpretation is confusing. (Put another way, there's no reason, Lisp-wise, for (.) to be a macro, and so per the principle of least power, it should be written as a function. And if it's a function, then a and b are definitely both variables in that expression.)

(. a "b") should compile to a.b for every string for which that is valid, and to a["b"] otherwise. The rules for what makes a valid ident are here, and per @lhorie, you should also exclude the list of reserved keywords, for compat with old browsers.

You might want to accept (. a :b) as equivalent to (. a "b"), as the pattern of using self-evaluating symbols as "easier-to-type strings" for identifiers is reasonably common in my experience. But that's not required. For example, a number of HTML templating libraries have functions of the form (div :foo "bar" "content") meaning <div foo="bar">content</div>.

[whacked]

@tabatkins which means

1. (. a one) compile to a[one]
2. (. a "one") compile to a.one
3. (. a ".one") compile to a[".one"]

one problem with not having an strict compile to bracket-access syntax is if you use a minifier, 2 is subject to renaming, but generally if you specified the string-quoted attribute already, you don't want it to be renamed [1]. In effect the dot accessor here acts like (.-one a) in cljs.

[1] http://squirrel.pl/blog/2013/03/28/two-ways-to-access-properties-in-clojurescript/

[anko]

@tabatkins

(. a "b") should compile to a.b for every string for which that is valid, and to a["b"] otherwise.

That's the problematic part.

For this reason—

for closure compiler, the difference between the two is that advanced mode can minify foo.bar into a.b, but will minify foo["bar"] into a["bar"] (which is useful if you're exporting symbols from a library for consumption)

—there needs to be a way to write both the a.b and a["b"] form of any b.

With your idea, for a given b for which both the a.b and a["b"] forms would be valid outputs, the a.b form will always be chosen, and the a["b"] form can't be expressed at all.

[tabatkins]

Ah, that's an interesting objection. In that case, how about everything always desugars to brackets, except self-evaluating symbols (:b) desugar to dot-access (and throw an error if they can't, so (. a :foo-bar) is invalid, but (. a "foo-bar") is fine).?

That gives us complete consistency, while allowing the minifiable a.b form with only a single character "tax" for syntax correctness.

[whacked]

@tabatkins to my knowledge the current state is to use (. a b) basically in the same way as (.-b a) in cljs, i.e. always compile to a.b; and (get a b) will always compile to a[b].

I haven't seen any discussion of the :keyword syntax, but @anko seems to prefer having extra syntax processors in transformation macros, so that's what I'm doing.

[anko]

to my knowledge the current state is to use (. a b) basically in the same way as (.-b a) in cljs, i.e. always compile to a.b; and (get a b) will always compile to a[b].

That's correct, with the small detail that . compiles string and number literals to square-bracket notation because it's the only sensible interpretation. But that's a detail.

---

As for :keyword: I'd like to avoid adding more types to the AST. Strings and identifiers are enough text-containing AST nodes for JS, so why not for eslisp? I'd rather have 1 more macro than 1 more AST node type.

[whacked]

@anko re: :keyword, that's fair. In https://github.com/Gozala/wisp :keyword is just sugar for "keyword" so the transform macro is trivial

[tabatkins]

JS has "just strings and identifiers" because it has syntax that lets it provide special rules for how to interpret them in different circumstances; that's how it can get away with a.b and a[b] meaning completely different things.

Lisp has a vastly simplified syntax, and so it has to be somewhat more explicit about a number of things. Or it can "fake" syntax, like cljs (validly) does, with specially-constructed names like (.-b a) There, the "identifier" is embedded into a larger identifier with a well-known structure, which is also a Lisp-y pattern. (But that syntax doesn't let you chain accesses, which is a useful part of the (.) macro.)

Ultimately I won't fight this too hard; if it's a general design philosophy to use identifiers like this, then ok, it just means I'm definitely not using the language. ^_^ I need to be able to predict how the language works, and when ordinary functions start invoking unusual parsing rules that need to be memorized, I'm out.

[lhorie]

For posterity, I had suggested this in the chat: (. a b) -> a[b], (. a "b") -> a["b"] and (. a 'b) -> a.b, under the rationale that quote is the lisp mechanism to work with a symbol's label as opposed to its evaluated value (but I acknowledged this isn't necessarily a popular idiom).

Re: :keyword, this type came up separately on the topic of dynamic property names in ES6 as a potential mechanism to express identifiers. I think, similarly, talking in terms of what should be "cast" to square bracket notation is a misguided notion, and that it's better to think in terms of denoting an identifier.

[anko]

I'm really warming up to @lhorie's quote suggestion:

• It would address @tabatkins' very valid concerns about predictability.
• It doesn't require adding any new AST types or macros.
• Actually, we can even discard the get macro that way.
• The same idiom could be used to also fix The object macro is not ES6-friendly #23, which I'm totally stumped on otherwise.
• User macros to revert the current behaviour are trivial to write.

Is there anything bad about it?

[whacked]

personal vote 👍

caveat is my experience is only from cljs and wisp, but within these families this change looks not bad, and I will use this feature

[tabatkins]

Quote is fine, yeah. In "real lisp" it causes the value to be of type SYMBOL, same as :b would. I'm not sure how that allows you to avoid adding a new AST type, tho - :b is also of type SYMBOL. (Technically it's of type KEYWORD, which is a subtype of SYMBOL.)

And yes, the object notation from #23 has the same problems. I was going to bring it up when this thread was resolved. ^_^

[anko]

Reopening to clarify that the proposed solution hasn't yet been implemented.

[dead-claudia]

Suggestion for this, inspired by my suggestion in #23: make static properties (. obj 'foo) and computed properties (. obj foo) or (. obj (func)). Does that sound like a good resolution for this?

[anko]

@IMPinball Yep, that's the plan; @lhorie outlined it similarly before too.

The issue blocking me from doing this right away is that currently 'foo compiles to the wrong value when used anywhere other than as a macro return value: 'foo → ({ atom: 'foo' });. This in turn is because user macros and built-in macros actually use different formats for representing the S-expression tree.

I'll hopefully soon have more time and attention to focus on it.

[dead-claudia]

@anko I think the difficulty is that sexpr-plus aliases 'foo as (quote foo) and returns that result, instead of combining the two. We'd have to create a new node type for sexpr-plus to fix that problem (it's bad behavior IMHO, but it'd require a major version bump as sexpr-plus is already past 1.0). Meanwhile, I'm tentatively implementing @lhorie's syntax.

[dead-claudia]

My tentative implementation is coming in a PR I'm currently working on, which will fix both this and #23.

[iovdin]

i'd expect (. (foo) (bar)) to be translated to foo().bar()
which actually translates to foo()[bar()]

[vendethiel]

You want ((. (foo) "bar")).

[iovdin]

@vendethiel right,
but for long chains which happens quite often it becomes not that readable
(. (foo) (bar) (baz)) i have to write ((. ((. (foo) "bar") "baz)))

[vendethiel]

You might be interested in https://www.npmjs.com/package/eslisp-chain

[iovdin]

@vendethiel thanks