3 Use Cases

This page discusses 3 different approaches to managing types definitions, and shows how the essential developer experience can be enhanced using Typal.

Table Of Contents

• Table Of Contents
• Naïve approach
• JSDoc approach
• Closure approach
  • Shell Command To Spawn Closure
  • Google Closure Compiler Warnings
  • Closure-Compatible JSDoc
  • Generated Externs
  • Result Of Compilation
  • Externs As Types
  • Annotating Types

The example given below will illustrate why Typal is extremely useful as the tool both for plain JSDoc management and JSDoc for Google Closure Compiler workflow.

Naïve approach: Let's implement a transform stream that updates data using regular expressions specified in the constructor:

import { Transform } from 'stream' export class Restream extends Transform { /** * Sets up a transform stream that updates data using the regular expression. * @param {Rule} rule The replacement rule. * @param {TransformOptions} [options] Additional options for _Transform_. */ constructor(rule, options) { super(options) this.rule = rule } _transform(chunk, enc, next) { this.push( `${chunk}`.replace(this.rule.regex, this.rule.replacer) ) next() } } /** * @typedef {Object} Rule The replacement rule. * @prop {RegExp} regex The regular expression. * @prop {(...args:string) => string} replacer Updates matches. * @typedef {import('stream').TransformOptions} TransformOptions */

In the file, we have defined a type using typedef, and imported a type from the internal Node.JS API. All is well, and we get our JSDoc autosuggestions that help us understand that what we're doing is correct.

However, there are 2 problems with that:

1. Google Closure Compiler does not understand typedefs a) without variable declaration underneath, b) with @properties and c) with functions in (...args: string) => string notation. The format for GCC typedef for our example would be the one below. And if we tried to use it, VSCode would not understand it, and we would loose the description of individual properties of the type.

   /**
    * @typedef {{ regex: RegExp, replacement: function(...string): string }}
    */
   var Rule
   
   // or, as a record type
   
   /** @record */
   var Rule
   /** @type {RegExp} */
   Rule.prototype.regex
   /**
    * @param {...string} args
    * @returns {string}
    */
   Rule.prototype.replacement = function(...args) {}

2. Google Closure Compiler does not understand @typedef {import('from').Name} syntax. It is currently not supported, and to be able to reference types from other packages, they must have externs. So for the TransformOptions, we need stream.TransformOptions externs. To reference types from the same package but across files, GCC will need types to be imported as ES6 imports (like how things were in 2018), e.g.,

   import Rule from './src' // pre-typedef import
   /**
    * @param {Rule} rule
    */
   const fn = (rule) => {}

3. The documentation that we wrote as JSDoc type declarations has to be copied and pasted into the README.md file manually, and all tables need to be also constructed.
4. When trying to create a new Restream instance, it is not clear what properties the Rule type should have, because VSCode does not expand that information:

   

JSDoc approach: Now let's refactor the code that we have, and place the types definitions in the types.xml file instead of the source code:

<types>
  <import from="stream" name="TransformOptions"
    link="https://nodejs.org/api/stream.html#stream_class_stream_transform" />
  <type name="Rule" desc="The replacement rule." noToc>
    <prop type="RegExp" name="regex">
      The regular expression.
    </prop>
    <prop type="(...args:string) => string" name="replacement">
      Updates matches.
    </prop>
  </type>
</types>

The types files support <import>, <type> and <prop> tags. We then update the source code to indicate the location of where types should be read from (there needs to be a newline before the end of the file):

import { Transform } from 'stream'

export class Restream extends Transform {
  /**
   * Sets up a transform stream that updates data using the regular expression.
   * @param {Rule} rule The replacement rule.
   * @param {TransformOptions} [options] Additional options for _Transform_.
   */
  constructor(rule, options) {
    super(options)
    this.rule = rule
  }
  // ...
}

/* typal example/restream/types.xml */

Then, we call the typal binary to get it to update the source: typal example/restream/index.js:

import { Transform } from 'stream'

export class Restream extends Transform {
  /**
   * Sets up a transform stream that updates data using the regular expression.
   * @param {Rule} rule The replacement rule.
   * @param {RegExp} rule.regex The regular expression.
   * @param {(...args:string) => string} rule.replacement Updates matches.
   * @param {TransformOptions} [options] Additional options for _Transform_.
   */
  constructor(rule, options) {
    super(options)
    this.rule = rule
  }
  // ...
}

/* typal example/restream/types.xml */
/**
 * @typedef {import('stream').TransformOptions} TransformOptions
 * @typedef {Object} Rule The replacement rule.
 * @prop {RegExp} regex The regular expression.
 * @prop {(...args:string) => string} replacement Updates matches.
 */

From that point onward, the JSDoc documentation is managed from the separate file. It can also be embedded into the Markdown, using the Documentary documentation pre-processor by adding the %TYPEDEF: example/restream/types.xml% marker in the README file:

import('stream').TransformOptions stream.TransformOptions

Rule: The replacement rule.

Name	Type	Description
regex*	RegExp	The regular expression.
replacement*	(...args:string) => string	Updates matches.

The link to the Rule type was also added to the Table of Contents, however it can be skipped if the type element had the noToc property set on it. We also added the link property to the import element to place a link to Node.JS API docs in documentation.

Another advantage, is that the Rule type was expanded into individual properties in JSDoc above the constructor method. It allows to preview all properties and their descriptions when hovering over functions:

Closure approach: Finally, if we want to allow our package to be compiled as part of other packages with GCC (or compile a binary from the lib we've written), we need to make sure the JSDoc is in the format that it accepts.

We create a simple program that uses our Restream library:	And run it with Node.JS:
import { Restream } from 'restream' const restream = new Restream({ regex: /__(.+?)__/, replacement(match, s) { return `<em>${s}</em>` }, }) restream.pipe(process.stdout) restream.end('__hello world__')	<em>hello world</em>

Let's try to compile a program using GCC now (using Depack) and see what happens:

Shell Command To Spawn Closure
java -jar google-closure-compiler-java/compiler.jar \ --compilation_level ADVANCED \ --language_out ECMASCRIPT_2017 --formatting PRETTY_PRINT \ --externs @depack/externs/v8/stream.js \ --externs @depack/externs/v8/events.js \ --externs @depack/externs/v8/global.js \ --externs @depack/externs/v8/nodejs.js \ --module_resolution NODE --output_wrapper "#!/usr/bin/env node 'use strict'; const stream = require('stream');%output%" \ --js node_modules/stream/package.json \ node_modules/stream/index.js \ example/restream/program.js \ example/restream/compat.js
The command above was generated with Depack call on the right, where: -c means Node.JS compilation (adds the wrapper, mocks and externs), -a means ADVANCED mode, and -p means pretty-printing.
	depack example/restream/program \ -c -a -p

Google Closure Compiler does not discover source code files the list of which must be passed manually. In addition, it does not work with internal Node.JS modules natively. The software that performs static analysis of programs to detect what files to feed to the compiler, as well as mocking Node.JS built-in modules in the node_modules folder and providing externs for them is called Depack.

After finishing its job, the compiler will give us warnings shown below, which tell us that the program was not type-checked correctly. Sometimes we can ignore warnings, but we loose the ability to ensure correct typing. It is also possible that the compiler will perform the advanced optimisations incorrectly by mangling property names (e.g., regex becomes a), but it is not the case here because all files are used together, but if we were publishing the library, the first parameter rule would not adhere to the Rule interface.

Google Closure Compiler Warnings
restream/index2.js:6: WARNING - Bad type annotation. Unknown type Rule * @param {Rule} rule The replacement rule. ^ restream/index2.js:8: WARNING - Bad type annotation. type not recognized due to syntax error. See https://git.io/fjS4y for more information. * @param {(...args:string) => string} rule.replacement The function ... ^ restream/index2.js:9: WARNING - Bad type annotation. Unknown type TransformOptions * @param {TransformOptions} [options] Additional options for _Transform_. ^ restream/index2.js:25: WARNING - Bad type annotation. expected closing } See https://git.io/fjS4y for more information. * @typedef {import('stream').TransformOptions} TransformOptions ^ restream/index2.js:26: WARNING - Bad type annotation. type annotation incompatible with other annotations. See https://git.io/fjS4y for more information. * @typedef {Object} Rule The replacement rule. ^
The warnings produced by the compiler tell us the points discussed in the beginning: the classic typedefs {Object} Rule, function types (...args:string) => string, and imports import('stream').TransformOptions are not understood.

This is because the traditional JSDoc annotation is not compatible with the compiler. To solve that, we need to compile JSDoc in Closure mode with Typal. First, we want to adjust our types to include more features:

Updated Types For Closure (view source)
<types namespace="_restream"> <import from="stream" name="TransformOptions" link="https://nodejs.org/api/stream.html#stream_class_stream_transform" /> <type name="Rule" desc="The replacement rule."> <prop type="!RegExp" name="regex"> The regular expression. </prop> <prop type="(...args:string) => string" closure="function(...string): string" name="replacement"> The function to update input. </prop> </type> <type type="!Array<!_restream.Rule>" name="Rules" desc="Multiple replacement rules."> </type> </types>
Annotate the nullability of our types using !, since there's attention to null in GCC, not like traditional JS. We also add the closure property to the prop elements to make them use that type in source code instead of the traditional one. The more readable type descriptions are retained to be placed in README documentation. This just means that the type attribute is what will be visible in documentation if using Documentary, but in code, the closure attribute will be used when compiling in Closure mode. Add the namespace, because we're going to generate externs and if there are other programs that define the Rule extern, there would be a conflict between the two. Adding namespace ensures that the chances of that happening are minimal. In addition, we prefix the namespace with _ because we'll put it in externs, and if we or people using our library called a variable restream, the compiler will think that its related to the extern which it is not because it's a namespace in externs, but an instance of Restream in source code. Finally, add another type Rules just to illustrate how to reference types across and within namespaces. Although defined in the same namespace, the properties need to give full reference to the type.

If we now compile the source code using the --closure flag (so that the command is typal example/restream/closure.js -c), our source code will have JSDoc that is fully compatible with the Google Closure Compiler:

The Source Code With Closure-Compatible JSDoc (view source)
import { Transform } from 'stream' export class Restream extends Transform { /** * Sets up a transform stream that updates data using the regular expression. * @param {!_restream.Rule} rule The replacement rule. * @param {!RegExp} rule.regex The regular expression. * @param {(...args:string) => string} rule.replacement Updates matches. * @param {!stream.TransformOptions} [options] Additional _Transform_ props. */ constructor(rule, options) { super(options) this.rule = rule } _transform(chunk, enc, next) { this.push( `${chunk}`.replace(this.rule.regex, this.rule.replacement) ) next() } }
There have to be some manual modifications to the source: We rename the @params to use the namespace and make it non-nullable since it's a thing in Closure, i.e., if we don't do it the type of the param will actually be (restream.Rule|null): @param {_!restream.Rule} rule; We also add the namespace to the internal module @param {!stream.TransformOptions}, because in Closure the externs are provided for the stream namespace by Depack. The following changes are introduced automatically by Typal after we started using the --closure mode:
/* typal example/restream/types2.xml */ /** * @suppress {nonStandardJsDocs} * @typedef {_restream.Rule} Rule The replacement rule. */ /** * @suppress {nonStandardJsDocs} * @typedef {Object} _restream.Rule The replacement rule. * @prop {!RegExp} regex The regular expression. * @prop {function(...string): string} replacement Updates matches. */
The Rule type is now defined using 2 @typedefs, which are also suppressed to prevent warnings. The reason for the first item is so that the type can be imported in other files from our package, using {import('restream').Rule}. This is so because {import('restream')._restream.Rule} does not work in VSCode. The second type stays as is, and is printed with the namespace. It is still not picked up by GCC, but the warning is suppressed. Instead, when we come to generate externs in a minute, their name will match _restream.Rule, and the param for the function will be recognised by the compiler.
/** * @suppress {nonStandardJsDocs} * @typedef {import('stream').TransformOptions} stream.TransformOptions */
The imports are now also suppressed (but the change will hopefully come into effect in the next version of the compiler), and printed with the namespace, so that we can refer to them in params and get both the autosuggestions, and Closure compatibility.
/** * @suppress {nonStandardJsDocs} * @typedef {_restream.Rules} Rules Multiple replacement rules. */ /** * @suppress {nonStandardJsDocs} * @typedef {!Array<!_restream.Rule>} _restream.Rules Multiple replacements. */
Any types within the namespace must refer to each other using their full name.

Before we continue to compilation, we still need to generate externs, because the Closure compiler does not know about the Rule type. Externs is the way of introducing types to the compiler, so that it can do type checking and property renaming more accurately. Once again, we place the /* typal example/restream/types2.xml */ marker in the empty externs.js file, and let Typal to the job with typal example/restream/externs.js --externs command (or -e).

Generated Externs For Restream (view source)
/* typal example/restream/types2.xml */ /** @const */ var _restream = {} /** * @typedef {{ regex: !RegExp, replacement: function(...string): string }} */ _restream.Rule /** * @typedef {!Array<!_restream.Rule>} */ _restream.Rules
The externs are generated with the Closure-compatible syntax and ready to be used for compilation of our example program.

To continue, we run depack example/restream/program -c -a -p --externs restream/externs.js again, and this time, Depack will pass the externs argument to the compiler as we request.

Result Of Compilation
#!/usr/bin/env node 'use strict'; const stream = require('stream'); const c = stream.Transform; class d extends c { constructor(a, b) { super(b); this.a = a; } _transform(a, b, f) { this.push(`${a}`.replace(this.a.regex, this.a.replacement)); f(); } } ;const e = new d({regex:/__(.+?)__/, replacement(a, b) { return `<em>${b}</em>`; }}); e.pipe(process.stdout); e.end("__hello world__");
stdout
java -jar /Users/zavr/node_modules/google-closure-compiler-java/compiler.jar \ --compilation_level ADVANCED --language_out ECMASCRIPT_2018 --formatting \ PRETTY_PRINT --externs example/restream/externs.js --package_json_entry_names \ module,main --entry_point example/restream/program.js --externs \ node_modules/@externs/nodejs/v8/stream.js --externs \ node_modules/@externs/nodejs/v8/events.js --externs \ node_modules/@externs/nodejs/v8/global.js --externs \ node_modules/@externs/nodejs/v8/global/buffer.js --externs \ node_modules/@externs/nodejs/v8/nodejs.js Modules: example/restream/compat.js Built-ins: stream Running Google Closure Compiler 20191224
stderr

Although we've generated the externs and passed them to the compiler, we don't actually need them here when generating a single executable file. Notice how the compiler didn't rename the regex and replacement properties of the rule variable, but the variable itself is stored inside of the class as a. This is precisely the point of externs — to prevent the compiler from mangling properties that can come from outside code. Now, if we were compiling a library for use by other developers, and publishing it, we would want to prevent mangling optimisation, and then we would use externs. However, this optimisation only happens in the ADVANCED mode, where all comments with JSDoc is stripped, making the library hard-to use by others. But when we create a program and not a library, we can avoid using the externs, and pass the types just as a source file using the --js flag. This will still result in type-checking but also produce the optimisation of variable names (though in case of Node.JS programs the gain is minimal because the difference in size is not that significant, but for the web it might be helpful).

Externs As Types
#!/usr/bin/env node 'use strict'; const stream = require('stream'); const c = stream.Transform; class d extends c { constructor(a, b) { super(b); this.a = a; } _transform(a, b, f) { this.push(`${a}`.replace(this.a.b, this.a.c)); f(); } } ;const e = new d({b:/__(.+?)__/, c(a, b) { return `<em>${b}</em>`; }}); e.pipe(process.stdout); e.end("__hello world__");
The new command is depack example/restream/program -c -a -p --js example/restream/externs.js and it produces correctly optimised code.

And so that's it! We've successfully compiled our Node.JS program with Google Closure Compiler using Depack as the CLI interface, and Typal as the utility to organise types, both for README documentation, JSDoc annotation and Compiler externs information. There is just one last thing to add.

Annotating Types

import { Restream } from 'restream' /** * The rule to enable `<em>` tag conversion from Markdown. * @type {_restream.Rule} */ const rule = { regex: /__(.+?)__/, replacement(match, s) { return `<em>${s}</em>` }, } const restream = new Restream(rule) restream.pipe(process.stdout) restream.end('__hello world__') /** * @suppress {nonStandardJsDocs} * @typedef {import('restream').Rule} _restream.Rule */

When writing code that imports types from libraries, we can use the {import('lib').Type} notation for VSCode to give us auto-completions, but we need to suppress it. However, because now we're naming imported types with the namespace, Closure will pick them up from externs if it finds it. Packages can publish their externs and point to them using the externs field in their package.json file, which will be read by Depack and passed to GCC in the --externs flag.