Add language support for GYB

[fpseverino]

Add syntax highlighting and language configuration for Swift GYB (.swift.gyb) files.

Motivation:

Resolves #1455

Language support for GYB can't be implemented via sourcekit-lsp, as GYB can contain arbitrary Python.

The TextMate grammar can't be an injection grammar because GYB files have a different file extension from normal Swift files, and also because sourcekit-lsp would activate and throw a lot of errors.

Additions:

• New swift-gyb language in package.json
• Language configuration for Swift GYB with code folding, brackets definition and autoclosing
• TextMate grammar for Swift GYB
• Icons for .swift.gyb files

Known issues:

• GYB code blocks (%{...}%) and control statements (% keyword: ... % end) are not always highlighted when they are inside Swift bodies (structs, extensions and funcs, in other words when not in the top level of the file) and do not have at least one level of indentation.
• Swift keywords are not always correctly highlighted when the preceding structure, function or variable name starts with a GYB block

[matthewbastien]

Thank you for putting this together, it's a great start! Unfortunately, I'm no expert when it comes to the language grammars in VS Code. Is there any way we could add tests for this at all?

CI is failing due to formatting issues. You'll want to run the following and push the changes:

npx prettier . --write

The unacceptable language filter also doesn't like the use of the word master in the tmLanguage file. However, since there's no way to get around using that schema, you can add syntaxes/swift-gyb.tmLanguage.json to the .unacceptablelanguageignore file.

[fpseverino]

Is there any way we could add tests for this at all?

Microsoft recommends using this package to test the grammar.
I will take a closer look at it and try to add tests ASAP.

[matthewbastien]

Oh and I just realized that you'll need to add the new language files to the .vscodeignore or they won't make it into the VSIX. We ignore everything by default so that we're explicit in what we include:

!swift-gyb.language-configuration.json
!syntaxes/**

[fpseverino]

I added some tests using the package I mentioned previously. To run them in my dev container I use the command:

./node_modules/.bin/vscode-tmgrammar-test 'test/unit-tests/**/*.test.swift.gyb' -g test/unit-tests/syntaxes/swift.tmLanguage.json -g test/unit-tests/syntaxes/MagicPython.tmLanguage.json

What do I have to do to make the tests run automatically?

[matthewbastien]

What do I have to do to make the tests run automatically?

You'll want to create a new script in the package.json that runs the grammar tests and then add that to the test script so that it gets run in CI. Something like:

{
  "scripts": {
    // ...
    "test": "vscode-test && npm run grammar-test",
    "grammar-test": "vscode-tmgrammar-test 'test/unit-tests/**/*.test.swift.gyb' -g test/unit-tests/syntaxes/swift.tmLanguage.json -g test/unit-tests/syntaxes/MagicPython.tmLanguage.json"
    // ...
  }
}