Sublime Forum

YAML Macros: Easy preprocessing for syntax definitions

#1

Why do I want YAML Macros?

Syntax definitions can have a lot of boilerplate where the same structure is repeated with slight variations. For example, in SQL, keywords are always case-insensitive and surrounded by word breaks:

- match: \b(?i:select)\b
  scope: keyword.other.select.sql

- match: \b(?i:from)\b
  scope: keyword.other.from.sql

- match: \b(?i:distinct|all)\b
  scope: keyword.operator.word.sql

This can be tedious to write, and it’s easy to make minor errors that are hard to spot. With YAML Macros, you can abstract this behavior away by writing a simple Python function:

def word(str):
    return r'(?:\b(?i:%s)\b)' % str

Then use this macro in your syntax definition:

- match: !word select
  scope: keyword.other.select.sql

- match: !word from
  scope: keyword.other.from.sql

- match: !word distinct|all
  scope: keyword.operator.word.sql

How does it work?

You write a Python script with your desired macros, then use those macros in a file with the .yaml-macros extension. The YAML Macros build system will apply your macros to the file and save it without the additional extension.

To indicate the location of the Python macros file, use the %TAG directive:

%YAML 1.2
%TAG ! tag:yaml-macros:sql_macros/
---
name: SQL (YAML Macros example)
…

The syntax used to invoke the macros is an obscure YAML feature called tags. It is mainly used to mark types for serialization purposes, but we have repurposed it here to denote macro invocations. This is perfectly kosher YAML syntax; we just provide PyYAML a “constructor” for each macro.

Sublime will automatically recognize these .yaml-macros files as YAML if they have a version declaration. In addition, the tags are gracefully handled by PackageDev’s specialized sublime-syntax syntax.

Can I see a practical example?

Sure! The repository includes a basic SQL syntax example. That syntax only recognizes very simple queries, but it illustrates how YAML Macros can be used in practice to factor out a lot of the complexity of creating a syntax.


This is an alpha release. I’m looking for feedback on the functionality and interface, as well as bug reports. It’s not in Package Control yet, but I intend to submit it later this week.

5 Likes

#2

I haven’t looked at the source yet, but I suggest you add/have a cli for this project in addition to the st package, because it will allow people to invoke it without st. A couple people requested that for package dev’s yaml conversion as well, but I really didn’t have this in mind back when I wrote it, resulting in the code being tied heavily to ST’s api and making it quite annoying to decouple. I’m not saying it’s impossible, but I still don’t see myself working on that soon.

Edit: well okay, that is surprisingly little code. Just consider my comment as a suggestion.

2 Likes

#3

Good idea.

0 Likes

#4

Oh, and since I saw you added a pr to the default channel just now, you’ll need to specify the dependency you rely on (pyyaml).

0 Likes

#5

Version 1.0.0 is out and should be available via Package Control.

0 Likes