Sublime Forum

Paramaterizing syntax files for similar (but not identical) syntaxes?

#1

I’m writing a syntax definition for a tool called ribosome. It’s a code generator that basically adds syntax to turn Ruby, Python, or JavaScript into a preprocessor.

These three variants of Ribosome differ only in the source language, and by extension the usage of source contexts (source.python vs. source.js vs. source.ruby).

However, I don’t want to maintain three files whose contents are more or less the same; I’d rather put the bulk of the file into a hidden syntax definition that is parameterized with something like this:

include: "Packages/Ribosome/Ribosome-impl.sublime-syntax"
  args:
    lang: "ruby"

Is it possible to do that?

0 Likes
#2

It is! YAML Macros 3.0 includes a new arguments feature that should do the trick. Unfortunately, the documentation for the new features is slated for the 3.1 release, but I’d be more than happy to help.

Here’s a quick-and-dirty example implementation:

%YAML 1.2
%TAG ! tag:yaml-macros:YAMLMacros.lib.arguments:
---
name: !format 'Ribosome - {language}'
scope: !format 'source.{extension}.dna'
file_extensions:
  - !format '{extension}.dna'

contexts:
  main:
    - match: ''
      push:
        - include: !format 'scope:source.{extension}'
      with_prototype:
        - include: dna

  dna:
    - match: ^\.
      scope: punctuation.definition.string.dna
      push:
        - clear_scopes: true
        - meta_scope: string.unquoted.dna
        - match: $
          pop: true

You’d pass arguments like language: "JavaScript", extension: "js".

0 Likes
#3

Hot damn. I’ll give this a try immediately and report back. Thanks!

0 Likes
#4

So, lemme see if I have this right. For each version of Ribosome, the YAML would be very small. How would I include the yaml-macros file and pass it arguments?

0 Likes
#5

(While writing this answer, I realized that I forgot to create a with macro. I’ve released YAML Macros 3.0.3 with the missing feature.)

0 Likes
#6

You’d have something like the following:

%YAML 1.2
%TAG ! tag:yaml-macros:YAMLMacros.lib.arguments,YAMLMacros.lib.include:
---
!with
  - language: JavaScript
    extension: js
  - !include_resource Packages/Ribosome/Ribosome-impl.sublime-syntax
1 Like
#8

It should be stripping the trailing underscore when importing the macro. Are you running version 3.0.3? I literally just added the with macro twenty minutes ago because I forgot it.

0 Likes
#9

Yep, I didn’t notice that post. Must’ve been blocked by something. It works beautifully now, thank you!

Related note; when it comes time to distribute this syntax, how can I do so such that the user doesn’t need to know or care about YAMLMacros?

0 Likes
#10

You can just use it as your internal build system. Sublime won’t do anything with the .yaml-macros files, but it will pick up the compiled .sublime-syntax files, so as long as you build your .yaml-macros source files before releasing, it doesn’t matter that the resultant .sublime-syntax files were compiled with YAML Macros. For examples, see the Vue Syntax Highlight and CFML packages.

A much more complicated example is JS Custom (not yet on Package Control), which dynamically recompiles syntaxes on the end user’s machine when their user settings change.

0 Likes
#11

So, just like I would with a traditional C program, my release cycle should be as follows:

  • Do some work
  • Compile the project on some kind of CI
  • Zip it up
  • Submit it to the relevant repos (in this case, the one for Sublime Text packages)
0 Likes
#12

Basically.

With YAML Macros, you do kind of have to build as you go so that you can test your changes. A complex syntax shouldn’t take more than a couple of seconds to build the first time, less on a fast machine. (Most of the parsing work is cached, which gives around a 50% speedup on subsequent builds.)

You could set up a more complex build system than simply pressing ctrl-b all the time, though I admit that I haven’t bothered myself.

1 Like
#13

Is there a way to install YAMLMacros with pip? I know there’s a CLI available, and that would really be useful in setting up a Travis build pipeline.

0 Likes
#14

At this time, it’s only available in the form of a sublime package. I might factor out most of the engine into a dependency at some point, but I don’t think that the Package Control dependency format is compatible with pip, either. It’s possible that a newer version of Package Control may make it easier to write a Sublime-compatible library that would work with other package managers.

0 Likes
#15

I almost have what I need to set up an automated build pipeline for this. Here’s where I stand:

On paper, I can apply these macros without running Sublime by running python3 cli.py < ~/ribosome-sublime/Ribosome-Python.sublime-syntax.yaml-macros, and with a properly-set PYTHONPATH. However, this is where I get stuck in practice; YAMLMacros can import sublime.py, but sublime.py can’t import sublime_api because it’s built in to the sublime_text executable.

So now I need to be able to run YAMLMacros/cli/cli.py using Sublime’s Python interpreter, but without using the GUI. Is that possible? I noticed that the executable has a --command flag. What does that do, and would it be useful to me?

0 Likes
#16

I’m not sure whether it’s possible to use the sublime interpreter in that fashion.

Currently, !include_resource only works within sublime. This is the result of an emergency patch because the general implementation was broken on Windows. Fixing that is a mid-priority issue; I probably won’t get to it this week.

However, I’m not sure that !include_resource would work anyway in the context of a CI system where the file may not be in the Packages directory of a Sublime install. For that, I’d use the regular !include macro. That should work just fine with the CLI. (It wouldn’t work if the target file is inside a zipped Sublime package, but this doesn’t matter unless you’re doing dynamic builds after the package is installed.)

0 Likes
#17

I can’t use the CLI because it needs the sublime_api module. So instead I’m trying to string together a bunch of subl -b --command <something> calls.

0 Likes
#18

You know what, I’ve decided to just use sed instead of YAML-Macros, as wrestling with it to use it for an automated build is becoming a big distraction. All the same, thank you for your help.

0 Likes