Sublime Forum

Extending a YAML syntax to highlight specific words

#1

I am a casual user of Sublime Text, and this is my first dive into changing syntaxes, and then customizing the colour scheme with a new scope.

So please forgive what I expect are newbie mistakes. I am also happy to be pointed to documentation that explains what I did wrong.

I work in visual effects, and we use an open source colour management tool, designed for VFX. Opencolor OCIO.

I only provide the below URL for interest, not because I expect you to need to go there.
https://opencolorio.readthedocs.io/en/latest/index.html

The configuration files are written in YAML, using a small set of the features. (I am also unfamiliar with YAML…)

One feature it uses is Verbatim Tags, as part of the data definition. ( https://yaml.org/spec/1.2-old/spec.html#tag/property/ )

These tags are used by the interpreter to extract the data from the file.

The standard YAML syntax highlighter highlights those tags perfectly well.

But I was hoping to extend the YAML definition a bit, to enable highlighting those specific Verbatim Tags. That would be based on the actual word inside the tag delimeters.

The goal being to give a specific highlighting to verbatim tags that consist of one of the words in the list, leaving other tags and verbatim tags alone.

Here’s a screenshot of the file with YAML syntax highlighted.

I have not been able to get the extended syntax to work yet. When I inspect the relevant colored text using Alt+Ctrl+Shift+P, the scope information seems to be untouched from the YAML syntax.

image

Below this point, I have added chunks of my code, code from the ST4 YAML syntax file, and a color scheme alteration.

I really apologize for the length of this. I’m trying to be thorough, and show the steps I took and why. I fully expect I have missed some critical piece of information. I confess that these kinds of files are in a very different area than I have paddled in before, so it has been confusing…

Here is the contents of my syntax file, called OCIO.sublime-syntax

%YAML 1.2

---
name: OpenColor OCIO
scope: source.yaml
extends: Packages/YAML/YAML.sublime-syntax
version: 2

file_extensions:
  - ocio

variables:
  # List of tags OCIO uses to identify the data types for the interpreter. The divider is a "pipe" symbol, which in regex is equivalent to "OR"
  ocio-tag-names: 'Rule|View|Views|Look|BuiltinTransform|ViewTransform|ColorSpace|GroupTransform|MatrixTransform|LogCameraTransform|ExponentWithLinearTransform|ExponentTransform|NamedTransform|LogAffineTransform|RangeTransform|FileTransform'
  #Assemble the tag names with the surrounding !<> and set up the expression to identify whole words
  ocio-verbatim-tag: '!<\b(ocio-tag-names)\b>'

#Looking in the Sublime Text YAML syntax file, it appears to be the property-body section that contains the matching for the verbatim tags.
contexts:
  property-body:
    - meta_prepend: true
    # !OCIO Tag Handle
    # http://yaml.org/spec/1.2/spec.html#tag/property/
    - match: 'ocio-verbatim-tag'
      scope: storage.type.tag-ocio.ocio
      pop: 1

I extracted the YAML syntax definition file in order to find the relevant sections where it was matching the verbatim tags.

From the YAML syntax definition I copied and altered this variable (lines 70-75):

c_ns_tag_property: |- # c-verbatim-tag | c-ns-shorthand-tag | c-non-specific-tag
    (?x:
        ! < {{ns_uri_char}}+ >
      | {{c_tag_handle}} {{ns_tag_char}}+
      | !
    )

However, as you can see, I altered it almost completely. I looked at the Oniguruma RegEx version Sublime Text uses, and I do not know what the {{ }} are doing. It appears to be a Oniguruma RegEx function, but I do not know what “n times continuations” etc means. So I got rid of them.

The section in the YAML syntax file I found and plundered is at lines 372-394. This is in the contexts: section.

contexts:
.
.
.
.  
  property:
    # http://yaml.org/spec/1.2/spec.html#node/property/
    - match: (?=!|&)
      push: property-body

  property-body:
    - meta_scope: meta.property.yaml
    # &Anchor
    # http://yaml.org/spec/1.2/spec.html#&%20anchor//
    - match: (&)({{ns_anchor_name}})(\S+)?
      captures:
        1: keyword.control.property.anchor.yaml punctuation.definition.anchor.yaml
        2: entity.name.other.anchor.yaml
        3: invalid.illegal.character.anchor.yaml
      pop: 1
    # !Tag Handle
    # http://yaml.org/spec/1.2/spec.html#tag/property/
    - match: '{{c_ns_tag_property}}(?=\ |\t|$)'
      scope: storage.type.tag-handle.yaml
      pop: 1
    - match: \S+
      scope: invalid.illegal.tag-handle.yaml
      pop: 1

In my extender file, I added a property-body context, with a meta-prepend: true

And then added a match and scope that I thought were valid.

The reason I did the prepend was that if there is a match for a verbatim tag with that exact name, then its done. It pops the match and scope out and doesn’t need to do the rest of the matches.

I also created a color-scheme variation using the new scope I created.

{
	"variables":
	{
		"ociopink": "hsl(300, 60%, 80%)"
	},
	"globals":
	{
	},
	"rules":
	[
		{
            "name": "OCIO Tag",
            "scope": "storage.type.tag-ocio",
            "foreground": "var(ociopink)",
            "font_style": "bold italic"
        }
		
	]
}

Thank you for any insight anyone may have.

Cheers

0 Likes

#2

This should be

ocio-verbatim-tag: '!<\b({{ocio-tag-names}})\b>'

Because to reference the variable you created it needs to be enclosed in double curly brackets, this is how the ST syntax engine resolves variable references, otherwise it matches the literal ocio-tag-names

0 Likes

#3

Ah. I took it as part of the Regex. Oops.

Sadly though, that didn’t work.

And in case it was failing to do a double substitution (ocio-tag-names into ocio-verbatim-tag), I did one substitution in the context area.

 - match: '(?:!<\b({{ocio-tag-names}})\b>)'
       scope: storage.type.tag-ocio.ocio
       pop: 1

But that didn’t work either.

So I did the full substitution of both variables into the match:

    - match: '(?:!<\b(Rule|View|Views|Look|BuiltinTransform|ViewTransform|ColorSpace|GroupTransform|MatrixTransform|LogCameraTransform|ExponentWithLinearTransform|ExponentTransform|NamedTransform|LogAffineTransform|RangeTransform|FileTransform)\b>)'
      scope: storage.type.tag-ocio.ocio
      pop: 1

And that DOES work.

image

I have no idea what to make of the failure to do the variable substitution.

0 Likes

#4

I have a memory that variables cannot contain dashes - try naming it with underscores instead.
If you have PackageDev installed it should highlight it differently

0 Likes

#5

Bingo. I replaced dashes with underscores, and it worked!

That should really be in the documentation.

I did try installing PackageDev, but I can’t see any way of assigning its syntax checking to an already existing syntax file.

—major edit----

The relevant syntax definitions from PackageDev are in a sub directory. View>Syntax>PackageDev>…

0 Likes

#6

Once you install it, it should automatically apply its own syntax when you open a syntax file; you don’t need to do it manually.

0 Likes