Looking for PackageDev alpha testers (.sublime-syntax highlighting and a lot more)

[FichteFoll]

Hello there,

in a recent push to finally have some sane syntax highlighting for .sublime-syntax files, I did all the required work and pushed it as a pre-release to the PackageDev package. Before releasing this to the public (I want to re-write all the other syntaxes, since they are severely lacking), I’d like to collect some feedback regarding this quite complex syntax definition.

To enable installation of pre-releases, open “Preferences: Package Control - User” via the command palette and add this:

    "install_prereleases": ["PackageDev"],

What I’m particularly interested in:

1. Does (?x) extended mode highlighting work as you would expect it to?
2. Do you see/use empty in your multi-line regular expressions and does the syntax definition handle this correctly?
3. Do you have a color scheme that highlights source source (like mine) and are you annoyed by some empty lines being highlighted as a result? (see first example screenshot)

---

Here’s a preview of the features:

[FichteFoll]

I just went through all .sublime-syntax files in the default packages and this is looking very good. Fixed a couple issues and pushed another pre-release to test.

What remains from the default syntaxes is:

1. \x{...} escapes. I’m not sure what format is accepted, but the doc states this:

   \x{7HHHHHHH} wide hexadecimal char (character code point value)

   and I see a couple violations of this (like \x{00} and \x{0000037D}).

2. '' escapes within single-quoted regexps like this: '''{{identifier}}(?!\'')\b'

3. “unexpected quantifiers” after an escape: '(?=<<<\s*''?({{identifier}})''?\s*$)'

4. “unexpected quantifiers” after a variable: '-?{{nmstart}}{{nmchar}}*'

I will look into these probably tomorrow since I gotta run now.

[deathaxe]

I checked PackageDev with my s840d_gcode.sublime-syntax. It looks really good right now. Good job.

I don’t use too complex extended regexes, but they look good for me.

I’m wondering only, if it was possible to highlight the following code as invalid

1. match: {{varname}}
2. match: [A-z]+

I ask because I struggled with those constructs several times in the past. YAML interprets them as mapping or sequence, but SublimeText rejects the syntax definition as it does not know about them.

To make syntax work it is important to quote them like this:

1. match: '{{varname}}'
2. match: '[A-z]+'

It’ sometimes not so easy to see at a glance, whether quoting is required or not.

[FichteFoll]

I’m wondering only, if it was possible to highlight the following code as invalid

Sure, no problem. I thought doing no matching at all would be sufficient, but then realized that I include normal YAML matching for the “contexts” block, which might not be indicative enough.

Would you prefer only matching the first character as invalid or the entire line? I think I’ll go with the line.

In other news, all the issues I mentioned in my previous post are now fixed.

[kingkeith]

Nice work!

Suggestions:

where there are lines like:

- include: Packages/YAML/YAML.sublime-syntax#comment
- include: scope:source.regexp.oniguruma#base-group-extended

it would be great if it scoped the following separately:

• scope and :
• # and the context name following it

so that I could highlight them differently in my color scheme.

Actually, it looks like the latter is partially covered as I see the following in the syntax definition, but it doesn’t seem to be working where I’d expect:

- match: '#'
  scope: punctuation.separator.context-name.sublime-syntax

---

The base scope: source.yaml.sublime-syntax is just source.yaml string.unquoted.yaml, I would expect source.yaml meta.contexts debug.scope.inline keyword.operator.accessor.syntax on the . chars

file_extensions: accepts an array, but it doesn’t scope it atm

clear_scopes isn’t always scoped correctly - check the ASP syntax in the Default packages

Nitpick: punctuation.separator.key-value should have a .yaml or .sublime-syntax ending

[FichteFoll]

it would be great if it scoped the following separately:

• scope and :

• and the context name following it

so that I could highlight them differently in my color scheme.

Both should receive a punctuation scope, but it seems I forgot the one for :. I intentionally scoped : the same as scope however because it is part of the prefix, sort of.
# is scoped fine though, as far as I can see. Can you provide an example for where it doesn’t work for you?

The base scope: source.yaml.sublime-syntax is just source.yaml string.unquoted.yaml, I would expect source.yaml meta.contexts debug.scope.inline keyword.operator.accessor.syntax on the . chars

Please elaborate on this, I don’t understand.

file_extensions: accepts an array, but it doesn’t scope it atm

clear_scopes isn’t always scoped correctly - check the ASP syntax in the Default packages

I include the default YAML syntax definition for these and it works fine for me. Do you perhaps have it overridden?

[kingkeith]

ah I had cheated and just used your syntax definition directly without installing PackageDev - installing the pre-release version fixes all of these issues. Sorry! I just need to tweak my color scheme quite a bit now…

[idleberg]

Not my pattern, but I don’t think it’s invalid

Also, conversion no longer works for me since upgrading to the prerelease. It fails to detect the file type, even for syntax as simple as this.

Input type not specified, auto-detecting...
Unable to detect file type.
```

[FichteFoll]

I’m pretty sure that highlighting is from bracket highlighter since the syntax definition wouldn’t do a match like this and the highlighted section is also round, indicating it was added with view.add_regions.

I’ll check the other issue when I’m at home.

[idleberg]

I’m pretty sure that highlighting is from bracket highlighter

That was my first thought, so I disabled it. However, I will double check as well when I get home

[FichteFoll]

Works fine for me, even with BracketHighlighter:

[FichteFoll]

Also, conversion no longer works for me since upgrading to the prerelease. It fails to detect the file type, even for syntax as simple as this.

Not sure what you’re expecting there. .sublime-syntax files cannot be converted to anything else, since .tmLanguage only describes a subset of what .sublime-syntax can do.

So yeah, in theory you could make json and plist out of it, but nothing uses that in combination with the schema.

[deathaxe]

Would you prefer only matching the first character as invalid or the entire line? I think I’ll go with the line.

I think highlighting the whole line is more obvious as the whole expression is invalid due to missing quotation.

[idleberg]

Not sure what you’re expecting there. .sublime-syntax files cannot be converted to anything else

I use it every now and then to figure out the differences between the old and the new syntaxes. But yeah, in my experience it doesn’t work most of the times. Anyway, I only wanted to let you know about the broken file type detection.

[idleberg]

I’m pretty sure that highlighting is from bracket highlighter

You were right, after disabling the package and restarting Sublime Text, it went away

[facelessuser]

BracketHighlighter shouldn’t just be highlighting one random bracket unless it is highlighting it as an unmatched bracket. I would double check the rule for that syntax and double check the actual code to make sure there isn’t an unmatched bracket problem. Though language rules are generally supported by the community, I wouldn’t mine looking into this one if there is a problem with BracketHighighter (and not just a local issue you are having).

[Landis]

To enable installation of pre-releases, open “Preferences: Package Control - User” via the command palette and add this:

"install_prereleases": ["PackageDev"],

Does Nothing for me.
I copied & pasted, literally:
“install_prereleases”: [“PackageDev”],
in Preferences > Package Control popup ‘Package Control:’ input line.
Nothing Happens. Why?

Landis

[Landis]

I got it…
Add to the Package Control > SETTINGS-USER text file. duh…

Landis

[315234]

Looks brilliant, thanks.

[gwenzek]

Hi FitchFoll, thanks for working on that.

I see you choose to not provide support for “\\” regexes like me, it’s indeed a bit tedious to write, and it will encourage syntax writer to uses the cleaner simple quote syntax ‘\’.

Comparing to my syntax,
I feel sad that you didn’t provide standard scopes for context name.
You put an “entity.name.context” on it meaning it would probably looks like crap in most color scheme.
Same thing for the “entity.name.constant”. I personally used “entity.name.class” and “entity.name.function”.
I know it’s the direction Wbond took by introducing “entity.name.struct” and others, but I think this hurts ST in the long term.

Anyway I think you could use more different scopes for regexes. Most of the thing are highlighted as “keyword.control” and “keyword.operator” which have often the same color. Plus I don’t see why ‘\d’ is a ‘keyword.control’ and not a ‘constant.language’.
But I like how ‘meta-scope’, ‘match’ and ‘push’ are scoped differently.
I think you should try your syntax with the various Default color scheme to see how it looks like.
Ideally one should not have to fine tune it’s color scheme for each language he uses.

Regarding the multi-line support I can’t say much since my syntax don’t usually rely on those.

So apart for this scope knit-picking, I think it’s a good job, and splitting the Onigurama regex into it’s own file will probably prove useful.